Skip to content

Streaming: startChunkedConversion / Stream.transform #19

Description

@dipu-bd

Today there is no chunked conversion — the base Converter inherits the SDK's startChunkedConversion, which throws UnsupportedError, so codec pipelines over files/sockets don't work with Stream.transform.

Architecture fit

  • Add per-codec chunked sinks that carry the bit-accumulator state across chunks, as an additive path beside the fast whole-buffer convert() — the whole-buffer fast path must not regress (differential + benchmark proof).
  • Start with the bit-regrouping codecs (base16/32/64) where the accumulator is the generic BitEncoder/BitDecoder state.

Effort: M–L

Checklist

  • startChunkedConversion overrides carrying accumulator state
  • Tests: chunk-boundary splits at every offset produce identical output
  • Differential vs whole-buffer convert() — byte-identical
  • Benchmark: whole-buffer path shows no regression (numbers reported)
  • README / CHANGELOG / example updated
  • Format / analyze / test (vm + node) green

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions