Skip to content

[COMPRESS-726] Count returned bytes in framed LZ4 and Snappy streams - #825

Open
agamal026 wants to merge 1 commit into
apache:masterfrom
agamal026:fix/framed-uncompressed-count
Open

agamal026 wants to merge 1 commit into
apache:masterfrom
agamal026:fix/framed-uncompressed-count

Conversation

@agamal026

@agamal026 agamal026 commented Oct 18, 2026 •

Copy link
Copy Markdown

Problem

The framed LZ4 and Snappy readers inherit getUncompressedCount() from CompressorInputStream, where it reflects their historical input/framing counter instead of decompressed output. A valid empty frame already reports 15 bytes for LZ4 or 10 for Snappy before returning any output.

This addresses the counter finding in COMPRESS-726 comment 18092163. The size-limit proposal in #777 explicitly leaves this correction separate; this PR does not implement or close that broader feature.

Change

Each framed reader keeps a separate output counter and advances it once when a public read successfully returns bytes, after the existing checksum and block-dependency operations. The inherited single-byte and read-based skip paths use the same accounting.

The tests cover empty streams, zero-length reads, single-byte and offset bulk reads, skipping, compressed and raw blocks, multiple chunks, concatenated frames, skippable metadata, and retaining prior output after a later framing failure. Existing compressed-byte accounting and decoder behavior are preserved.

Validation

Source: 7e6477880259d60d4dfa60a5e34e4ea7875176b1, based on upstream master at 611cad06b66842692870c6ae98f50de7ce4f25d5.

The hosted jobs use the unchanged upstream command, with no explicit goal or added test exclusions:

mvn --errors --show-version --batch-mode --no-transfer-progress

RAT, Checkstyle, PMD and japicmp passed throughout. Javadoc passed on Java 11 and later; the existing Java 8 profile omits that goal. The owned test merge differs from this contribution only by removing two inherited weekly workflow schedules; all source, test and Maven build inputs match.

The earlier local default build stopped on two TAR tests requiring nonzero fixture owner/group IDs. Both also fail on exact unchanged upstream source in the same UID/GID 0 environment; the linked local packet preserves that control. The hosted full builds resolve this validation limitation.

Reproduce the focused regression

Run the command below in clean checkouts of the test-only baseline and the submitted fix:

mvn --batch-mode --no-transfer-progress --errors --show-version \
  -Dtest=FramedCompressorInputStreamStatisticsTest,FramedLZ4CompressorInputStreamTest,FramedLZ4CompressorRoundtripTest,FramedSnappyCompressorInputStreamTest \
  test

The recorded baseline result is 89 tests, 39 counter-assertion failures, zero errors or skips. The same four test files with the fix produce 89 passing tests. The baseline preserves upstream production and Maven configuration; the public verification records above identify the exact source and test hashes.

The hosted validation used merge 76d0130283eb142314cf11bb42825798e9279a05. Its comparison with the submitted head contains only the two inherited schedule removals described above.

Upstream CI status, checked 22 October 2026: Java CI, CodeQL, and Dependency Review report action_required with no jobs. The successful 19-job Maven matrix linked above is owned-fork validation; these upstream workflows have not executed.

Contribution checklist and AI disclosure

  • Read the contribution guidelines.
  • Read the ASF Generative Tooling Guidance.
  • AI was used: ChatGPT (GPT-6 Astra Pro) generated the implementation, tests, analysis and PR text. A separate automated source review is included in the local evidence packet; no personal human review is claimed.
  • Ran the default Maven build successfully.
  • Added behavioral regressions that fail without the runtime correction.
  • Described the problem, change and resulting behavior.
  • Kept a single focused commit with a Jira-linked subject and explanatory body.

Track decompressed output separately from the historical framing/input
counters. Advance the new counters once per successful public read, after
existing checksum and block-dependency work.

Add focused counter regressions and reuse raw-block, multiblock and IWORK
fixtures. The unchanged-source baseline has 39 counter failures in 89 tests;
the same selected suite passes with this correction.

This follows the counter finding in COMPRESS-726 comment 18092163 and does
not implement the decompression-limit feature tracked by that ticket.

Generated-by: ChatGPT (GPT-6 Astra Pro)
@agamal026

Copy link
Copy Markdown
Author

The three upstream workflow runs are awaiting maintainer approval. Could a maintainer authorize them when ready? The unchanged default Maven matrix passed all 19 jobs on the owned fork, and the PR description links the complete before-and-after evidence and job logs. The submitted source is 7e64778; no CI configuration changes are included in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant