Correct pileup record state and count accounting

[SuhasSrinivasan]

Review status: Author-reviewed and ready for ONT review.

Summary

This PR consolidates two related pileup correctness patches that both affect per-record state and bedMethyl count accounting:

• Clear cached generic-pileup modification state when an MM iterator is exhausted.
• Correct combined and dynamic count-slot accounting, reverse anchoring, overlapping optimized masks, and duplicate --modified-bases selections.

The resulting history retains the seven focused count-accounting commits followed by one focused generic-state commit.

Severity

High — scientific correctness

Affected commands can exit successfully while reporting incorrect modified/canonical counts, duplicated output rows, omitted combined-C modifications, or incorrectly anchored reverse-strand status counts.

Reproduced behavior and expected results

Accepted input	Previous behavior	Correct behavior
Modified record followed by an empty-MM record	The second record can inherit the preceding modification	Coverage 2 partitions exactly into 1 modified and 1 canonical call
Combined C calls containing multiple C modification codes	Some admitted C modifications are omitted or misclassified	Coverage equals canonical + modified + other-modified counts
Compact C-only or A/C/T combined slots	Normal and high-depth paths can disagree	Exact byte-identical rows across normal/high-depth and thread counts
Same raw modification code selected for different canonical bases	Calls can share the wrong dynamic slot	Slots are keyed by canonical base and modification code
Reverse non-CpG combined output	Ancillary statuses can use inconsistent anchors	All statuses share the correct reverse motif anchor
Duplicate modified-base selections	Duplicate rows or inconsistent preset/writer state	Exact duplicates are normalized once

Root causes

• update_mods_iter2 left four cached fields unchanged when its iterator returned None.
• Combined-C accounting recognized too narrow a set of modification codes and did not consistently preserve filtered counts.
• Compact and explicit dynamic offsets did not consistently include canonical-base identity.
• Reverse ancillary categories did not all apply the same motif-coordinate transformation.
• An optimized debug assertion rejected valid overlapping internal masks.
• Exact duplicate modified-base selections were propagated to multiple downstream consumers.

Implementation

• Clear mod_pos, canonical_base, pos_base_mod_call, and mod_strand on iterator exhaustion.
• Count every admitted combined-C modification and saturate filtered counts.
• Key explicit dynamic slots by (canonical base, modification code).
• Use a common reverse anchor calculation for all dynamic status categories.
• Retain deterministic first-hit routing for valid overlapping masks.
• Stably deduplicate exact modified-base selections before preset and writer construction.
• Merge both branches' processor tests into one nonduplicated test module.

Preserved behavior and non-goals

• No output schema or CLI option changes.
• No threshold, scheduler, I/O, MM/ML parser, or performance redesign.
• Existing behavior is unchanged for unaffected records and count categories.
• Motif interval ownership and MM-group cardinality changes remain in separate review units.

Testing

Exact final revision: 18b3ff23a3ecb146611df2679a59682ce09d5430
Base: 5cecc3fb3a9336068d9e3c68d5c08d678153dd2c
Platform: macOS on Apple silicon
Dependency resolution: ignored Cargo.lock SHA-256 78876ab4a98da30caad167744d1c8a0875c27edad7b0b3ad5f8a1891f78604ea; hts-sys 2.2.0

Test layer	Command or evidence	Result
Generic state regression	cargo test --offline --locked -p mod_kit generic_pileup_clears_mod_state_between_records --lib -- --test-threads=1	1 passed
Processor count/state module	cargo test --offline --locked -p mod_kit pileup::pileup_processor::tests --lib -- --test-threads=1	5 passed
Pileup CLI regressions	cargo test --offline --locked -p modkit --test test_pileup -- --test-threads=1	22 passed, 10 ignored
Full workspace gate	cargo test --offline --locked --workspace --all-targets --no-fail-fast -- --test-threads=1	190 passed, 14 ignored, 0 failed
Formatting and diff hygiene	Stable rustfmt --check on all three changed files; git diff --check upstream/master...HEAD	Passed
Patch preservation	Stable patch IDs and tree comparison against the pre-consolidation carrier	All seven original commits preserved exactly
Independent review	Rust, bioinformatics, and supervisor reviews of the exact consolidated tree	No blockers

cargo clippy -D warnings is not currently a usable repository-wide gate because the unchanged baseline first fails in safe-record and contains hundreds of existing warnings. No new production-line Clippy diagnostic was identified in this diff.

Scientific validation

• Count conservation holds for every new exact oracle.
• The generic two-record case is exactly 1 modified + 1 canonical at coverage 2.
• Combined-C unit coverage is exactly 5 = 1 canonical + 4 modified.
• Normal, dynamic, and high-depth rows agree for compact A/C/T fixtures.
• Duplicate selections remain byte-idempotent.
• Reverse ancillary categories retain the expected coordinate, strand, coverage, and category counts.

Reviewer guide

1. Review the final update_mods_iter2 exhausted-iterator branch.
2. Review the combined-C and dynamic-slot offset helpers in pileup_processor.rs.
3. Review reverse anchoring and duplicate-selection normalization.
4. Run the processor module and test_pileup commands above.
5. Confirm the eight commits remain individually understandable and reversible.

Author checklist

• Every included bug has an accepted-input regression.
• Regressions were demonstrated red on their relevant parent revisions and green after their fixes.
• The consolidated diff contains no unrelated ancestry or files.
• Rust and scientific reviews found no blocker.
• The exact final head passed the complete workspace test gate.
• All performed and inapplicable checks are disclosed.