Preserve independent MM-group calls and support larger code sets

[SuhasSrinivasan]

Review status: Author-reviewed and ready for ONT review.

Addresses #675 and the MM-code cardinality portion of #676.

Summary

This PR fixes two related limitations in the optimized pileup MM-tag scanner:

• Same-base MM groups now advance independently, so one group reaching the selected query position cannot prematurely advance another group that only counted down to zero there.
• The fixed 16-code arrays are replaced by SmallVec<[ModCodeState; 16]>, preserving inline storage for the existing common case while accepting valid tags with more than 16 code states.

The pileup motif-mask limit previously included in this PR has been removed and consolidated into PR #702 with the other motif ownership and validation changes.

Severity

High — scientific correctness and reliable completion.

The progression defect can silently lose or misassign accepted modification calls while pileup exits successfully. The fixed-capacity representation can panic on a valid MM tag containing more than 16 code states.

Root cause and fix

The optimized scanner shared one sequence cursor across MM groups. A group could become zero while a different group selected the current position; the old emission and advancement loops then treated both groups as ready.

Each ModCodeState now has an explicit_at_position flag. A scan clears the existing flags, marks only the states explicitly ready at the selected position, and advances only those states. Keeping the flag in the state uses existing struct padding and requires no per-position allocation.

The same state consolidation removes the 16-slot assertion. Up to 16 states remain inline and allocation-free. A 17th state spills once during adapter construction; later growth is amortized. The hot-loop complexity remains linear in read length and active code-state count.

Reproduction and expected behavior

For a two-base CC read with MM:C+m?,0;C+h?,1; and ML:[200,250]:

• Before: optimized pileup reports only q0/h, losing q0/m and q1/h.
• After: optimized and generic pileup both report q0/m and q1/h with the corresponding probabilities and exact count categories.

For valid tags with 17 code states:

• Before: the optimized adapter panics at its fixed-capacity assertion.
• After: construction succeeds and forward/reverse MM and ML cursors retain the correct query position, code identity, and probability.

Testing

Tested exact head: 9fb9aea763aa1b78ac1172fa6d6734e7955ccc70.

• Parent-red CLI regression: the new independent-group integration test fails on exact upstream parent 5cecc3f with only q0/h, then passes on this head with exactly q0/m and q1/h.
• cargo test --offline --locked -p mod_kit base_mods_adapter_tests --lib -- --nocapture: 13 passed, 0 failed. This includes forward/reverse cursors, ML stride, one multi-code group, many one-code groups, ChEBI codes, 16-inline/17-spilled states, and independent same-base progression.
• cargo test --offline --locked -p modkit --test test_pileup -- --nocapture: 17 passed, 10 ignored, 0 failed.
• cargo test --offline --locked --workspace --all-targets --no-fail-fast -- --test-threads=1: 185 passed, 14 ignored, 0 failed.
• Changed-file rustfmt --check and git diff --check: passed. Whole-workspace formatting still reports unrelated pre-existing formatting differences in untouched files.
• CPU regression check on a 898,198,413-byte direct-RNA BAM (1,632,831 reads, four code states): upstream median 6.87 s; scanner carrier median 6.90 s (+0.44%). All six outputs were byte-identical: 60,959 rows, 9,989,028 bytes, SHA-256 a85202b69e79e708656e71858e4243cb1cc51e22a44bb9f65650a23dacb407fa. The scanner production source is byte-identical between that benchmarked carrier and this head. This supports no material regression for the normal inline path; it is not presented as a benchmark of the >16 spilled path.
• Independent Rust, bioinformatics, CPU/performance, and supervisor reviews: approved the exact head without blockers.

Output and compatibility

• Output changes only for records affected by independent same-base MM progression or the former 16-state limit.
• CLI options, bedMethyl schema, thresholding, ordering, and threading are unchanged.
• Adds direct dependency smallvec; the version already resolves transitively in the tested dependency set.

Non-goals

• Pileup motif ownership, combine-strand validation, and the eight-bit motif-mask limit are handled together in PR Correct pileup motif ownership, validation, and mask limits #702.
• No ambiguity/IUPAC policy change, generic-worker redesign, scheduler/I/O redesign, malformed-tag recovery, or GPU change.

Reviewer guide

1. Review the two-base red/green integration oracle in modkit/tests/test_pileup.rs.
2. Review explicit_at_position and the advance condition in base_mods_adapter.rs.
3. Review the fixed-array-to-SmallVec<ModCodeState> conversion, especially forward/reverse MM and ML cursor movement.

Author checklist

• The PR body contains the observed and expected behavior without requiring the linked issues.
• The change is restricted to the MM scanner and its regression tests.
• The regression is demonstrably red on the exact parent and green on the final head.
• Focused, full-workspace, formatting, scientific-output, and performance checks are listed above.
• The exact final head was independently reviewed before marking this PR ready.