Skip to content

test(cli): every row kind arrives in any order - #774

Merged
blafourcade merged 1 commit into
nextfrom
test/every-row-kind-arrives-in-any-order
Sep 4, 2026
Merged

test(cli): every row kind arrives in any order#774
blafourcade merged 1 commit into
nextfrom
test/every-row-kind-arrives-in-any-order

Conversation

@blafourcade

Copy link
Copy Markdown
Contributor

Why

A re-read appends, so one session's lines sit in different orders on two machines and nothing a consumer does controls it. The report is meant to be blind to that, and cost-report.unit.test.ts already checks it — with four records and a single reversal.

Four records cannot cover what has been added since. Two axes now mix key kinds inside one Map, which is exactly where insertion order leaks into output:

Axis Row Key
by_flow the journal witnessed it the FlowInterval object
by_flow only the tool named it the skill's name, a string
by_flow joined neither a symbol
by_agent the tool named the agent the agent's name
by_agent the main thread a symbol
by_agent the tool never names one a second symbol

Rows are ranked by size and ties broken on the row's own key. A tie-break that forgets part of the key, or a survivor picked as group[0], answers differently for the same records in a different order — and nothing failed when either was broken.

Verified against the real thing first

30,222 records of a live sink, shuffled within every day file with a seeded shuffle, produced a byte-identical report to the unshuffled run (md5 equal). Three consecutive unshuffled runs were byte-identical too, and all eleven --axis artefacts stable across runs. The property holds at scale; this is the guard that keeps it.

The test

fast-check over 300 permutations of a fixture carrying every row kind once: both flow kinds and the remainder, all three agent attributions, a named prompt and one that named none, two tools, a pair with identical figures only a tie-break can order, and two records sharing one billed_request_id so pickDeterministically is actually reached.

A second test asserts the fixture exercises what the property is about — a permutation of records that produce one kind of row proves nothing about mixed keys.

Guard Mutation that killed it
the agent tie-break keeps the row's key return "" — 1
the flow tie-break keeps the row's key return "" — 1
a superseded group's survivor does not depend on arrival pickDeterministically returns candidates[0] — 1

The third survived the first version of this fixture, which had no two records sharing a billed_request_id — the function was never reached, so the guard was decorative. Adding the pair is what made it real, and the coverage test is what stops that regressing quietly.

Gates: 3456 tests / 309 files, tsc, biome ci (2 pre-existing warnings, 0 errors), knip, jscpd, bundle within budget, 0 broken links.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp

A re-read appends, so one session's lines sit in different orders on two
machines. The report is meant to be blind to that, and the existing check is
four records and a single reversal.

Four records cannot cover what has been added since. `by_flow` now keys an
interval-derived row on a `FlowInterval` object, a tool-stated one on the skill's
name, and the remainder on a symbol; `by_agent` keys two of its three rows on
symbols. Mixed key kinds in one Map are exactly where insertion order leaks into
output, since rows are ranked by size and ties broken on the row's own key —
and nothing failed when either tie-break was broken.

Verified against the real thing before writing it: 30,222 records of a live
sink, shuffled within every day file, produced a byte-identical report to the
unshuffled run. Three consecutive runs were byte-identical, and all eleven
`--axis` artefacts stable.

fast-check over 300 permutations of a fixture carrying every row kind once, plus
a second test asserting the fixture exercises all of them — a permutation of
records that produce one kind of row proves nothing about mixed keys.

The `pickDeterministically` mutation survived the first fixture, which had no two
records sharing a `billed_request_id`, so the function was never reached. Adding
the pair is what made that guard real.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp
@blafourcade
blafourcade requested a review from a team as a code owner September 4, 2026 23:46
@blafourcade
blafourcade merged commit a6f7412 into next Sep 4, 2026
17 checks passed
@blafourcade
blafourcade deleted the test/every-row-kind-arrives-in-any-order branch September 6, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant