Skip to content

test(claims): expand the golden behavioral contract to GOLDEN-19 - #179

Merged
Aparnap2 merged 1 commit into
mainfrom
test/golden-contract-expansion
Oct 21, 2026
Merged

Aparnap2 merged 1 commit into
mainfrom
test/golden-contract-expansion

Conversation

@Aparnap2

Copy link
Copy Markdown
Owner

What changed

Two files, test-only:

File Change
fixtures/golden_cases.json +10 cases appended (GOLDEN-10..19). Cases 01–09 byte-identical (machine-verified), version unchanged at 1.0.0, nothing removed, no existing expected value rewritten.
apps/api/internal/claims/parity_test.go +750/−15: a coverage registry replacing the hard-coded count + prefix list, a two-way fixture↔registry contract test, 9 new helpers, 10 new tests.

No production Go file, Python artifact, CI workflow, dependency, or doc changed. Base: main@254c4b92dddf59a758a133c1edbf7618dcc011b2. Commit: df3309a.

The gaps this closes

MALFORMED_CLAIM, TENANT_MISMATCH, INVALID_EVENT_ID and UNKNOWN_STATUS were referenced by zero Go test files. Six legal edges had no asserted coverage either — because ClaimStatusInvestigationRequired and ClaimStatusWaitingForEvidence appeared in no test file at all.

Re-derived edge counts (the previously reported "8 of 13" was wrong on both sides): the graph has 14 legal edges; 4 were asserted in any Go file before this change (one of them PostgreSQL-gated only); 10 were asserted nowhere; one of those is covered by open PR #178, leaving 9 genuinely uncovered on this base. This change covers 6 of the 9. DB-independent legal-edge assertions go from 3 → 9.

Case Covers Expectation
GOLDEN-10-malformed-claim MALFORMED_CLAIM (was 0 coverage) empty status / version < 1 / empty tenant → MALFORMED_CLAIM; malformed precedes tenant mismatch
GOLDEN-11-tenant-mismatch TENANT_MISMATCH (was 0) acting tenant ≠ claim tenant → TENANT_MISMATCH, and it wins over blank event-id, stale version and illegal edge
GOLDEN-12-invalid-event-id INVALID_EVENT_ID (was 0) whitespace-only and empty idempotency key → INVALID_EVENT_ID; wins over version and legality
GOLDEN-13-unknown-status UNKNOWN_STATUS (was 0) unrecognized source or target → UNKNOWN_STATUS, not ILLEGAL_TRANSITION; version check precedes the status check
GOLDEN-14-validating-to-exception VALIDATING → EXCEPTION success, EXCEPTION, v1→v2, key recorded, input unmutated, one audit event
GOLDEN-15-exception-to-investigation EXCEPTION → INVESTIGATION_REQUIRED INVESTIGATION_REQUIRED, v2, one audit event
GOLDEN-16-investigation-to-waiting-evidence INVESTIGATION_REQUIRED → WAITING_FOR_EVIDENCE WAITING_FOR_EVIDENCE, v2, one audit event
GOLDEN-17-waiting-evidence-to-hitl WAITING_FOR_EVIDENCE → HITL HITL, v2, one audit event
GOLDEN-18-hitl-to-action-pending HITL → ACTION_PENDING ACTION_PENDING, v2, one audit event
GOLDEN-19-action-pending-to-actioned ACTION_PENDING → ACTIONED ACTIONED, v2, one audit event

Every expectation is derived from the documented check order in statemachine.go (malformed → tenant → event-id → idempotent replay → version → known status → legality), the transition graph, and the canonical chain in docs/AGENT_OPERATING_CONTRACT.md. No business rule was invented.

The harness no longer maintains a second count

goldenExpectationByID is a 19-entry coverage registry of hardcoded literals, matched against the fixture in both directions: every fixture id appears exactly once, every registered id is present exactly once, every fixture case has Go coverage, and every case's fixture expected equals the hardcoded Go expectation. A fixture case without coverage now fails (fixture case %q has no Go coverage) instead of passing silently. The previous len(doc.Cases) != 9 assertion and the nine-entry prefix list are gone.

Each new case re-asserts the fixture's inputs against hardcoded literals, runs the scenario through the direct Go APIs, asserts the observed outcome against hardcoded literals, then asserts the fixture's expected against the same literals. Proven live rather than echoed: mutating the fixture broke the tests, and separately mutating the Go expectation broke them with the observed value reported — the expected side is real implementation output, not a fixture echo.

Verification

Run by me against real PostgreSQL (claimops-postgres, claimops_app with rolsuper=f, rolbypassrls=f, so RLS is genuinely enforced):

go test ./... -count=1                        -> 41 packages ok, 0 failures   (4 more packages ran than without a DSN)
go test -race -count=1 ./...                  -> 37 packages ok, 0 failures, no data races
go vet ./... / go build ./...                 -> clean
go test ./internal/claims/... -v -count=1     -> 24/24 PASS, 0 skips

The PG-gated suites that the authoring agent could not reach were run for real and pass: TestClaimSubmit_* (6/6), TestDecision_ExpireFromHITL_Live, TestDecision_ExpireFromNonHITL_Rejected_Live, TestDecision_DuplicateIsReplay, TestDecision_ConcurrentIdenticalOneMutation, TestDecision_ConcurrentDifferentEventsConflict, TestHITL_ExpiredExcludedFromPending.

Not run (services absent, not silently skipped): Mockoon contract tests (gate returns 404 on GET /v1/policies/POL-001), localgcp GCS/Pub/Sub, GCW emulator, and the Groq live-LLM matrix. The authoring agent's gofmt binary and go clean -testcache were blocked by its shell policy; formatting was verified with go fmt plus a read-only go/format equivalent, and every run used -count=1, which bypasses the test cache. uv/ruff/pytest were not run — no Python changed.

Findings — deferred, not encoded

  • F1 (doc defect, out of scope): statemachine.go:12-14 claims golden case 08 witnesses "stale-version wins over legality", but case 08's edge REGISTERED → DOCUMENTS_RECEIVED is legal, so it witnesses only that a stale version is rejected. The precedence is real (check 5 precedes check 7) but unwitnessed in Go.
  • F2 (new cross-implementation divergence): the fixture's expected.audit_actions is ["claim.status.transition"] — the Python action string — while Go emits "claim.transitioned". Asserting the fixture's value in Go would fail. The new cases therefore use audit_count (language-independent: both implementations record exactly one audit event per successful transition) instead of the action string.
  • F3 (terminal_state deferred): the field is read by neither implementation and its meaning differs across cases; GOLDEN-01's value ("VALIDATING") contradicts the outcome both suites assert (REGISTERED, v2). Correcting it would require ratifying a definition no ADR or contract establishes, so it is documented in the parity_test.go package header rather than silently rewritten. (The one contract-supported correction would be GOLDEN-01 → "REGISTERED"; say the word and it can be applied with the reasoning stated.)
  • F4 (new Go/Python definition divergence): Go treats a malformed claim as Status == "" || Version < 1 || Tenant == ""; the Python spec tests absence (None), not emptiness, and its model makes version < 1 and a blank tenant unconstructible, so status="" would be UNKNOWN_STATUS there rather than MALFORMED_CLAIM. GOLDEN-10 therefore encodes the authoritative Go definition and cannot be satisfied by the Python spec artifact. This warrants its own issue.
  • F5: no Python test asserts GOLDEN-10..19, so those ten are currently a Go-side contract. Deliberate and stated — this change is Go-scoped.

Residual, not silently dropped: edges REGISTERED → DOCUMENTS_RECEIVED, DOCUMENTS_RECEIVED → VALIDATING and VERIFIED → CLOSED remain unasserted; ACTIONED → VERIFIED remains PostgreSQL-gated; the STALE_VERSION-before-ILLEGAL_TRANSITION precedence remains unwitnessed (F1); READY_FOR_REVIEW terminality remains delegated to PR #178 (no overlap with this change — no terminality assertion and no READY_FOR_REVIEW edge here, and no symbol collisions).

Status

Test-only, deterministic (fixed literal ids, fixed literal dates, exact int64 paise; no uuid4(), no time.Now(), no randomness, no environment-dependent expectation, no secrets, no patient data). Not authorized to merge — opened as a draft. Release status NOT QUALIFIED; this change qualifies nothing and closes no entry in docs/KNOWN_LIMITATIONS.md.

@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b3afd38c-72da-46ea-853a-f49ed2515fae
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The shared fixture carried nine cases and the Go harness asserted a
hard-coded count plus a hard-coded prefix list. Ten contract gaps had
no Go coverage at all, and four error codes (MALFORMED_CLAIM,
TENANT_MISMATCH, INVALID_EVENT_ID, UNKNOWN_STATUS) were referenced by
zero test files.

Adds GOLDEN-10..19 covering the four error codes and the legal edges
VALIDATING->EXCEPTION, EXCEPTION->INVESTIGATION_REQUIRED,
INVESTIGATION_REQUIRED->WAITING_FOR_EVIDENCE, WAITING_FOR_EVIDENCE->HITL,
HITL->ACTION_PENDING and ACTION_PENDING->ACTIONED, each asserting the
error category, the resulting state and version, that the input claim is
not mutated, and the audit effect.

The harness no longer hard-codes a case count: goldenExpectationByID is
a coverage registry matched against the fixture in both directions, and
a fixture case without Go coverage fails the contract test. The known
fixture-schema inconsistencies (decorative terminal_state and
audit_actions fields, and the GOLDEN-01 terminal_state conflict) are
deferred and documented in the package header rather than silently
rewritten.

Cases 01-09 are byte-identical; the fixture was edited additively only.
Test-only: no production Go file, Python artifact, CI workflow, or
dependency change. Release status remains NOT QUALIFIED.
@Aparnap2
Aparnap2 force-pushed the test/golden-contract-expansion branch from df3309a to b0c9c7a Compare October 21, 2026 06:47
@Aparnap2
Aparnap2 marked this pull request as ready for review October 21, 2026 06:52
@Aparnap2
Aparnap2 merged commit 7ad446c into main Oct 21, 2026
7 checks passed
Aparnap2 added a commit that referenced this pull request Oct 21, 2026
The rebase onto main@7ad446c (which merged #180 and #179) made three
statements in this change false as written:

- The ADR, the decision-log entry and KL-18 all said the cross-language
  fixture 'contains nine cases'. After #179 it contains nineteen. None
  of the nineteen covers EXPIRED, so the substantive claim is unchanged;
  only the count was stale.
- The ADR's Base line still named main@254c4b9; it is now written against
  main@7ad446c.
- The Context said 'no document in this repository ever ratified it'.
  This ADR ratifies it, so the claim needs its temporal qualifier. The
  statemachine.go reference is likewise now stated as 'before this ADR',
  since this change corrects that header.

Also fixes KL-18's position: it was inserted between KL-16 and KL-17
instead of after KL-17, so the register's rows were out of order.

Co-Authored-By: opencode <no-reply@opencode.ai>
Aparnap2 added a commit that referenced this pull request Oct 21, 2026
* docs(adr): ratify the claim expiry contract (ADR-011, S5/APA-26)

EXPIRED has been implemented, wired, tested and deployed for a long time,
but no document in this repository ever ratified it. Its semantics were
attested only by code comments citing a Linear id the repo does not
record, while the file calling itself the specification of this lifecycle
did not contain the status at all.

ADR-011 records the approved semantics: EXPIRED is terminal-for-round
(no out-edges; terminality by absence from claims.Transitions), reachable
only as HITL -> EXPIRED or ACTION_PENDING -> EXPIRED (HITL-wait timeout).
Timeout expiry can never approve, reject, close, escalate or otherwise
decide a claim; post-expiry APPROVE/REJECT/ESCALATE/HOLD are illegal and
APPROVE from HITL was already illegal. Re-drive after expiry requires an
explicit new decision/round, never an implicit edge.

Reconciled documentation:
- statemachine.go's header no longer claims the file is 'the Go port of the
  Python specification' - packages/domain/state_machine.py is a demoted
  artifact, not a mirror, and does not define EXPIRED. Comment-only change.
- AGENT_OPERATING_CONTRACT.md item 5 now lists EXPIRED in the chain.
- packages/domain/state_machine.py's docstring states it is superseded for
  lifecycle semantics and is not a mirror. Docstring-only change.
- 09_DECISION_LOG.md and 06_PROGRESS.md carry the decision.
- KNOWN_LIMITATIONS.md gains KL-18 registering the narrower Python graph
  rather than reconciling it silently.

No lifecycle behaviour changed. The implementation already behaved this
way and the existing tests already pin it: claims/expiry_red_test.go,
handlers/decision_expiry_test.go (4 PG-gated),
repository/postgres/hitl_expiry_test.go.

The statemachine.go comment about golden case 08 is also inaccurate but
is deliberately left alone: that correction is a separate focused change.

Co-Authored-By: opencode <no-reply@opencode.ai>

* docs(adr): correct ADR-011 facts after rebasing onto #179

The rebase onto main@7ad446c (which merged #180 and #179) made three
statements in this change false as written:

- The ADR, the decision-log entry and KL-18 all said the cross-language
  fixture 'contains nine cases'. After #179 it contains nineteen. None
  of the nineteen covers EXPIRED, so the substantive claim is unchanged;
  only the count was stale.
- The ADR's Base line still named main@254c4b9; it is now written against
  main@7ad446c.
- The Context said 'no document in this repository ever ratified it'.
  This ADR ratifies it, so the claim needs its temporal qualifier. The
  statemachine.go reference is likewise now stated as 'before this ADR',
  since this change corrects that header.

Also fixes KL-18's position: it was inserted between KL-16 and KL-17
instead of after KL-17, so the register's rows were out of order.

Co-Authored-By: opencode <no-reply@opencode.ai>

---------

Co-authored-by: opencode <no-reply@opencode.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant