refactor(assert): Give assertions a kind and failures their parts - #89
Merged
Conversation
An Assertion was a func returning error, so a failure could only ever be a sentence. Nothing downstream could ask which assertion failed, what it wanted or what it got without parsing English, which is why machine-readable output (#45) had nothing to serialize. Assertion is now an interface: Kind() names the family, and Check returns (*Failure, error). Failure carries Kind, Target, Expected and Actual alongside the Message, and the message is kept rather than derived from the parts -- sentences like "expected to be non-empty, got nothing" do not decompose into Expected and Actual, and a formatter that tried would drift the first time a wording changed. The two returns separate outcomes that a single error conflated. A *Failure means the response was read and disagreed. An error means the assertion could not be evaluated at all: an undecodable body, a gojq runtime fault, a body that is not JSON. Both still fail the run and still print identically -- doOnce collects them into one list, so --help's promise that an undecodable body fails the body assertions and leaves the others alone is unchanged, exit code and all. The distinction exists for #45, where "the service is wrong" and "we could not tell" are different answers. Constructors stay closures behind a small adapter rather than becoming thirteen one-method structs, which is what keeps each one readable as a single expression. Check stamps the kind onto the failure, so Kind() and Failure.Kind cannot disagree. No output changes. Every failure message is byte-identical to before, verified by diffing both binaries across thirteen failing scenarios covering all seven kinds, and by the end-to-end suite passing unchanged. Refs #56 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CrknafJSP5hF8u865cbnqX
The refactor's own tables assert on the message, which is exactly the part that did not change; nothing covered the fields that did. These are what #45 will serialize, so a change to them is a change to a public contract rather than an internal detail, and it should fail a test rather than surprise a consumer. Two tests. The first walks one failing case per kind and checks Kind, Target, Expected and Actual, including that Check stamps the kind so Kind() and Failure.Kind cannot drift apart. The second covers the split the interface exists to draw: an assertion that holds reports neither return, a body that could not be decoded is an error rather than a Failure with invented Expected/Actual, and a status assertion against that same response still passes. No end-to-end test accompanies this. The refactor changes no observable behaviour, and the one invariant it could have broken -- an undecodable body failing the body assertions with exit 93 while the others pass -- is already covered end-to-end and passes unchanged. Refs #56 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CrknafJSP5hF8u865cbnqX
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A monitoring agent that wants to know which assertion failed has to parse English, because a failure is only ever a sentence.
Assertionis afunc(*httpResponse) error, so everything a consumer might want — which check ran, what it wanted, what it got — exists solely inside a formatted string. That is why--json(#45) has been blocked: there is nothing to serialize.The
errorreturn also conflates two answers that are not the same:body: expected "zzz", got "boom"body: response is br-encoded and was not decodedjq[.n == 1]: expected true, got falsejq[.n == 1]: <gojq runtime fault>"The service is wrong" and "we could not tell" are different operational conclusions, and today nothing downstream can distinguish them.
Note that this ticket's other half — making the pattern constructors return errors instead of panicking — shipped in
4d5dc06three hours after #56 was filed, as part of closing #17. The ticket body has been rewritten to match.Solution
Make
Assertionan interface whoseCheckreturns the failure's parts, not just its prose.Before:
After:
(nil, nil)holds,(*Failure, nil)disagreed,(nil, error)could not be evaluated. Both failure paths still collect into one list indoOnceand still exit93— the split is for #45, not for the exit codes.No output changes, and that is checked rather than asserted
Both binaries were run against thirteen failing scenarios covering all seven kinds, and the output diffed:
The twelve distinct messages compared include every shape that does not decompose into expected/actual —
header[X-Missing]: expected to match ".*", missing,redirect: wrong HTTP status: got 500 (…),jq[.missing]: expected true, got null. The end-to-end suite passes unchanged, which is the same check enforced in CI.Two deliberate choices
FailurekeepsMessagerather than deriving it. Sentences likeexpected to be non-empty, got nothingandexpected OK, got 500 ("500 Internal Server Error")do not reconstruct fromExpectedandActual. A formatter that tried would drift the first time a wording changed, and the drift would surface as a failing end-to-end test rather than a compile error. The parts and the prose are written together, at the same site, and neither derives from the other.Constructors stay closures behind a small adapter instead of becoming thirteen one-method structs — the functional style is what keeps each one readable as a single expression, and only the result ever needed structure.
Checkstamps the kind onto the failure, soKind()andFailure.Kindcannot drift apart and no constructor repeats itself.Other Changes
Location) is now shared by both redirect assertions, which is what keeps their wording identical rather than merely equal today.check(a, res)helper flattening the two returns to the single error the existing tables compare against, so those tables are otherwise untouched.93while status and header assertions still pass — is already covered end-to-end and passes unchanged.Related:
🤖 Generated with Claude Code