Skip to content

fix(engine)!: keep compiler diagnostics and exit on spec problems - #335

Merged
OmarAlJarrah merged 2 commits into
mainfrom
fix/engine-diagnostic-flow-and-exit-codes
Aug 9, 2026
Merged

fix(engine)!: keep compiler diagnostics and exit on spec problems#335
OmarAlJarrah merged 2 commits into
mainfrom
fix/engine-diagnostic-flow-and-exit-codes

Conversation

@OmarAlJarrah

Copy link
Copy Markdown
Member

Summary

Two defects in how a spec problem travels out of the engine, fixed together because they share a seam.

Diagnostics were replaced instead of merged. engine.Run ran diags = doc.Diagnostics after a
successful compile, so a compiler that returns its findings without also storing them on the document
lost all of them, error severity included. The one compiler in the tree fills both lists, which is
why nothing caught this: the only stub that returned unstored diagnostics returned a nil document
too, and a nil document short-circuits one branch earlier. The direction is the perverse part —
with the validate pass enabled (the default) the finding count went to zero, and with
--skip-validate it did not. Turning validation on removed findings, and the CLI then exited 0 on
a spec its compiler had refused outright.

Run now merges the two lists and writes the union to both, so a finding survives whichever channel
its compiler used. compilers.Compiler's doc comment now says what each list is, since the engine
was relying on an agreement the contract never stated.

Spec problems escaped as Go errors and became usage exit codes. An undecodable file, an
unrecognized format, Swagger 2.0, and a version no compiler claims were all fmt.Errorf out of
Run, and cmd/morphic maps any Go error to exit 2 — the code a bad flag gets. So
morphic compile v4.yaml answered no compiler registered for format openapi@4.0 with exit 2 while
the compiler's own openapi/unsupported-version diagnostic for those exact bytes stayed unreachable
from the shipped binary. A CI wrapper telling "this spec is broken" from "you invoked the tool
wrong" misclassified every one of them.

Sniff returns a diagnostic and a comma-ok instead of an error — nothing it can hit is an I/O or a
programmer error — and Run returns the shape it already blesses as legal: a nil document alongside
diagnostics. Four codes in the engine's own namespace: engine/undecodable-source,
engine/unrecognized-format, engine/unsupported-format, engine/no-compiler-for-format. The CLI
needed no exit-code change; a nil document was already exit 1.

Exit-code taxonomy

Code Meaning
0 Clean: the spec lowered and no diagnostic reached --fail-on. Also every help request.
1 The spec has problems: a diagnostic at or above --fail-on, or it produced no document at all — undecodable, unrecognized format, unsupported format, no compiler for its version.
2 The invocation or the filesystem was wrong: a bad flag or argument, a spec that could not be read, an output that could not be written.

Nothing about a spec's own contents reaches 2. This is the taxonomy README already documented and
the code did not implement. README's wording is tightened to name what falls in each bucket, and to
describe the diagnostic line's location field now that it has a third form — absent, for a finding
raised before any document existed, which is what these new codes carry.

Notes on adjacent work

Test plan

  • engine: TestEngine_RunKeepsDiagnosticsFromEitherChannel drives a stub compiler whose stored and
    returned lists are set independently, across both SkipValidate modes. Unlike the existing
    nilDocCompiler it hands back a non-nil document, so it reaches the fold rather than stopping a
    branch short of it. Three of its six rows fail on the old code.
  • engine: TestEngine_RunSniffProblemsAreDiagnostics and TestEngine_RunLookupMiss assert a
    diagnostic, a nil document, error severity and ir.NoSource provenance in place of a Go error.
    TestSniff_Formats asserts codes rather than error substrings.
  • cmd/morphic: TestRun_SpecProblemsExitOne runs the real CLI over one spec per class and asserts
    exit 1, the code on stderr, empty stdout, and no usage block.
  • cmd/morphic: TestRenderDiagnostics_WithAndWithoutSourcePath pins all three rendered location
    forms as whole lines rather than fragments.
  • Each of the three production changes was reverted in turn with the tests left in place; the
    matching test went red every time and green again on restore.
  • Full gate green: gofmt, go vet, golangci-lint, go build, coverage at 100%.

Breaking

engine.Sniff changes signature from (compilers.SourceFormat, error) to
(compilers.SourceFormat, ir.Diagnostic, bool); it is exported, so an embedder calling it directly
must adapt. engine.Run no longer returns a Go error for the four classes above — it returns a
*Result with a nil Document and diagnostics — and morphic compile exits 1 rather than 2 for
them, which anything scripting the CLI will observe.

Closes #56
Closes #58

Run replaced the compiler's returned diagnostics with the document's own
list, so a compiler that reports without also storing lost every finding
at every severity. It now merges the two and writes the union to both.

An undecodable file, an unrecognized format, Swagger 2.0 and a version
no compiler claims all left Run as Go errors, which the CLI renders as
exit 2 — the code a bad flag gets. They are problems with the spec, so
they now come back as diagnostics with a nil document and exit 1, and
the usage code is left to actual misuse.
@OmarAlJarrah OmarAlJarrah changed the title fix(engine): keep compiler diagnostics and exit on spec problems fix(engine)!: keep compiler diagnostics and exit on spec problems Aug 9, 2026
The README conflict this branch predicted, plus one it could not: #312 added a
validate command whose TestRun_ValidateEngineFailures asserts the old taxonomy —
an unrecognized format exiting 2. This change makes that a diagnostic and exit 1,
so the row had to move with it. It now asserts the code, and a comment says why
the two rows in that table exit differently: a format nothing recognizes is a
problem with the spec, an unreadable file is a problem with the invocation.

Both commands land on the new behaviour identically, since validate reaches it
through the shared pipeline rather than a second copy.

README: #312's two-command flag table already sits above the conflict, so this
branch's pre-validate table is dropped and its new prose kept — the three
location forms and the corrected exit-code taxonomy — reworded to keep #312's
"both commands" framing.

Two tests are gone against main and both are this branch's own deletions, not
the merge's: TestEngine_RunSniffError and TestRun_ParseUnknownSpecFails assert
the Go-error behaviour being inverted, and are replaced by
TestEngine_RunSniffProblemsAreDiagnostics and TestRun_SpecProblemsExitOne.
@OmarAlJarrah
OmarAlJarrah merged commit e85c8cd into main Aug 9, 2026
1 check passed
@OmarAlJarrah
OmarAlJarrah deleted the fix/engine-diagnostic-flow-and-exit-codes branch August 9, 2026 11:47
OmarAlJarrah added a commit that referenced this pull request Aug 9, 2026
The three conflicts this branch predicted, resolved as it said: compile.go keeps
#312's specOptions split and gains the pretty field, command_test.go keeps
#312's two-list block and gains "pretty", and the README keeps #312's
two-command flag table with the -o wording changed and a --pretty row added.
#335 also landed on the README since, so its diagnostics and exit-code prose is
kept whole rather than reverted to this branch's older paragraph.

One interaction the branch could not predict: #330's
TestRun_TerminatorAsFlagValue asserts the artifact contains `"name": "Tiny"` —
the indented spelling. That test is about which argument "--" became, not about
formatting, and -o is compact now, so it matched text it never meant to pin. It
decodes the artifact instead, which is what it was always asking.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant