Skip to content

Cover compiler diagnostics in the canonical conformance suite - #2

Merged
konojunya merged 1 commit into
mainfrom
jj/expand-diagnostic-conformance
Sep 3, 2026
Merged

Cover compiler diagnostics in the canonical conformance suite#2
konojunya merged 1 commit into
mainfrom
jj/expand-diagnostic-conformance

Conversation

@konojunya

Copy link
Copy Markdown
Contributor

Background

The canonical conformance suite covered normalized IR but only one compiler diagnostic. Portable diagnostic consumers need stable coverage of every Stack 1.0 code emitted by compiler stages 1 through 4 before the WASM and CLI surfaces are introduced.

Implementation

  • Add eleven invalid cases covering the remaining compiler diagnostic families and deterministic multi-diagnostic ordering.
  • Exercise invalid UTF-8, BOM handling, CRLF byte offsets, Unicode scalar columns, syntax failures, semantic validation, layout scope, dense-node warnings, and complexity limits.
  • Mark the two encoding sources as binary so Git preserves their exact bytes.
  • Add a standard-library coverage validator that requires all 22 compiler-stage codes while leaving renderer-stage diagnostics out of scope.
  • Run the coverage validator in CI and document the expanded suite.

Testing

  • check-jsonschema --check-metaschema schemas/*.json
  • check-jsonschema --schemafile schemas/normalized-ir.schema.json conformance/valid/*/expected.ir.json
  • find conformance -name expected.diagnostics.json -print0 | xargs -0 check-jsonschema --schemafile schemas/diagnostic-expectations.schema.json
  • python scripts/validate-compiler-diagnostics.py
  • STACK_SPECIFICATION_DIR=../specification cargo +stable test --features conformance --test conformance in stack-sh/compiler
  • git diff --check

Review focus / concerns

Please review the end-exclusive byte, line, and Unicode scalar ranges, especially the binary encoding fixtures and deterministic ordering in multi-diagnostic cases. Human-readable diagnostic wording remains deliberately outside the expectation documents.

@konojunya
konojunya merged commit 8a4cf2e into main Sep 3, 2026
1 check passed
@konojunya
konojunya deleted the jj/expand-diagnostic-conformance branch September 3, 2026 04:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant