fix(tri_reader): indentation-aware rewrite so .tri specs actually parse - #93
Merged
Conversation
The .tri parser walked the byte stream with per-section cursor functions
and had no notion of block scope. Combined with a duplicated storage/
version/level/type dispatch block, a readKey that broke *at* ':' (leaving
the cursor stranded so sub-parsers saw an empty block), a positional
parseStorage that ran parseInt on the "bits" key text, and a parseExponent
that consumed-then-re-parsed its fields, parse() returned an all-zero Spec
for every canonical spec (storage.bits=0, exponent.bits/bias/max=0).
tri_gen masked this because its emitters ignore the parsed Spec.
Replace the Parser with a two-phase, indentation-aware design:
1. lex — significant lines (indent, list-item marker, text),
quote-aware inline-comment stripping.
2. buildNodes — fold lines into an indentation tree; children are
strictly-more-indented lines, so a nested block cannot
over-consume into the next top-level section.
The tree is then mapped onto the typed Spec by key lookup, which is
scope-safe by construction. Both YAML block sequences (fields/test_vectors)
and mapping-style blocks (ops, level-5 types) fold uniformly.
Memory is now arena-owned: parse() stores an ArenaAllocator in
spec.arena_state and dupes all strings/slices into it; Spec.deinit frees
the arena in one shot (leak-clean under std.testing.allocator). The public
API (Spec, load, parse, sub-structs) is unchanged, so tri_gen and
check_tri_hashes still compile and run.
Add tri-reader-tests wiring in build.zig via addAnonymousImport +
@embedfile (specs/ lives outside tools/gen/), asserting:
gf8.tri -> storage.bits=8, exponent.bits=3, bias=3, max=7, mantissa_bits=4
gf16.tri -> storage.bits=16, exponent.bits=6, bias=31, max=63, mantissa_bits=9
This is step 0 of docs/gft-spec-first-plan.md: parse() now returns real
spec values, unblocking a spec<->code consistency guard (the kind that
would have caught the GF8 bias=7 bug, #84).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
tri_reader.parsereturned an all-zeroSpecfor every canonical spec. Verified:parse(specs/gf8.tri)andparse(specs/gf16.tri)(valid input) came back withstorage.bits = 0andexponent.bits/bias/max = 0.tri_genmasked this because its emitters ignore the parsedSpec(template copiers — everygenC/genRust/genZig/genCppdoes_ = spec;).Root cause was structural, not a single typo. The old parser walked the byte stream with per-section cursor functions and had no notion of block scope, compounded by:
parseSpec— twoelse if (eql(key,"storage"))branches; the firstconsumeValue()shadowed the realparseStorage()(same for version/level/type).readKeybroke at:without consuming it, so after a header likestorage:the cursor sat on:and the sub-parser's firstreadKeyreturnednullimmediately.parseStoragewas positional — ranparseInton thebitskey text;catch 0swallowed the error and desynced the cursor.parseExponentconsumed then re-parsed bits/bias/max/min from the advanced cursor (garbage →InvalidCharacter).Fix
Replaced the
Parserwith a two-phase, indentation-aware design:lex— split into significant lines carrying(indent, is_item, text); quote-aware inline#-comment stripping.buildNodes— fold lines into an indentation tree; children are strictly-more-indented lines, so a nested block cannot over-consume into the next top-level section.Specby child-key lookup, scope-safe by construction. YAML block sequences (fields,test_vectors) and mapping-style blocks (ops, level-5types) fold uniformly.Memory is now arena-owned:
parsestores anArenaAllocatorinspec.arena_stateand dupes all strings/slices into it;Spec.deinitfrees the arena in one shot (leak-clean understd.testing.allocator, replacing the previous hand-maintained per-field frees).The public API (
Spec,load,parse, all sub-structs) is unchanged, sotri_gen.zigandcheck_tri_hashes.zigstill compile and run.Tests
Added
tri-reader-tests, wired intobuild.zigviaaddAnonymousImport+@embedFile(specs live outsidetools/gen/, so they can't be embedded directly):gf8.trigf16.tri(plus field-layout and NaN/Inf special-value assertions).
zig build test→ 148/148 pass (146 prior + 2 new)zig build,zig build gen→ exit 0;zig fmtclean.trispecs changed, soTRI-HASHES.mdis unaffected.Why it matters
Step 0 of
docs/gft-spec-first-plan.md:parsenow returns the real spec values, unblocking a parse-time spec↔code consistency guard — the kind that would have caught the GF8bias=7bug (#84) at the spec level — and is the prerequisite for turningtri_geninto a genuine generator.Note for reviewers
tri_gen --verboseprints nothing because itsstderr()buffer is never flushed before exit — a pre-existing bug intri_gen.zig, independent of this change and left untouched (out of scope). Can be spun off separately.🤖 Generated with Claude Code