Skip to content

fix(tri_reader): indentation-aware rewrite so .tri specs actually parse - #93

Merged
gHashTag merged 1 commit into
mainfrom
claude/keen-hertz-fb603f
Aug 7, 2026
Merged

fix(tri_reader): indentation-aware rewrite so .tri specs actually parse#93
gHashTag merged 1 commit into
mainfrom
claude/keen-hertz-fb603f

Conversation

@gHashTag

@gHashTag gHashTag commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Problem

tri_reader.parse returned an all-zero Spec for every canonical spec. Verified: parse(specs/gf8.tri) and parse(specs/gf16.tri) (valid input) came back with storage.bits = 0 and exponent.bits/bias/max = 0. tri_gen masked this because its emitters ignore the parsed Spec (template copiers — every genC/genRust/genZig/genCpp does _ = spec;).

Root cause was structural, not a single typo. The old parser walked the byte stream with per-section cursor functions and had no notion of block scope, compounded by:

  1. A duplicated dispatch block in parseSpec — two else if (eql(key,"storage")) branches; the first consumeValue() shadowed the real parseStorage() (same for version/level/type).
  2. readKey broke at : without consuming it, so after a header like storage: the cursor sat on : and the sub-parser's first readKey returned null immediately.
  3. No indentation/dedent tracking — sub-block loops had no block scope and would over-consume into the next top-level section.
  4. parseStorage was positional — ran parseInt on the bits key text; catch 0 swallowed the error and desynced the cursor.
  5. parseExponent consumed then re-parsed bits/bias/max/min from the advanced cursor (garbage → InvalidCharacter).

Fix

Replaced the Parser with a two-phase, indentation-aware design:

  1. lex — split into significant lines carrying (indent, is_item, text); quote-aware inline #-comment stripping.
  2. buildNodes — fold lines into an indentation tree; children are strictly-more-indented lines, so a nested block cannot over-consume into the next top-level section.
  3. Mapper — walk the tree onto the typed Spec by child-key lookup, scope-safe by construction. YAML block sequences (fields, test_vectors) and mapping-style blocks (ops, level-5 types) fold uniformly.

Memory is now arena-owned: parse stores an ArenaAllocator in spec.arena_state and dupes all strings/slices into it; Spec.deinit frees the arena in one shot (leak-clean under std.testing.allocator, replacing the previous hand-maintained per-field frees).

The public API (Spec, load, parse, all sub-structs) is unchanged, so tri_gen.zig and check_tri_hashes.zig still compile and run.

Tests

Added tri-reader-tests, wired into build.zig via addAnonymousImport + @embedFile (specs live outside tools/gen/, so they can't be embedded directly):

spec storage.bits exp.bits bias max phi.mantissa_bits
gf8.tri 8 3 3 7 4
gf16.tri 16 6 31 63 9

(plus field-layout and NaN/Inf special-value assertions).

  • zig build test148/148 pass (146 prior + 2 new)
  • zig build, zig build gen → exit 0; zig fmt clean
  • No .tri specs changed, so TRI-HASHES.md is unaffected.

Why it matters

Step 0 of docs/gft-spec-first-plan.md: parse now returns the real spec values, unblocking a parse-time spec↔code consistency guard — the kind that would have caught the GF8 bias=7 bug (#84) at the spec level — and is the prerequisite for turning tri_gen into a genuine generator.

Note for reviewers

tri_gen --verbose prints nothing because its stderr() buffer is never flushed before exit — a pre-existing bug in tri_gen.zig, independent of this change and left untouched (out of scope). Can be spun off separately.

🤖 Generated with Claude Code

The .tri parser walked the byte stream with per-section cursor functions
and had no notion of block scope. Combined with a duplicated storage/
version/level/type dispatch block, a readKey that broke *at* ':' (leaving
the cursor stranded so sub-parsers saw an empty block), a positional
parseStorage that ran parseInt on the "bits" key text, and a parseExponent
that consumed-then-re-parsed its fields, parse() returned an all-zero Spec
for every canonical spec (storage.bits=0, exponent.bits/bias/max=0).
tri_gen masked this because its emitters ignore the parsed Spec.

Replace the Parser with a two-phase, indentation-aware design:
  1. lex        — significant lines (indent, list-item marker, text),
                  quote-aware inline-comment stripping.
  2. buildNodes — fold lines into an indentation tree; children are
                  strictly-more-indented lines, so a nested block cannot
                  over-consume into the next top-level section.
The tree is then mapped onto the typed Spec by key lookup, which is
scope-safe by construction. Both YAML block sequences (fields/test_vectors)
and mapping-style blocks (ops, level-5 types) fold uniformly.

Memory is now arena-owned: parse() stores an ArenaAllocator in
spec.arena_state and dupes all strings/slices into it; Spec.deinit frees
the arena in one shot (leak-clean under std.testing.allocator). The public
API (Spec, load, parse, sub-structs) is unchanged, so tri_gen and
check_tri_hashes still compile and run.

Add tri-reader-tests wiring in build.zig via addAnonymousImport +
@embedfile (specs/ lives outside tools/gen/), asserting:
  gf8.tri  -> storage.bits=8,  exponent.bits=3, bias=3,  max=7,  mantissa_bits=4
  gf16.tri -> storage.bits=16, exponent.bits=6, bias=31, max=63, mantissa_bits=9

This is step 0 of docs/gft-spec-first-plan.md: parse() now returns real
spec values, unblocking a spec<->code consistency guard (the kind that
would have caught the GF8 bias=7 bug, #84).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@gHashTag
gHashTag merged commit 1f597ca into main Aug 7, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant