Skip to content

fix(runtime): make deep term operations stack-safe - #43

Merged
mremond merged 3 commits into
mainfrom
fm/task-2cb3e3c7
Nov 14, 2026
Merged

mremond merged 3 commits into
mainfrom
fm/task-2cb3e3c7

Conversation

@mremond

@mremond mremond commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Why

Programs building deeply nested iolists should not crash when comparing, encoding, flattening or hashing their data.

What changed

  • Head-nested iolists use heap-allocated work stacks for ordering, encoding, flattening and hashing.
  • Portable data-term hashes match Erlang/OTP instead of Nera's earlier hash.

Before and after

Before: All four operations crashed on a 200,000-level iolist; shallow hashes also differed from Erlang/OTP.

After: The same input produces matching ordering, 1,000,000 flattened bytes, 3,200,002 encoded bytes and hash 108113888 on native Nera and BEAM.

This changes runtime behavior, not a screen; native and BEAM program output directly demonstrates the result.

Checked

Live validation: Demonstrated for this version. 3 of 3 scenarios driven live in the product.

Scenario Result Evidence
Compare, encode, flatten and hash a 200,000-level head-nested iolist. ✅ Passed live Thirty native/BEAM runs and direct program readings agree on ordering, every flattened byte, every encoded byte and the exact hash.
Hash a real 4-GiB binary using both arities and inside a container. ✅ Passed live Native results match BEAM; preserved pre-correction artifacts fail the same assertions.
Compare million-field tuples that differ immediately. (supplementary) ✅ Passed live All comparisons complete correctly; retained duration and memory readings support indexed continuation without a timing guarantee.
  • Full local gate: 2,177 workspace tests and 493 Erlang tests passed; strict lint, allocator, stacktrace and both cross-object checks passed.
  • Related deep operations: Equality, ordered-set traversal, malformed-iolist recovery and 180 binary, sub-binary and container hash readings match BEAM.
How it was checked

cargo build --release --locked -p nera-runtime, then cargo build --release --locked -p nera

cargo clippy --workspace --all-targets --locked -- -D warnings

cargo test --workspace --locked --no-fail-fast

python3 scripts/test-allocator-config.py

./target/release/nera test -j 1 --timeout 30

python3 scripts/test-stacktrace-order.py

scripts/test-cross.sh for x86_64-apple-darwin and aarch64-unknown-linux-gnu; object generation is not target-native execution.

The shared deep_term_ops fixture checks complete encoded and flattened bytes without recursively decoding the deep ETF term.

Independent code and proof reviews cleared the complete change and its retained exact-version validation.

Risk

Risk above low: Blast radius, medium — Shared runtime ordering, encoding, flattening and hashing paths change.; Undo, medium — Previously persisted native hashes may need recomputing.; Novelty, medium — Adds an iterative OTP-compatible portable hasher.

Full risk assessment
Dimension Level Why
Blast radius medium Shared runtime ordering, encoding, flattening and hashing paths change.
Behavior low Corrects crashes and nonportable hash results.
Undo medium Previously persisted native hashes may need recomputing.
Evidence low Direct native and BEAM output, full gate and real boundary probes passed.
Sensitivity low No sign-ins, permissions, payments or stored data are modified.
Novelty medium Adds an iterative OTP-compatible portable hasher.
Dependencies low No dependencies added or updated.
Not done

Fun hashes remain native-specific; deep ETF decoding and pathological nested map-key sorting remain outside this change.

Two external OneCached fixtures were skipped because their modules are unavailable.

Pipeline

Updates from git push no-mistakes

Infinitary checked this version: an independent review that also judged the documents, and live validation the review accepted.

@github-actions github-actions Bot added this to the 0.1.1 milestone Nov 14, 2026
@mremond
mremond merged commit 560a2e9 into main Nov 14, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant