Make puffscript self-hosting - #1
Open
andrewkchan wants to merge 7 commits into
Open
Conversation
…nter casts, heap builtins - 'import def name(params) type;' declares a host function imported from WASM module 'env' - 'export def' exports a function from the compiled module - int<->pointer casts allow user code to implement heap allocators - pointer-to-struct cast syntax: 'Token~(expr)' - __heap_end__/__grow_heap__ builtins wrap memory.size/memory.grow - address-of now supports struct members and dereferences (&p~.member) - CLI tools: puffc (compile), wat2wasm (assemble), run (execute with env imports) - turn off WAT debug comments to keep output manageable for large programs Co-authored-by: Andrew Chan <andrewkchan@users.noreply.github.com>
… 4MB Both changes make WAT output reproducible by a self-hosted compiler: - float constants no longer depend on JS shortest-round-trip double formatting - the larger in-memory stack accommodates compiler workloads (string temporaries) Co-authored-by: Andrew Chan <andrewkchan@users.noreply.github.com>
- selfhost/util.puff: bump allocator over grown WASM memory, Vec/Buf/Str helpers - selfhost/scanner.puff: token definitions and scanner, differentially tested against the reference scanner (token streams and error messages match) - fix __memcpy__ builtin declaration to match its 3-arg WASM implementation - tools: dump.ts (reference-compiler debug dumps), build-selfhost.sh Co-authored-by: Andrew Chan <andrewkchan@users.noreply.github.com>
Differentially tested against the reference parser: token streams, s-expressions, parse errors, and exit codes match over the fixture corpus (extracted from test.ts) and the selfhost sources themselves. Also fixes a latent reference-compiler bug where emitDebugComments computed s-expressions eagerly (infinite recursion on self-referential struct pointer types) even with debug comments disabled. Co-authored-by: Andrew Chan <andrewkchan@users.noreply.github.com>
Full port of type resolution/checking including out-of-order global resolution, symbol cycle detection, liveness analysis, and coercions. Error output matches the reference resolver over all 116 fixtures (parser/resolver error fixtures, e2e programs, and selfhost sources). Co-authored-by: Andrew Chan <andrewkchan@users.noreply.github.com>
The puffscript compiler written in puffscript (selfhost/) now: - emits WAT byte-identical to the reference compiler on all fixtures - compiles its own source byte-identically to the reference compiler - reaches a bootstrap fixpoint: stage2 (self-compiled) output == stage1 output Co-authored-by: Andrew Chan <andrewkchan@users.noreply.github.com>
- test.ts: bootstrap fixpoint test (stage1==stage2==stage3), output-equivalence tests vs the reference compiler, and an end-to-end run of a program compiled by the self-hosted compiler - tools/bootstrap.sh + 'npm run bootstrap' - README: self-hosting instructions, import/export, pointer casts, heap builtins - regenerate demo.js bundle Co-authored-by: Andrew Chan <andrewkchan@users.noreply.github.com>
andrewkchan
marked this pull request as ready for review
August 13, 2026 07:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Puffscript is now self-hosting:
selfhost/contains a complete puffscript compiler written in puffscript (~5,600 lines) that compiles its own source and reaches a bootstrap fixpoint. Its WAT output is byte-identical to the TypeScript reference compiler's on every tested input, and its error messages match too.Language extensions (TypeScript reference compiler)
import def name(params) type;— host functions imported from WASM moduleenv; the CLI runner providesgetchar/putchar/puterr/exitwired to stdin/stdout/stderr (file I/O)export def— export functions from the compiled moduleint↔pointer casts and pointer-to-struct cast syntax (Node~(p)) — enables user-space heap allocators__heap_end__()/__grow_heap__(n)builtins (wrapmemory.size/memory.grow) — dynamic memory&p~.member)__memcpy__builtin declaration fixed to match its 3-arg WASM implementationSelf-hosted compiler (
selfhost/)Faithful port of the reference pipeline:
util.puff(bump allocator over grown WASM memory, vectors, buffers, strings),scanner.puff,ast.puff,sexpr.puff,parser.puff(panic-flag error recovery mirroring exception unwinding),resolver.puff(out-of-order global resolution, cycle detection, liveness),backend.puff,main.puff(stdin→WAT-on-stdout driver).Verification
tools/extract-fixtures.ts,tools/compare-selfhost.sh)test.tsself-hosting suite:Tooling
tools/puffc.ts(reference CLI),tools/run.ts(WASM runner with host env),tools/wat2wasm.ts,tools/dump.ts(debug dumps),tools/build-selfhost.sh,tools/bootstrap.sh(npm run bootstrap).