Escape characters a pattern cannot carry as bytes

[FrayxRulez]

Stacked on #55, whose rewrite this relies on. Review that one first.

writeString serializes one byte per JavaScript code unit:

chunks.push(new Uint8Array(str.split('').map(char => char.charCodeAt(0))))

Uint8Array keeps the low byte, so every character above U+00FF is silently mangled. Unlike the escapes #55 deals with, this hits characters a grammar writes literally. BQN's function pattern upstream:

'function': /•[\w¯.∞π]+[\w¯.∞π]*/,

and in grammars.dat today:

/"[\w¯.\x1eÀ]+[\w¯.\x1eÀ]*/

• (U+2022) became ", ∞ (U+221E) became \x1e, π (U+03C0) became À. So BQN's function token matches a double quote followed by word characters, and fights the string-literal pattern for it.

33 patterns across 13 languages: agda, apl, applescript, aql, art/arturo, bqn, false, julia, kusto, openqasm/qasm, sas.

Two changes:

• sanitize now spells anything above U+007F as \uXXXX, so a pattern reaches the table as ASCII and every reader gets a code point rather than a byte. An astral character is two code units in JavaScript and so becomes a surrogate pair, which Expand unicode escapes Boost does not read #55 joins back together.
• writeString encodes UTF-8 and counts bytes rather than code units. That is a no-op for the table as it stands — with patterns escaped, every string in it is ASCII — but it is the actual bug, and the next non-ASCII title would hit it. (kumir's is KuMir (КуМир); it survives only because the language is on the unsupported list.)

UnicodeEscapes.h gains the matching rule: a class member above 0x7F now has to be lifted out, not just one above 0xFF. Boost truncates [¯] to a single byte just as it truncates [∧], and ¯ is two bytes in UTF-8.

Verification

Regenerating and checking the table:

• 0 patterns still carry a byte above 0x7F, down from 33
• 3958 of 3958 compile under both Boost and java.util.regex, and the two engines agree on every pattern over the check corpus — the same gate Check every pattern against both regex engines on a pull request #56 adds

Tokenizing under all 386 listed languages and comparing libprisma against a Java reader of the same table, on a corpus of real ∧ λ ∀ ≤ 😀 text: apl, applescript, art, arturo, bqn and julia stop diverging, nothing regresses, and the ASCII corpus stays at 386 of 386.

🤖 Generated with Claude Code