Check every pattern against both regex engines on a pull request

[FrayxRulez]

Stacked on #55, which adds the header tools/probe.cpp shares with the loader. Review that one first.

grammars.dat is read by libprisma with Boost and by the Android client with java.util.regex. Nothing currently holds it to meaning the same thing to both, and the failure that taught us this was silent: \v compiles cleanly in both engines and means U+000B in one and the whole vertical whitespace class — newline included — in the other, which made ini values run past the end of their line. A "does it compile" gate would not have caught it.

So three nets, cheapest first. The workflow regenerates the table from the pull request's own components/ and generate.js before checking it, since the committed one is only refreshed on master.

tools/check-escapes.js — every escape in the table has to be one both engines read the same way. The allowlist is what normalizeEscapes is expected to leave behind; anything else fails with what it means where. This is the net for escapes that compile fine and mean something else. Run against the table as it was before #53, it reports \v and \0 and nothing else.

tools/PatternProbe.java — compiles all 3958 patterns with java.util.regex. It also applies the one rewrite a Java client has to do for itself: outside MULTILINE, $ means end of input in ECMAScript but "or before a final line terminator" in Java, and \z is the Java spelling of the first. That cannot live in the table, where \z would read as a literal z. Against the pre-#53 table it fails 165 patterns.

tools/probe.cpp — the same with Boost, through the same UnicodeEscapes.h the loader uses, so what it tests is what ships.

Both probes write down every offset each pattern matches in tools/corpus.txt, and the last step diffs the two files. That is the check that catches a divergence in meaning rather than in syntax. The corpus is deliberately ASCII: Boost matches bytes where java.util.regex matches characters, so anything above U+007F diverges on offsets alone and would drown out the bugs being looked for.

On the current table, with #55 in: 3958 of 3958 compile in both, and the two engines agree on every pattern.

🤖 Generated with Claude Code