Grammar: accept combining marks/ZWNJ/ZWJ as prose content; restrict entity regex to semicolon-terminated names (bd-96fswwce, bd-v8qc9zyc)

[cscheid]

Summary

• Combining marks (Mn/Mc/Me) and the join controls ZWNJ/ZWJ (U+200C/U+200D) were absent from pandoc_str's content character classes, so any of them in prose was a hard parse error: Devanagari vowel signs (का — i.e. any Hindi text), NFD/decomposed accents (cafe + U+0301), enclosing marks, ZWNJ/ZWJ between letters (Persian/Indic joining), and the combining sequences that named entities like ≂̸ decode to (discovered while implementing Decode named HTML entity references in prose (bd-named-entities-w6xbfftj) #488). Pandoc folds all of these into Str verbatim.
• Adds PANDOC_COMBINING_MARKS = \p{M}\u{200C}\u{200D} to grammar.js as a single-char alternative and in the word-continuation class — the same fix shape as bd-6kewx (which added non-ASCII Po/Pc). ZWJ inside emoji sequences is unaffected (longest match wins). parser.c/grammar.json are regenerated.
• Restores ≂̸ to the named_entities.qmd roundtrip fixture, as promised when it was omitted in Decode named HTML entity references in prose (bd-named-entities-w6xbfftj) #488.

Discovered-from bd-named-entities-w6xbfftj (#488). Plan: claude-notes/plans/2026-08-10-combining-marks-parse.md.

Test plan

• 6 new corpus tests (test/corpus/combining_marks.txt) and 6 new pampa coverage tests, all verified failing pre-fix
• Full corpus suite 551/551; tree-sitter test green
• New roundtrip fixture combining_marks.qmd (literal NFD/Devanagari/ZWNJ text)
• pampa + tree-sitter-qmd suites 4326/4326 on the rebased base; full workspace suite 11237/11237 pre-rebase
• hub-client build:all (WASM from regenerated parser) + test:ci 131/131
• Manual: q2 render of literal combining-mark text — output byte-verified (NFD preserved, not normalized)

Note: single-shot cargo xtask verify was blocked locally by an environmental ephemeral-port leak (orphaned Jupyter kernels, being handled separately); all verify legs passed individually. CI here is the independent single-shot check.

🤖 Generated with Claude Code

[posit-snyk-bot]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[cscheid]

Pushed a second, directly-related grammar commit onto this branch: 05c2454e fixes bd-v8qc9zyc — html_entity_regex() now filters html_entities.json to semicolon-terminated names, removing the 106 mangled legacy alternatives (&AMP → bogus AM). CST behavior pinned by 3 new corpus tests (&AM; case verified failing pre-fix); no user-visible AST change. Full cargo xtask verify green locally for the stacked pair.

About the macOS CI failure on the previous run: it is a pre-existing, unrelated flake, now diagnosed and filed as bd-zazptk5s. The automerge filesystem storage splays doc ids into prefix subdirectories; when two random ids differ in the 2-char prefix only by case (2x… vs 2X…), case-insensitive APFS reuses the first splay directory and ids reconstructed from paths come back with the wrong case — the failed assertion differs by exactly one letter's case at position 2. Same signature observed locally today in a different test of the same file. Linux CI (case-sensitive) can't hit it.

🤖 Generated with Claude Code