Lower LaTeX presentation into the MathExpression semantic core - #615
Merged
Conversation
String-to-tree half of the two-layer math model document-schema.js 3.2 defines: a pinned exact-version temml parser (the one math component not worth hand-writing; the pin gives every stored presentation string one defined parse) feeds a lowering that is mechanical where notation is unambiguous (frac -> math:divide, radicals -> math:sqrt or an exact 1/n exponent, scripted Sigma/Product -> sum/prod binders owning the rest of their term, digit runs -> lowest-terms rationals, subscripts -> distinct symbol identities, superscripts -> math:pow unless the table curates the scripted form as one symbol) and degrades everything context-starved (juxtaposition, overloaded operators, integrals, text, compound subscripts, binomials, align/cases) to unparsed nodes carrying the verbatim source span plus a named diagnostic -- total, never a throw, never a silent guess. Symbol-table support: a conservative sentence-level where/let prose scanner seeds curated entries, command-to-glyph mapping covers Greek and the letterlike symbols, and unresolved glyphs mint merge-ready entries so every emitted sym reference resolves. lintMathCoherence re-parses, re-lowers, and compares every stored presentation against its stored content layer -- divergence is a warning carrying provenance under the schema's atomic pair-edit rule, never an automatic re-derivation.
…he read path markdown-codec recognises $$ display blocks and \( \) inline spans but deliberately carries them through as raw LaTeX text; this pass (the model-level placement #563 settled on, so every format that can carry LaTeX benefits from one lowering) turns them into embedded formula blocks carrying presentation, lowered content, presentation-MathML from the same parse, and provenance -- display blocks replace their MathBlock paragraph, inline spans leave their paragraph and follow it (the odf and docx inline-equation position convention), and an all-math paragraph is consumed by its formula. A formula that cannot parse keeps its verbatim presentation with an empty MathML array, and the plain-text stand-in gains the presentation-LaTeX fallback so untypesettable math still renders as its own source text rather than a bare marker. The write side reconstructs real markdown math from formula blocks carrying a presentation layer (display $$ blocks, or an inline marker run when provenance says the span arrived inline), so the verbatim string round-trips; formulas with no LaTeX keep the stand-in flatten. The document symbol table is seeded from its own prose before lowering, so a formula referencing a prose-defined glyph resolves to the curated entry. lowerLatex/latexToFormula/lintMathCoherence are exported as the direct entry points for callers holding LaTeX, and the workerd suite proves the pinned temml parser runs in a Workers isolate.
Code Coverage OverviewLanguages: TypeScript TypeScript / unitThe overall coverage in commit 0e593a9 in the Show a code coverage summary of the most impacted files.
|
Contributor
|
🎉 This PR is included in version 2.3.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #572.
Lowers LaTeX presentation into the
MathExpressionsemantic core document-schema.js 3.2 defined, settling the transform-placement question from #563: parsing happens at the format edge, lowering at the model level, so every input format that can carry LaTeX shares one implementation.Pipeline:
"temml": "0.13.4", no caret. The pin is the point, not tidiness: the lowering consumes temml's internal parse API, which carries no stability guarantee across releases, and the storage contract says a stored presentation string has one defined parse. Bumping the pin means re-runningsrc/latex/lower.test.ts, whose table cases pin the parse-node shapes this version produces. temml is pure JavaScript, its parser never touches the DOM, andtest/workers/latex.test.tsproves the whole lowering path under workerd.\frac→math:divide, radicals →math:sqrt(or an exact1/nexponent for\sqrt[n]), a scripted Sigma/Product with limits → asum/prodbinder owning the rest of its term, digit runs → lowest-terms exact rationals (3.14→157/50, BigInt-exact at any length), subscripts → distinct symbol identities through the table (x_1is neverxtimes1), superscripts →math:powunless the table already curates the scripted form as one symbol, named functions (\sin,\ln, ...) consume their argument the way binders consume their summand.mc^2,f(x),2(x+1)— multiplication and function application are both defensible readings and LaTeX cannot say which), unmapped operators (\pm,\approx), integrals (the grammar's binders are exactly sum and prod),\textprose, compound subscripts (a_{i+1}), binomials,align/cases— each becomes anunparsednode carrying the verbatim source span plus a named diagnostic fromLATEX_DIAGNOSTIC_CODES. Visible data, curable later through the round-trip-safe semantic editing the schema defines; never a parse failure, never a silent guess.symreference resolves. The markdown read pass seeds the table from the document's own prose before lowering, so a formula referencing a prose-defined glyph resolves to the curated entry.lintMathCoherence(pkg)re-parses and re-lowers every stored presentation string against the document's own symbol table and compares with the stored content layer (canonicalised: sorted keys, rationals compared by reduced value). Divergence is a warning carrying provenance — somebody edited one layer deliberately under the schema's atomic pair-edit rule — and the lint re-derives nothing.readMarkdownContentlowers markdown-codec's preserved$$display blocks and\( \)inline spans into embedded formula blocks carrying presentation + content + presentation-MathML from the same parse + provenance. Real consequences:markdownToPdftypesets math through the STIX engine instead of rendering raw LaTeX text,markdownToDocxwrites real editable OMML,markdownToOdtwrites real embedded formula sub-documents, and the write side reconstructs the same markdown math syntax from the verbatim presentation layer so the string round-trips. Direct entry points for callers holding LaTeX:lowerLatex,latexToFormula; the lint is exported beside them.Supersedes #81 (standalone LaTeX→MathML translator): LaTeX parses at the format edge through the pinned temml and lowers to the semantic core defined by document-schema.js #15, from which MathML rendering derives.
Generated by Claude Code