mdl 1: strict parsing (property keys, ';', no '/', '' escapes); trailing commas everywhere (#732)

[ako]

Closes #732. Plan item 2.2 of PROPOSAL_mdl_beta_syntax_freeze.md; implements ADR-0010 R11 behind the mdl 1; header (ADR-0011). Part of #714.

What changes

Item	Under mdl 1;	Without the header (mdl 0)
1. Property keys	In the REST client (service, basic auth, operation), published REST service, business event service, model, knowledge base, consumed MCP service, agent and agent body blocks: an unknown key, or a value its key does not take (Response: json from $X, Body: status as $Y, Path: 42, Model: GPT), is an error that names the key, the list, the known keys and the key it most likely meant.	Read as before (ignored, or read by its shape); MDL-V1-PROP / MDL-V1-PROPVALUE warnings.
2. Terminators	A statement without ; is an error, and so is the SQL*Plus / line.	Accepted; MDL-V1-SEMI / MDL-V1-SLASH.
3. Escapes	'' is the only escape; a backslash is an ordinary character ('C:\temp' is that path, 'C:\' is complete).	Unchanged (\n \r \t \\ \' are escapes); MDL-V1-ESCAPE for each literal whose value would change.
4. Trailing commas	Allowed in every bracketed list.	Same (additive).

mdl 1 stays a preview (langver.Frozen = V0), so no headerless script changes behaviour.

Design choices the ADRs did not settle

• Trailing commas are one lexer rule, not COMMA? in ~70 list rules. I did the grammar version first (a script added COMMA? after every bracketed (COMMA x)*). It made every list's loop decision LL(2), so an unknown item after a comma was reported as no viable alternative at input ',Height' instead of expecting {…}, which lost the annotation-property hints (Feature request: expose annotation HEIGHT in the domain-model annotation surface mendixlabs/mxcli#1014) and would degrade every list's errors. The lexer version (TRAILING_COMMA: ',' {p.isTrailingComma()}? -> skip) drops a comma whose next significant character (skipping whitespace and comments) closes (), {} or [], and not one that follows an opener or another comma, so () stays the only empty list and (,) / (a,,) stay errors. It also covers expression argument lists; f(a,) means f(a).
• The escape rule is decided before lexing. It changes where a literal ends, so the lexer needs it. langver.ScanHeader reads the header from the source (same trivia rules as the grammar). An mdl 1 script is lexed from a parser.StrictEscapeStream (declared in the lexer's @members, since mdl/grammar/parser is generated), which a predicate on STRING_LITERAL checks. Every token keeps its stream, so the visitor reads each literal under the rule it was lexed with: the 242 unquoteString(x.GetText()) calls became unquoteStringLit(x) mechanically, and TestStringLiteralsAreReadUnderTheirEscapeRule fails on any new unquoteString( call in the visitor. The alternatives were worse: a package-level flag is not safe for concurrent Builds, and rewriting token text would double backslashes in the places that pass raw token text through.
• Terminators are read off the statement's last two tokens, not its own SEMICOLON/SLASH: the microflow, nanoflow and workflow rules end in SEMICOLON? SLASH? themselves, and create java action … as $$…$$; in SEMICOLON?. I found this by running PedApp's describe output through check under mdl 1;.
• Schemas list what each visitor reads, plus what describe writes. For example, the agent tool block accepts ToolType and Document, which describe writes and create ignores (see follow-ups). Shapes follow what the visitor actually reads: the model's Provider takes a name only, because a string there was dropped, while the knowledge base's reads either.
• Did-you-mean moved from the executor's OData check into mdl/suggest (refactor commit). It gained a two-edit fallback that counts a swap of neighbouring letters as one edit, for names of five letters or more (Verison, Passwd, Enabeld). The OData did-you-mean gets this too.
• Codes: MDL-V1-SEMI, MDL-V1-SLASH, MDL-V1-ESCAPE, MDL-V1-PROP, MDL-V1-PROPVALUE, declared as langver.Changes next to their code, as Microflow list operations as one statement per Studio Pro activity; set mandatory (mdl 1) #733 and create or modify as the one idempotent create; view-entity identity carry; if not exists everywhere; describe emits create or modify #731 do.
• Under mdl 1 a \' that ends a literal gets a hint naming the doubled apostrophe, because the parse errors that follow point elsewhere.

Test plan (what I ran)

• make build; make lint (Go + TS), both passed.
• go test ./mdl/visitor/ ./mdl/executor/ ./mdl/grammar/ ./mdl/langver/ ./mdl/suggest/ ./cmd/mxcli/syntax/, plus (lexer change) ./cmd/mxcli/ ./cmd/mxcli/testrunner/ ./mdl/formatter/ ./mdl/linter/... ./mdl/repl/ ./mdl/deprecation/: all pass.
• go test -tags integration ./mdl/roundtrip/: passes. No allowlist entries added or struck.
• New tests. Parse tests run under both versions; exec tests run ValidateProgram and ExecuteProgram under mdl 0 and Build under mdl 1:
  • mdl/visitor/trailing_comma_test.go: 17 lists, each checked under both versions to build the same AST as the form without the comma, plus the (,)/(a,,) controls.
  • mdl/visitor/strict_terminators_test.go: missing ;, the / line, a rule that ends in its own ;, and end; + / on a microflow. Terminated scripts are the controls.
  • mdl/visitor/string_escapes_test.go: values under both versions, 'C:\data' and 'it''s' as controls, where a literal ends, warning line, the source guard, and the hint.
  • mdl/visitor/strict_properties_test.go: 15 unknown or mis-shaped cases across all listed documents. The control accepts every key and shape each list reads, and describe's forms, under mdl 1 with no warning. mdl 0 keeps Response: json from $X as the body.
  • mdl/executor/strict_parsing_test.go: each item under both versions through check and exec. mdl 0 warns with the code and runs every statement; mdl 1 refuses with the item's message; exec stores C:<tab>emp under mdl 0 and C:\temp under mdl 1.
  • mdl/langver TestScanHeader, mdl/suggest TestClosest.
• Revert checks (each change removed, its tests seen failing, then restored):
  • Trailing commas: the old lexer made 14 of the 36 subtests fail. Published REST with a trailing comma even panics in the visitor on the error-recovered tree.
  • Terminators: renaming ExitStatement failed the visitor tests and 6 executor subtests. The first implementation (statement-level SEMICOLON/SLASH) fails TestTerminatorsOfRulesThatEndInTheirOwn.
  • Escapes, three separate reverts:
    • the old STRING_LITERAL rule fails "where a literal ends";
    • forcing unquoteStringLit to mdl 0 fails the mdl 1 values;
    • skipping noteBackslashEscapes fails every mdl 0 warning assertion.
  • Properties: an early return in checkProperty made 16 visitor subtests and 4 executor subtests fail.
• By hand: the proposal's "Before" script passes check with 5 warnings (SEMI, SLASH, SEMI, ESCAPE, PROP did you mean 'Path') and is refused under mdl 1;.
• No Studio Pro check: nothing here changes what is written to a model, only what parses and what a string literal's value is under mdl 1.

Follow-ups (not in this PR)

• describe output is refused under mdl 1. It writes a / line after about 20 document types, and no ; after a page's }. Under mdl 0 that output now warns MDL-V1-SLASH / MDL-V1-SEMI. This belongs to canonical describe (2.6) or fmt --upgrade (1.3), and must land before describe emits the header.
• The agent tool block's ToolType and Document, which describe writes, are ignored by create agent, so a microflow tool loses its document on a round trip. Separately, create agent resolves a tool's document only when ToolType == "mcp", but the visitor sets "MCP".
• Strict keys for the other property lists (R11 "then everywhere"): OData already refuses (MDL-ODATA01); pages and widgets, settings, and alter … set lists do not.
• QuoteString (visitor) still writes \\ for a backslash. That is right for the headerless MDL it builds, and must change once generated MDL carries mdl 1;.

🤖 Generated with Claude Code