From 8f08e2298f31d6225313abb6fd79864ceee479cd Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 23 Sep 2026 11:51:52 +0000 Subject: [PATCH 01/47] docs(proposal): implementation plan for first-class expressions (#750) Extend PROPOSAL_first_class_expressions.md with a grounded plan: slot inventory with the grammar rule and describer for each, four slices with file tables, a test plan (T1-T8), BSON/version section. Corrections from reading the grammar: - the "expression must not consume , or )" cost is already solved twice in widgetPropertyV3; the real cost is value-form ambiguity - calculated attributes and REST/OData mappings are not expression slots; pluggable expression props and workflow due date are - `dynamicclasses: [ ... ]` (the syntax #750 proposes) already passes check and is dropped as a []string; split out as slice 0 bug fix Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../PROPOSAL_first_class_expressions.md | 243 ++++++++++++++++-- 1 file changed, 220 insertions(+), 23 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_first_class_expressions.md b/docs/11-proposals/PROPOSAL_first_class_expressions.md index 066de42c44..a5cac1f968 100644 --- a/docs/11-proposals/PROPOSAL_first_class_expressions.md +++ b/docs/11-proposals/PROPOSAL_first_class_expressions.md @@ -2,6 +2,7 @@ title: First-class expressions for expression-typed MDL properties status: draft date: 2026-09-08 +revised: 2026-09-23 related: - https://github.com/mendixlabs/mxcli/issues/750 - PROPOSAL_expression_type_checking.md @@ -55,7 +56,8 @@ where Distance > 0 and contains($L/Name, 'abc'); So the machinery exists and is used. The defect is narrower and more tractable: **specific properties are declared as generic quoted-string slots** while the expression grammar sits unused beside them. `dynamicclasses` is handled by name -in `mdl/backend/pagemutator/mutator.go:2526` as an ordinary string property; it +in `mdl/backend/pagemutator/mutator.go` (`case "dynamicclasses"` in +`setRawWidgetPropertyMut`) as an ordinary string property; it never meets an expression rule at all. ## 2. There are two expression families, not one @@ -100,10 +102,20 @@ visitor. That is the shape of every XPath-family slot. Brackets here would be actively misleading: `[…]` reads as XPath everywhere else in MDL, and this is not XPath. - The cost is real and worth stating: an undelimited expression has to end - somewhere, and inside a `(key: value, …)` property list that means the - expression grammar must not consume the `,` or `)`. This is the one place the - work is more than additive, and it is why the XPath half should ship first. + An undelimited expression has to end somewhere, and inside a + `(key: value, …)` property list that means the expression grammar must not + consume the `,` or `)`. *(Revised 2026-09-23 after reading the grammar.)* This + is already solved in the same property list, twice: `dataSourceExprV3` + embeds a bare `expression` after `where` and is followed by `, sort by …` or + the next property, and `propertyValueV3`'s array alternative is + `LBRACKET expression (COMMA expression)* RBRACKET`. `expression` has no + top-level `COMMA` or `RPAREN` production — commas occur only inside + `LPAREN … RPAREN` of a call — so it stops at the property separator by + construction. The remaining cost is ambiguity, not termination: a bare + expression overlaps the other value forms (`'text'`, `42`, `true`, + `Mod.Enum.Value` are all valid expressions). §6.2 resolves that by giving the + expression alternative only to the named slots, never to the generic + `IDENTIFIER COLON propertyValueV3`. ### 3.2 Backward compatibility @@ -139,14 +151,34 @@ in any slice's acceptance criteria so it is not mistaken for a round-trip bug. ### 3.4 Scope -Ship in family order, XPath first because it is additive: - -1. **XPath family** — audit for slots still taking a quoted constraint. `sync … - where` is done; `targeting users xpath` (#1006's slot) is the obvious next. -2. **Expression family, single-value slots** — `dynamicclasses`, - `DynamicCellClass`. Highest-value by the count in §1, and the ones #750 names. -3. **Expression family, inside property lists** — page-variable defaults, - calculated attributes. Needs §3.1's termination question settled first. +*(Revised 2026-09-23 against the grammar; the original list named two slots +that are not expression slots and missed three that are.)* + +Ship in family order, XPath first because it is additive. The full slot +inventory, with the rule each lives in today, is §6.1. + +0. **Close the silent drop first** (§6.2, slice 0). The syntax #750 proposes, + `dynamicclasses: [ … ]`, is *already accepted* and throws the value away. + This is a bug fix, not part of the feature, and should not wait for it. +1. **XPath family** — `targeting users|groups xpath` (#1006's slot). `sync … + where` is done. +2. **Expression family, widget slots** — `DynamicClasses`, `DynamicCellClass`, + and every pluggable-widget property whose schema type is `expression`. + Highest-value by the count in §1, and the ones #750 names. +3. **Expression family, other statements** — page/snippet variable defaults, + workflow `due date`, `ALTER PAGE SET DynamicClasses = …`. + +Two of #750's targets are **not** expression slots and drop out: + +- **Calculated attributes.** MDL writes them `calculated by Mod.Microflow` + (`MDLDomainModel.g4`, `attributeConstraint`) — Mendix computes the value with + a microflow, and no expression is stored. Attribute `default` already takes + `literal | expression`. +- **REST/OData "filter and mapping expressions".** No such slot exists in + `MDLService.g4`: REST `Path:` / `Body: template` are `{param}` text templates + with their own escaping, and OData/REST mappings bind attributes by name. If a + real expression slot turns up there it joins slice 3; nothing is designed for + it speculatively. ## 4. What this unlocks @@ -157,15 +189,180 @@ the executor it is a string that was never parsed. Converting a slot to the first-class form is therefore the precondition for checking it, and the two proposals compose rather than compete. -## 5. Open question +## 5. Open questions + +1. **Whitespace inside string literals is not safe to normalise, and at least + one code path did.** Folding a multi-line constraint with `strings.Fields` + collapses runs of whitespace *inside* quoted literals too, so + `'two spaces'` silently becomes `'two spaces'` — a change to the value being + matched on, in a place nobody would look. Found and fixed in the offline-sync + describer, where the fold is now quote-aware. Whether any other expression or + XPath path folds or normalises whitespace without tracking quote state is + **unaudited**, and it is the kind of defect that leaves no trace: the document + stays valid and the build stays green. + + A sibling found while writing §6: `buildPropertyValueV3`'s array branch and + `buildMicroflowArgV3` both use ANTLR `expr.GetText()`, which concatenates + tokens *without* the hidden-channel whitespace — `if $x then 'a' else ''` + becomes `if$xthen'a'else''`. Literals survive (one token each); keywords and + operators fuse. Every new slot must go through `buildExpression` → + `expressionToString`, never `GetText()`. Whether the microflow-argument path + is live-broken for `if`-expressions is untested. + +2. **Does `expressionToString` round-trip Studio Pro's spelling?** A stored + `if $currentObject/Featured then 'x' else ''` re-emitted through + parse → `expressionToString` may differ in whitespace, parenthesisation or + keyword case. That is harmless to Mendix, but it is a *write* on the next + `exec` of unchanged `describe` output, which ADR-0008's idempotence rule + treats as churn. Needs one measurement per slot (§7, T4) before slice 2 + merges; if it churns, store the source text of the parsed span + (the token-stream text between `GetStart()` and `GetStop()`, which keeps + whitespace) rather than the re-rendered AST. + +3. **`describe` fallback when a stored value does not parse.** A project can + hold an expression our grammar does not accept (a newer function, a + Studio-Pro-only construct). `describe` must then emit the quoted form rather + than bare text that its own output cannot re-parse. Proposed rule: emit bare + only if the stored string parses as `expression` *and* re-renders to itself; + otherwise quote. Confirm this output variance is acceptable. + +4. **Pluggable expression properties are resolved by schema, not by name.** + Slice 2 either (a) lets any generic property take a bare expression and + rejects it at check time when the widget schema says the slot is not + `expression`-typed, or (b) adds the bare form only for the named properties + and leaves pluggables quoted. (a) is recommended — it follows the precedent + of the datasource/action generic branches (#956), where the grammar admits + the form and the executor decides by the declared kind — but it is the one + place this change widens a generic rule, so it wants a maintainer decision. + +## 6. Implementation plan + +### 6.1 Slot inventory + +| Slot | Family | Grammar today | Describer today | +|---|---|---|---| +| widget `DynamicClasses:` | expression | generic `IDENTIFIER COLON propertyValueV3` (`MDLPage.g4`, `widgetPropertyV3`) | `cmd_pages_describe_output.go` — `mdlQuote(w.DynamicClasses)` | +| datagrid column `DynamicCellClass:` | expression | generic, aliased to schema key `columnClass` (`mdl/types/widget_item_aliases.go`) | `cmd_pages_describe_output.go` — `mdlQuote(col.DynamicCellClass)` | +| pluggable property of schema type `expression` | expression | generic | `cmd_pages_describe_pluggable.go` | +| `ALTER PAGE SET DynamicClasses = … ON w` | expression | `alterPageAssignment` (`MDLParser.g4`) | n/a | +| page/snippet `Variables: { $v: T = '…' }` | expression | `variableDeclaration: VARIABLE COLON dataType EQUALS STRING_LITERAL` | `cmd_pages_describe.go` — `mdlQuote(defaultVal)` | +| workflow / user-task `due date '…'` | expression | `DUE DATE_TYPE STRING_LITERAL` (`MDLWorkflow.g4`) | `cmd_workflows.go` — `mdlQuoted(DueDate)` | +| user task `targeting users/groups xpath '…'` | XPath | `TARGETING … XPATH STRING_LITERAL` | `cmd_workflows.go` — `mdlQuoted(us.XPath)` | +| offline `sync … where` | XPath | `WHERE (xpathConstraint \| STRING_LITERAL)` | **done** | +| widget `Visible:` / `Editable:` | XPath-shaped | `xpathConstraint` | **done** | + +Explicitly not yet in: workflow `timer '…'`, `decide by veto '…'`, +`fallback '…'` and `description '…'`. Verify the stored kind of each against the +reflection data before adding one — a timer delay and a veto outcome are not +obviously expressions, and adding a slot on a guess is how #750's target list +went wrong. + +### 6.2 Slices + +Each slice is one PR (CLAUDE.md "Scope & atomicity"). + +**Slice 0 — bug: `[ … ]` on an expression slot is accepted and dropped.** + +Measured on `d34c6803`: -**Whitespace inside string literals is not safe to normalise, and at least one -code path did.** Folding a multi-line constraint with `strings.Fields` collapses -runs of whitespace *inside* quoted literals too, so `'two spaces'` silently -becomes `'two spaces'` — a change to the value being matched on, in a place -nobody would look. Found and fixed in the offline-sync describer, where the fold -is now quote-aware. +```mdl +container c1 (dynamicclasses: [ if $currentObject/Featured then 'is-featured' else '' ]) { } +``` + +`mxcli check` → `Syntax OK … Check passed!`. The value parses as +`propertyValueV3`'s array alternative, `buildPropertyValueV3` returns a +`[]string`, and both readers discard it: `WidgetV3.GetDynamicClasses` → +`GetStringProp` returns only a `string`, and `datagrid_column.go` (`columnClass`) +guards with `if sv, isStr := v.(string)`. By reading, the widget is written with +no dynamic class — the #999 shape (checks clean, exec succeeds, value vanishes). +Confirm on a project with `exec` + `describe` before fixing. + +Fix: a check-time violation (next free `MDL-WIDGET` id) when an expression-typed +property (`expressionWidgetProps` in `validate_widgets.go`, plus +`DynamicCellClass`) holds a non-string value, naming the fix. Test first in +`mdl/executor/`; prove it by reverting the check. Append a finding to +`.claude/skills/fix-issue/findings/.jsonl`. + +**Slice 1 — XPath family: `targeting … xpath [ … ]`.** + +| File | Change | +|---|---| +| `mdl/grammar/domains/MDLWorkflow.g4` | `TARGETING (USERS \| GROUPS)? XPATH (xpathConstraint \| STRING_LITERAL)` | +| `mdl/visitor/` (workflow visitor) | bracket branch → `normalizeXPathTokens(buildXPathString(…))`, as `sync … where` does | +| `mdl/executor/cmd_workflows.go` | emit `xpath [ … ]` instead of `mdlQuoted(us.XPath)`, users and groups | +| `mdl-examples/doctype-tests/24-workflow-examples.mdl` | bracket form beside the quoted form | + +**Slice 2 — expression family, widget slots.** + +| File | Change | +|---|---| +| `mdl/grammar/domains/MDLPage.g4` | `widgetPropertyV3`: add `(IDENTIFIER \| keyword) COLON expression` **after** every existing generic branch, so `'text'`, numbers, booleans, qualified names and `[ … ]` keep their current parse and only what those reject (`if …`, `$v/Attr + …`, calls) reaches it. `make grammar`; watch for new ambiguity reports. | +| `mdl/visitor/visitor_page_v3.go` | new branch storing an `ast.ExpressionValue` built with `buildExpression(ctx)` under the property name — never `GetText()` (§5.1) | +| `mdl/ast/ast_page_v3.go` | `ExpressionValue` type; `GetStringProp` renders it, so every existing reader (`GetDynamicClasses`, column props) gets the same string it gets today | +| `mdl/executor/cmd_microflows_helpers.go` | move `expressionToString` to where `mdl/ast` can use it without importing `executor` | +| `mdl/backend/widgetobj/datagrid_column.go` | accept `ExpressionValue` for `DynamicCellClass` | +| `mdl/executor/validate_widgets.go` | bare expression on a property whose resolved schema type is not `expression` → violation (§5.4 option a) | +| `mdl/executor/cmd_pages_describe_output.go`, `cmd_pages_describe_pluggable.go` | emit bare when §5.3's rule allows, else `mdlQuote` | +| `.claude/skills/mendix/` (page, datagrid, styling skills) | rewrite the §1 quote runs in the bare form; `make sync-skills` | + +**Slice 3 — expression family, other statements.** + +| File | Change | +|---|---| +| `mdl/grammar/domains/MDLPage.g4` | `variableDeclaration: VARIABLE COLON dataType EQUALS (STRING_LITERAL \| expression)` — a lone `STRING_LITERAL` keeps its legacy meaning (§6.3) | +| `mdl/grammar/domains/MDLWorkflow.g4` | `DUE DATE_TYPE (STRING_LITERAL \| expression)` in workflow and user-task clauses | +| `mdl/grammar/MDLParser.g4` | `alterPageAssignment`: expression alternative, validated against the property's type | +| matching visitors, `cmd_pages_describe.go`, `cmd_workflows.go` | as slice 2 | + +### 6.3 The one semantic trap: a quoted value keeps meaning "expression text" + +For every slot above, `'…'` today means "the Mendix expression, quoted". After +the change a `STRING_LITERAL` in these slots still means that — not a Mendix +string literal: + +```mdl +dynamicclasses: 'if $x then ''a'' else ''''' -- quoted expression (legacy) +dynamicclasses: if $x then 'a' else '' -- bare expression (new), same bytes +dynamicclasses: 'a' -- quoted expression: the text a (!) +``` -Whether any other expression or XPath path folds or normalises whitespace -without tracking quote state is **unaudited**, and it is the kind of defect that -leaves no trace: the document stays valid and the build stays green. +The last stores `a` before and after this change, which Mendix reads as an identifier, not the class `a`. +That is today's behaviour; the plan does not make it worse, and the +skills showing the bare form is what steers people off it. Recorded here so no +reviewer "fixes" the grammar by making a `STRING_LITERAL` a string value — that +would silently re-interpret every existing script. The ordering in slice 2 +(`propertyValueV3` before `expression`) is what enforces it. + +## 7. Test plan + +Every test follows CLAUDE.md "Working Rules": written first, proven by reverting +the fix, and any "nothing changed" assertion carries a control. + +| # | Layer | Test | +|---|---|---| +| T1 | parser | `mdl/visitor` table test per slot: bare form parses to `ExpressionValue`; `'text'`, `42`, `true`, `Mod.E.V`, `[a, b]` keep their current AST shape (guards slice 2's ordering) | +| T2 | parser | `(dynamicclasses: if $a then 'x' else '', class: 'c')` — the expression stops at `,` and `class` is still its own property | +| T3 | encoding | per slot, quoted and bare forms produce byte-identical stored strings (the §3.2 guarantee); control: a different expression does not | +| T4 | idempotence | `describe` → `exec` → `describe` is stable and the second `exec` writes nothing (`canon.Reconcile` elides); control: an edited expression does write. Settles §5.2 | +| T5 | describe | a stored value that does not parse is emitted quoted, and that output re-parses (§5.3) | +| T6 | check | slice 0: `[ … ]` on `DynamicClasses` / `DynamicCellClass` is a violation; control: the quoted expression on the same widget is clean | +| T7 | integration (`-tags integration`) | `03-page-examples.mdl`, `24-workflow-examples.mdl` gain bare-form cases; `mxcli docker check` passes with no CE0117 | +| T8 | runtime (`.claude/skills/verify-in-runtime.md`) | a page with a bare `dynamicclasses` renders the class on a featured object and not on another — the only test that proves the value reached the client | + +## 8. BSON and version compatibility + +**No BSON change.** Every slot already stores its expression as a string +(`Forms$Appearance.DynamicClasses`, the pluggable property's `Expression` field +built in `widgetobj/builder.go`, the page variable default, the workflow +`DueDate`, the user task's XPath). The feature changes only how MDL spells that +string and how `describe` prints it; the bytes written are identical by T3. No +new `$Type`, no storage-name question, no `canon.identityFields` row, and no +version gate: nothing about the encoding moves. + +## 9. Acceptance + +- Slice 0 merged independently, as a bug fix. +- For each converted slot: T1–T5 green; `describe` emits the bare/bracket form; + the quoted form still parses and stores the same bytes. +- `make build && make test && make lint` pass; the §1 count of four-plus quote + runs in the skills is re-run and is zero for converted slots. From cb1bbdcc0dc30f4d0917fd3f73ebdbfcea33d238 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 24 Sep 2026 14:28:25 +0000 Subject: [PATCH 02/47] fix: resolve association-path join alias written with uppercase AS MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit extractAliasMap matched `join a/Mod.Assoc/Mod.Entity AS x` case-insensitively but then recovered the path by trimming a literal lowercase "as" off the match. With `AS` (the spelling DESCRIBE prints) the path kept a trailing " AS", the end-anchored entity regex failed, and the alias was never mapped — so every column from it skipped type inference and a wrong pass-through string length passed `check --references`, failing the build with CE6770. The final path segment is now captured as its own regex group. Fixes #652 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_0154S1aRziq1CyrFepxqb5gg --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../652-oql-path-join-uppercase-as.mdl | 35 +++++++++++++++++++ mdl/executor/oql_alias_map_test.go | 33 +++++++++++++++++ mdl/executor/oql_type_inference.go | 17 +++++---- 4 files changed, 77 insertions(+), 9 deletions(-) create mode 100644 mdl-examples/bug-tests/652-oql-path-join-uppercase-as.mdl create mode 100644 mdl/executor/oql_alias_map_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 24a55faedb..f4d13a3ccd 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -688,3 +688,4 @@ {"date": "2026-09-23", "area": "mdl-executor", "symptom": "upstream #1176: DESCRIBE prints `all` on an import activity that returns ONE object — `$objectResponse = import from mapping M.IMM($s) all;` — which reads as a list import. Reported on v0.23.0 / Studio Pro 11.12.3, after #881 was believed to have settled import ranges", "cause": "#881 made `formatImportMappingRange` always emit a range keyword, because at the time a missing keyword let the range fall back to the variable's cardinality and store First. The later runtime fix (unauthored range written as All explicitly) made bare and `all` build the same activity, but the describe side was never revisited, so `all` kept printing where it was only noise", "file": "`mdl/executor/cmd_microflows_format_action.go` (`formatImportMappingRange`: return \"\" for All against SingleObject); tests `mdl/executor/cmd_microflows_import_range_test.go` (`TestImportRange_ObjectResultDescribesWithoutAll`); example `mdl-examples/bug-tests/1176-import-mapping-object-describes-without-all.mdl`", "insight": "**This was not #881 regressing — it was #881's own workaround outliving its reason.** 'DESCRIBE must never emit nothing' was a guard against the builder's then-broken default; once the builder wrote a missing keyword as All explicitly, the guard became pure noise, and nothing linked the two sites. When a formatter emits something 'because the builder would otherwise infer X', put that reason in a test that asserts the builder equivalence (bare vs keyword build the same activity), so fixing the builder flags the formatter. Proving the omission safe needs that equivalence on a real project, not just the unit test: on 11.12.3 both spellings store byte-identical ResultHandling (ConstantRange{SingleObject:false} + ObjectType), `mx check` 0 errors, and exec'ing the described text reports 'Unchanged microflow'. Wrong turn to skip: a JSON diff of two EMPTY extractions prints 'IDENTICAL' — `bson dump` emits ordered Key/Value lists, not objects; check the extraction is non-empty before trusting a diff", "refs": ["#881", "#1176"]} {"area": "mdl-executor", "date": "2026-09-24", "refs": ["#1173"], "symptom": "`ALTER ENTITY ADD ATTRIBUTE Region: String(200)` passed `mxcli check -p --references`, exec printed \"Added attribute 'Region' to entity MyFirstModule.SaleStats\" with exit 0, and `mx check` then failed CE6770 \"View Entity is out of sync with the OQL Query.\" The attribute was written as DomainModels$StoredValue with no OQL column behind it", "cause": "execAlterEntity's ADD/DROP ATTRIBUTE branches treat every entity as a table: nothing asked isViewEntity, although CREATE ASSOCIATION and bulk ALTER ENTITIES already did. The check-time AlterEntityStmt case only resolved the module and enumerations", "file": "`mdl/executor/cmd_entities.go` (viewEntityAttributeSetRefusal, AlterEntityAddAttribute/DropAttribute guards), `mdl/executor/validate.go` (validateViewEntityAttributeSet); test `mdl/executor/alter_entity_view_test.go`; bug-test `mdl-examples/bug-tests/1173-alter-view-entity-attribute.mdl`", "insight": "**Measure every ALTER op on a view before choosing the fix, not just the reported one** — one mxbuild per op on 11.12.1: ADD → CE6770, DROP → CE6770, MODIFY to the wrong type → CE6770 but MODIFY to the matching type → 0 errors (so MODIFY's failure is a type mismatch, a different gap, not this one), RENAME → 0 errors (the OqlViewValue binds the column by its Reference/alias, not the attribute name), SET COMMENT → 0. Refusing all four attribute ALTERs would have blocked a working RENAME. **Refuse rather than bind**: the issue offers \"create an OqlViewValue bound to the matching alias\", but for ADD there is no matching alias — the query has no such column — so any write stays CE6770; the query is the declaration, so the refusal points at `create or modify view entity`, verified to build clean with the added column. Guard both layers: check must see a view the SCRIPT creates (sc.viewEntities) as well as a stored one (findEntity + isViewEntity), or check passes a script exec stops halfway. Bulk ALTER ENTITIES already excluded views (e.Source/OqlQuery), so the single-entity path was the only entry. Control: stubbing both guards fails all four refusal tests with \"accepted — mxbuild reports CE6770\""} {"date": "2026-09-24", "area": "mdl-executor", "symptom": "upstream #1175: a `--` comment inside a view entity's select list produced false MDL030 — `select column 1 has no as alias: '-- the customer's running total'` plus a second one for the text after the comment's comma. Reported on v0.23.0 as an apostrophe bug", "cause": "Every static OQL check (ValidateOQLSyntax, ValidateOQLTypes, inferOQLTypes, viewAssociationColumns) works on `Query.RawQuery`, which is stored verbatim and so keeps the author's comments. parseSelectColumns splits on top-level commas with no notion of a comment, so the comment became a column and each comma in it another", "file": "`mdl/executor/oql_comments.go` (`stripOQLComments`, called at the top of the four entry points in `oql_type_inference.go` / `oql_view_associations.go`); tests `mdl/executor/validate_oql_comments_test.go`; example `mdl-examples/bug-tests/1175-oql-comment-is-not-a-select-column.mdl`", "insight": "**The apostrophe was a red herring: a comment with no apostrophe fails the same way** — measured before theorising, and it moved the fix from the quote-skipping in topLevelKeywordIndex to the comment itself. Strip at the entry points, not inside the helpers: the checks also run regexes over the whole query (division, association-path, reserved word), and a comment containing `from`, `/` or `a.B.C_D` would trip those too. Blank comments to spaces of the same length rather than deleting them, so any offset computed on the stripped text still indexes the original. Do NOT strip in the visitor — the stored query keeps the comments, which is the author's documentation. A type-check test using the same query passed without the fix (comment columns infer no type), so it was dropped rather than kept as a test that detects nothing", "refs": ["mendixlabs/mxcli#1175"], "rules": ["MDL030"]} +{"area":"mdl/executor","date":"2026-09-24","symptom":"`mxcli check -p … --references` passes a view entity whose association-path join is written `join s/Mod.A_B/System.UserRole AS r` (uppercase AS) even when a pass-through column from `r` declares the wrong string length; mxbuild then fails with CE6770 \"View Entity is out of sync with the OQL Query.\" Lowercase `as` reports MDL031 correctly.","cause":"extractAliasMap matched the path join case-insensitively ((?i)…(?:as\\s+)?) but then recovered the path from match[0] with strings.TrimSuffix(path, \"as\") — case-sensitive — so with `AS` the path kept a trailing ` AS`, the end-anchored lastEntity regex failed, and the alias was never mapped. Every column from that alias silently went without type inference.","file":"mdl/executor/oql_type_inference.go","insight":"A (?i) regex followed by string surgery on the whole match reintroduces case sensitivity by the back door: capture every piece you need as its own group instead of trimming it back out. The tell is a single control table varying only the case of one keyword — every other keyword in upper case was harmless, which points straight at code that handles that one token outside the regex. An unresolved alias is silent (the checker skips unknown types rather than reporting), so the symptom is a check that PASSES; test at extractAliasMap directly, and control with the unfixed binary on a real project (it printed `Check passed!` for AS, the error for as). DESCRIBE prints AS in upper case, so round-tripped OQL hits this by default.","refs":["#652"],"ce":["CE6770"],"rules":["MDL031"]} diff --git a/mdl-examples/bug-tests/652-oql-path-join-uppercase-as.mdl b/mdl-examples/bug-tests/652-oql-path-join-uppercase-as.mdl new file mode 100644 index 0000000000..d359833ea9 --- /dev/null +++ b/mdl-examples/bug-tests/652-oql-path-join-uppercase-as.mdl @@ -0,0 +1,35 @@ +-- Bug test for issue #652: an uppercase `AS` on an association-path join left +-- its alias unresolved, switching off type checks for its columns. +-- +-- Reported on Mendix 11.12.1: with `RoleName: String(200)` and +-- +-- join s/MyFirstModule.Sale652_UserRole/System.UserRole AS r +-- +-- `mxcli check v.mdl -p App.mpr --references` printed `Check passed!`, and +-- mxbuild then rejected the view with CE6770 "View Entity is out of sync with +-- the OQL Query." The same query with a lowercase `as` reported the MDL031 +-- length mismatch. DESCRIBE prints `AS` in upper case, so this is the ordinary +-- shape of round-tripped OQL. +-- +-- Cause: extractAliasMap recovered the path by trimming a literal lowercase +-- "as" off a case-insensitive match. The final path segment is now its own +-- regex group. +-- +-- Expected: this file passes `check --references` (String(100) is correct). +-- Changing RoleName to String(200) must report "inherits length 100 from +-- source attribute System.UserRole.Name" whichever case `as` is written in. + +create persistent entity MyFirstModule.Sale652 ( Amount: Integer ); + +create association MyFirstModule.Sale652_UserRole + from MyFirstModule.Sale652 to System.UserRole; + +create view entity MyFirstModule.RoleSales652 ( + RoleName: String(100), + Total: Integer +) as ( + select r.Name as RoleName, sum(s.Amount) as Total + from MyFirstModule.Sale652 as s + join s/MyFirstModule.Sale652_UserRole/System.UserRole AS r + group by r.Name +); diff --git a/mdl/executor/oql_alias_map_test.go b/mdl/executor/oql_alias_map_test.go new file mode 100644 index 0000000000..4a92ece997 --- /dev/null +++ b/mdl/executor/oql_alias_map_test.go @@ -0,0 +1,33 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import "testing" + +// TestExtractAliasMap_PathJoinAliasCase guards #652: an association-path join +// whose alias is introduced by an uppercase `AS` resolved to nothing, because +// the path was recovered from the match with a case-sensitive trim of "as". +// The alias then had no entity, every column from it went without type +// inference, and a pass-through length mismatch (CE6770) passed `check`. +// DESCRIBE of a Studio Pro model prints `AS` in upper case, so this is the +// ordinary shape of round-tripped OQL. +func TestExtractAliasMap_PathJoinAliasCase(t *testing.T) { + cases := []struct { + name, oql, alias, want string + }{ + {"lower as", `select r.Name from M.Sale as s join s/M.Sale_UserRole/System.UserRole as r`, "r", "System.UserRole"}, + {"upper AS", `select r.Name from M.Sale as s join s/M.Sale_UserRole/System.UserRole AS r`, "r", "System.UserRole"}, + {"mixed As", `select r.Name from M.Sale as s left join s/M.Sale_UserRole/System.UserRole As r`, "r", "System.UserRole"}, + {"no as", `select r.Name from M.Sale as s join s/M.Sale_UserRole/System.UserRole r`, "r", "System.UserRole"}, + {"quoted end, upper AS", `select o.Id from M.Line AS l JOIN l/M.Line_Order/M."Order" AS o`, "o", "M.Order"}, + {"multi-hop, upper AS", `select c.Name from M.Line AS l JOIN l/M.Line_Order/M.Order/M.Order_Customer/M.Customer AS c`, "c", "M.Customer"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := extractAliasMap(tc.oql) + if got[tc.alias] != tc.want { + t.Errorf("alias %q resolved to %q, want %q (map: %v)", tc.alias, got[tc.alias], tc.want, got) + } + }) + } +} diff --git a/mdl/executor/oql_type_inference.go b/mdl/executor/oql_type_inference.go index 0bed0832fe..886ae616d5 100644 --- a/mdl/executor/oql_type_inference.go +++ b/mdl/executor/oql_type_inference.go @@ -105,17 +105,16 @@ func extractAliasMap(oql string) map[string]string { // qualified name, so `m` resolved to nothing: no type inference for any of // its columns, and `m.ID` unrecognisable as an association column. That join // form is the ordinary way to reach a related entity in Mendix OQL. + // + // The final segment is captured as its own group. It used to be recovered + // by trimming the alias and a literal "as" off the match, which was + // case-sensitive while the pattern is not — so `... AS m`, the spelling + // DESCRIBE prints, left the alias unresolved (#652). pathPattern := regexp.MustCompile( - `(?i)\b(?:from|join)\s+[A-Za-z_]\w*(?:/` + oqlIdent + `\.` + oqlIdent + `)+\s+(?:as\s+)?([A-Za-z_]\w*)`) - lastEntity := regexp.MustCompile(`(` + oqlIdent + `\.` + oqlIdent + `)\s*$`) + `(?i)\b(?:from|join)\s+[A-Za-z_]\w*(?:/` + oqlIdent + `\.` + oqlIdent + `)*/(` + + oqlIdent + `\.` + oqlIdent + `)\s+(?:as\s+)?([A-Za-z_]\w*)`) for _, match := range pathPattern.FindAllStringSubmatch(oql, -1) { - alias := match[1] - // The path is everything between the keyword and the alias. - path := strings.TrimSuffix(strings.TrimSpace(match[0]), alias) - path = strings.TrimSpace(strings.TrimSuffix(strings.TrimSpace(path), "as")) - if seg := lastEntity.FindStringSubmatch(path); seg != nil { - aliasMap[alias] = unquoteQualifiedOQLName(seg[1]) - } + aliasMap[match[2]] = unquoteQualifiedOQLName(match[1]) } return aliasMap From cd5775c8d45066fc4ae6a1eb5a4efd4e75ec23a3 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 24 Sep 2026 23:46:25 +0000 Subject: [PATCH 03/47] feat: mxcli layout flows re-arranges microflows and nanoflows The flow counterpart of `mxcli layout`, and the alternative to a RESET LAYOUT clause on CREATE MICROFLOW (mendixlabs/mxcli#837): layout is a separate, opt-in operation on a stored flow, as it is for domain models. It reuses the one layout engine flows have. The stored flow is described to MDL, stripped of every layout annotation, and rebuilt exactly as CREATE builds it (dry-run, with a new ResetLayout build option so a hand-placed StartEvent is not carried over). Only the geometry of that build is kept: it is paired back onto the stored objects by walking both graphs from the start event, and patched into the stored BSON. Positions, sizes, connection indexes and bezier vectors change; $IDs, GUIDs and every property MDL cannot express do not. The write goes through UpdateRawUnit, so canon.Reconcile elides a no-op second run. The pairing doubles as the safety check: a flow whose description does not rebuild into the same graph is skipped with the reason instead of laid out by guesswork. A pass-through merge (one flow in, one out), which DESCRIBE omits, is placed on the rebuilt edge it sits on. Measured on the Studio Pro-drawn flows in testdata/expr-checker: 23 of 27 laid out, 4 skipped (branches sharing merges), second run writes nothing, mx check 11.6.6 reports 0 errors before and after. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012qfvyasDvxj5Zqrn4bxczi --- cmd/mxcli/cmd_layout.go | 4 +- cmd/mxcli/cmd_layout_flows.go | 210 +++++ cmd/mxcli/cmd_layout_flows_test.go | 112 +++ mdl/executor/cmd_microflows_build.go | 16 +- mdl/executor/cmd_microflows_layout.go | 939 +++++++++++++++++++++ mdl/executor/cmd_microflows_layout_test.go | 418 +++++++++ 6 files changed, 1697 insertions(+), 2 deletions(-) create mode 100644 cmd/mxcli/cmd_layout_flows.go create mode 100644 cmd/mxcli/cmd_layout_flows_test.go create mode 100644 mdl/executor/cmd_microflows_layout.go create mode 100644 mdl/executor/cmd_microflows_layout_test.go diff --git a/cmd/mxcli/cmd_layout.go b/cmd/mxcli/cmd_layout.go index 979532681f..6e821f54ca 100644 --- a/cmd/mxcli/cmd_layout.go +++ b/cmd/mxcli/cmd_layout.go @@ -51,7 +51,9 @@ what the new relationships require. This REPLACES the positions of every entity in the modules it touches, including any you arranged by hand. Use --dry-run to see the moves first. Marketplace -modules and System are never touched.`, +modules and System are never touched. + +To re-arrange microflows and nanoflows, use 'mxcli layout flows'.`, Example: ` mxcli layout -p app.mpr mxcli layout -p app.mpr --module CapTrack mxcli layout -p app.mpr --dry-run`, diff --git a/cmd/mxcli/cmd_layout_flows.go b/cmd/mxcli/cmd_layout_flows.go new file mode 100644 index 0000000000..f27c1dfc09 --- /dev/null +++ b/cmd/mxcli/cmd_layout_flows.go @@ -0,0 +1,210 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "fmt" + "io" + "os" + "slices" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/executor" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/spf13/cobra" +) + +// cmd_layout_flows.go re-arranges microflows and nanoflows — the flow +// counterpart of `mxcli layout`, and the answer to "reset the layout" that +// does not bolt a clause onto CREATE MICROFLOW. +// +// It uses the same layout engine as CREATE: a flow laid out here looks exactly +// like the same flow created from MDL without @position. Only coordinates are +// written — the flow's content, IDs and every property MDL cannot express are +// left as stored — which is also why it can lay out a flow drawn in Studio Pro +// without rewriting it through MDL. + +var layoutFlowsCmd = &cobra.Command{ + Use: "flows [Module.Flow ...]", + Short: "Re-arrange microflows and nanoflows with the CREATE MICROFLOW layout", + Long: `Re-arrange microflows and nanoflows on their canvas. + +The layout is the one CREATE MICROFLOW uses for a flow written without +@position: the main path runs left to right, branches fan out below it, loops +are sized to their bodies. Name the flows to lay out, or pass --module to lay +out every microflow and nanoflow in a module. + +Only positions change: activity and event positions and sizes, parameter +positions, and the anchors and curves of sequence flows. Content, element IDs +and everything else about the flow are left exactly as stored. + +A flow whose description does not rebuild into the same graph — something MDL +cannot yet express — is skipped with the reason, never laid out by guesswork. + +Running this twice changes nothing the second time. It REPLACES the positions +of every flow it touches, including any arranged by hand; use --dry-run to see +what would move first. Marketplace modules and System are never touched unless +--include-marketplace is given.`, + Example: ` mxcli layout flows -p app.mpr MyModule.ACT_Order_Submit + mxcli layout flows -p app.mpr --module MyModule + mxcli layout flows -p app.mpr --module MyModule --dry-run`, + RunE: func(cmd *cobra.Command, args []string) error { + return runLayoutFlows(cmd, args) + }, +} + +func init() { + layoutFlowsCmd.Flags().StringSliceVar(&layoutModules, "module", nil, + "module whose flows to lay out (repeatable)") + layoutFlowsCmd.Flags().BoolVar(&layoutDryRun, "dry-run", false, + "report what would move without writing") + layoutFlowsCmd.Flags().BoolVar(&layoutIncludeMarketplace, "include-marketplace", false, + "also lay out flows in Marketplace modules (a module update replaces them, so this is normally pointless)") + layoutCmd.AddCommand(layoutFlowsCmd) +} + +// flowTarget is one flow to lay out. +type flowTarget struct { + kind string + name ast.QualifiedName +} + +func runLayoutFlows(cmd *cobra.Command, args []string) error { + projectPath, _ := cmd.Flags().GetString("project") + if projectPath == "" { + return fmt.Errorf("no project given: pass -p ") + } + if _, err := os.Stat(projectPath); err != nil { + return fmt.Errorf("project not found: %s", projectPath) + } + if len(args) == 0 && len(layoutModules) == 0 { + return fmt.Errorf("name the flows to lay out, or pass --module: laying out every flow in the project is too broad to do by default") + } + + exec, logger := newLoggedExecutor("subcommand") + defer logger.Close() + defer exec.Close() + exec.SetQuiet(true) + connectProg, _ := visitor.Build(fmt.Sprintf("CONNECT LOCAL '%s'", visitor.QuoteString(projectPath))) + for _, stmt := range connectProg.Statements { + if err := exec.Execute(stmt); err != nil { + return err + } + } + + targets, err := flowLayoutTargets(exec, args) + if err != nil { + return err + } + return layoutFlowTargets(cmd.OutOrStdout(), exec, targets) +} + +// flowLayoutTargets resolves the named flows and the flows of every --module. +// A name the project does not have is an error, as a --module typo is in +// `mxcli layout`: otherwise it would report success having done nothing. +func flowLayoutTargets(exec *executor.Executor, args []string) ([]flowTarget, error) { + var targets []flowTarget + seen := map[string]bool{} + add := func(kind, qn string) { + if seen[qn] { + return + } + seen[qn] = true + mod, name, _ := strings.Cut(qn, ".") + targets = append(targets, flowTarget{kind: kind, name: ast.QualifiedName{Module: mod, Name: name}}) + } + + // Both --module and a named flow go through the Marketplace/System filter + // the domain model layout uses: naming a flow in a Marketplace module is as + // pointless as naming the module, since the next update replaces both. + modules, err := exec.Modules() + if err != nil { + return nil, err + } + if len(layoutModules) > 0 { + wanted := map[string]string{} + for _, m := range layoutModules { + if t := strings.TrimSpace(m); t != "" { + wanted[strings.ToLower(t)] = t + } + } + mods, err := layoutTargets(modules, wanted) + if err != nil { + return nil, err + } + for _, m := range mods { + mfs, nfs, err := exec.ListFlowNames(m.Name) + if err != nil { + return nil, err + } + for _, qn := range mfs { + add("microflow", qn) + } + for _, qn := range nfs { + add("nanoflow", qn) + } + } + } + + for _, arg := range args { + mod, name, ok := strings.Cut(strings.TrimSpace(arg), ".") + if !ok || mod == "" || name == "" { + return nil, fmt.Errorf("%q is not a qualified flow name (Module.Flow)", arg) + } + mods, err := layoutTargets(modules, map[string]string{strings.ToLower(mod): mod}) + if err != nil { + return nil, err + } + mod = mods[0].Name // Mendix resolves module names case-insensitively + mfs, nfs, err := exec.ListFlowNames(mod) + if err != nil { + return nil, err + } + qn := mod + "." + name + switch { + case slices.Contains(mfs, qn): + add("microflow", qn) + case slices.Contains(nfs, qn): + add("nanoflow", qn) + default: + return nil, fmt.Errorf("no microflow or nanoflow named %s", qn) + } + } + return targets, nil +} + +func layoutFlowTargets(out io.Writer, exec *executor.Executor, targets []flowTarget) error { + changed, unchanged, refused := 0, 0, 0 + for _, t := range targets { + res, err := exec.LayoutFlow(t.kind, t.name, layoutDryRun) + if err != nil { + return fmt.Errorf("%s %s: %w", t.kind, t.name, err) + } + switch { + case res.Refused != "": + refused++ + fmt.Fprintf(out, "%s: skipped — %s\n", res.Name, res.Refused) + case !res.Changed(): + unchanged++ + fmt.Fprintf(out, "%s: already laid out (%d objects)\n", res.Name, res.Objects) + case layoutDryRun: + changed++ + fmt.Fprintf(out, "%s: %d of %d objects and %d flows would move\n", res.Name, res.Moved, res.Objects, res.Flows) + default: + changed++ + fmt.Fprintf(out, "%s: moved %d of %d objects, re-anchored %d flows\n", res.Name, res.Moved, res.Objects, res.Flows) + } + } + + switch { + case len(targets) == 0: + fmt.Fprintln(out, "Nothing to lay out.") + case layoutDryRun: + fmt.Fprintf(out, "Dry run: %d flows would change, %d already laid out, %d skipped. Re-run without --dry-run to apply.\n", + changed, unchanged, refused) + default: + fmt.Fprintf(out, "%d flows laid out, %d already laid out, %d skipped.\n", changed, unchanged, refused) + } + return nil +} diff --git a/cmd/mxcli/cmd_layout_flows_test.go b/cmd/mxcli/cmd_layout_flows_test.go new file mode 100644 index 0000000000..0b6c0a170d --- /dev/null +++ b/cmd/mxcli/cmd_layout_flows_test.go @@ -0,0 +1,112 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "bytes" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// flowLayoutFixture connects an executor to a copy of the shared fixture. Its +// flows were drawn in Studio Pro; MyFirstModule is the project's own, while +// Administration and FeedbackModule come from the Marketplace. +func flowLayoutFixture(t *testing.T) (*bytes.Buffer, func(args ...string) error) { + t.Helper() + dst := t.TempDir() + if err := os.CopyFS(dst, os.DirFS("../../testdata/expr-checker")); err != nil { + t.Fatalf("copy fixture: %v", err) + } + exec, logger := newLoggedExecutor("test") + t.Cleanup(func() { logger.Close(); exec.Close() }) + exec.SetQuiet(true) + prog, _ := visitor.Build("CONNECT LOCAL '" + visitor.QuoteString(filepath.Join(dst, "minimal.mpr")) + "'") + for _, stmt := range prog.Statements { + if err := exec.Execute(stmt); err != nil { + t.Fatal(err) + } + } + out := &bytes.Buffer{} + return out, func(args ...string) error { + targets, err := flowLayoutTargets(exec, args) + if err != nil { + return err + } + return layoutFlowTargets(out, exec, targets) + } +} + +func withLayoutFlags(t *testing.T, modules []string, dryRun, marketplace bool) { + t.Helper() + prevM, prevD, prevI := layoutModules, layoutDryRun, layoutIncludeMarketplace + t.Cleanup(func() { layoutModules, layoutDryRun, layoutIncludeMarketplace = prevM, prevD, prevI }) + layoutModules, layoutDryRun, layoutIncludeMarketplace = modules, dryRun, marketplace +} + +func TestLayoutFlows_NamedFlowIsResolvedByKind(t *testing.T) { + withLayoutFlags(t, nil, true, true) + out, layout := flowLayoutFixture(t) + // One nanoflow and one microflow, the module spelled in the wrong case. + if err := layout("feedbackmodule.ACT_SubmitFeedback", "FeedbackModule.SUB_Feedback_Sanitize"); err != nil { + t.Fatal(err) + } + got := out.String() + for _, want := range []string{"FeedbackModule.ACT_SubmitFeedback:", "FeedbackModule.SUB_Feedback_Sanitize:", "Dry run: 2 flows would change"} { + if !strings.Contains(got, want) { + t.Errorf("output lacks %q:\n%s", want, got) + } + } +} + +// A typo must not report success having done nothing. +func TestLayoutFlows_UnknownFlowIsAnError(t *testing.T) { + withLayoutFlags(t, nil, true, false) + _, layout := flowLayoutFixture(t) + err := layout("MyFirstModule.NoSuchFlow") + if err == nil || !strings.Contains(err.Error(), "MyFirstModule.NoSuchFlow") { + t.Errorf("err = %v, want one naming the missing flow", err) + } +} + +// A flow in a Marketplace module is refused as the module itself is by +// `mxcli layout`, unless --include-marketplace says otherwise. +func TestLayoutFlows_MarketplaceFlowNeedsTheFlag(t *testing.T) { + withLayoutFlags(t, nil, true, false) + out, layout := flowLayoutFixture(t) + // Control: the project's own module needs no flag. + if err := layout("MyFirstModule.MyFirstLogic"); err != nil { + t.Fatalf("own module: %v", err) + } + if err := layout("Administration.ChangeMyPassword"); err == nil || !strings.Contains(err.Error(), "Marketplace") { + t.Errorf("err = %v, want a Marketplace refusal", err) + } + + withLayoutFlags(t, nil, true, true) + out.Reset() + if err := layout("Administration.ChangeMyPassword"); err != nil { + t.Fatalf("with --include-marketplace: %v", err) + } + if !strings.Contains(out.String(), "would move") { + t.Errorf("output: %s", out.String()) + } +} + +// A refused flow is reported and the batch carries on. +func TestLayoutFlows_ModuleBatchSkipsWhatDoesNotRoundTrip(t *testing.T) { + withLayoutFlags(t, []string{"FeedbackModule"}, true, true) + out, layout := flowLayoutFixture(t) + if err := layout(); err != nil { + t.Fatal(err) + } + got := out.String() + if !strings.Contains(got, "FeedbackModule.VAL_Feedback: skipped") { + t.Errorf("VAL_Feedback not reported as skipped:\n%s", got) + } + if !strings.Contains(got, "FeedbackModule.ACT_SubmitFeedback:") || !strings.Contains(got, "would move") { + t.Errorf("the batch did not carry on past the refusal:\n%s", got) + } +} diff --git a/mdl/executor/cmd_microflows_build.go b/mdl/executor/cmd_microflows_build.go index 0b9a9e68ae..1a0ea6ebd9 100644 --- a/mdl/executor/cmd_microflows_build.go +++ b/mdl/executor/cmd_microflows_build.go @@ -24,8 +24,13 @@ import ( // proposed flow must not touch the project, and a refusal that aborted the // build would leave the user with no diff at all rather than a diff plus the // warning exec will give them anyway. +// +// ResetLayout is for `mxcli layout flows`, which rebuilds a stored flow only for +// its geometry: nothing is carried over from the flow being replaced, not even a +// hand-placed StartEvent, so every position is the layout engine's own. type buildFlowOpts struct { AllowCreate bool + ResetLayout bool } // builtFlow is a Microflow assembled from a statement, plus what the write @@ -399,7 +404,7 @@ func buildMicroflowFromStmt(ctx *ExecContext, s *ast.CreateMicroflowStmt, opts b // pinned the start of every rewritten flow, stranding it across the // canvas from activities the same script had just moved (#951). An // explicit @start(x, y) on the first statement overrides both. - startPosition: storedStartPosition(ctx, existingID), + startPosition: carriedStartPosition(ctx, existingID, opts), posX: 200, posY: 200, baseY: 200, // Base Y for happy path @@ -749,3 +754,12 @@ func lookupFolder(ctx *ExecContext, moduleID model.ID, folderPath string) (model } return current, true } + +// carriedStartPosition is storedStartPosition unless the build is resetting the +// layout, which carries nothing over. +func carriedStartPosition(ctx *ExecContext, existingID model.ID, opts buildFlowOpts) *model.Point { + if opts.ResetLayout { + return nil + } + return storedStartPosition(ctx, existingID) +} diff --git a/mdl/executor/cmd_microflows_layout.go b/mdl/executor/cmd_microflows_layout.go new file mode 100644 index 0000000000..092c3260cf --- /dev/null +++ b/mdl/executor/cmd_microflows_layout.go @@ -0,0 +1,939 @@ +// SPDX-License-Identifier: Apache-2.0 + +// Package executor - `mxcli layout flows`: re-arrange an existing microflow or +// nanoflow with the same layout engine CREATE uses. +package executor + +import ( + "context" + "fmt" + "reflect" + "sort" + "strings" + + "go.mongodb.org/mongo-driver/bson" + "go.mongodb.org/mongo-driver/bson/primitive" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// Laying out a flow is a rebuild whose only product is coordinates. +// +// The layout engine is the flow builder: it places each activity while it +// creates it, so there is no separate pass to call on a stored graph. Rather +// than grow a second engine that works on stored graphs — and drifts from the +// first — the stored flow is described to MDL, stripped of every layout +// annotation, and built again exactly as CREATE builds it. That build is thrown +// away except for its geometry, which is paired back onto the STORED objects and +// patched into the stored BSON. +// +// Patching the stored bytes, rather than writing the rebuilt flow, is the point: +// every $ID, every GUID and every property MDL cannot express stays as it is. +// Only RelativeMiddlePoint, Size, the flows' connection indexes and their bezier +// vectors change. So a flow laid out here is laid out exactly as CREATE would +// lay out its DESCRIBE with the annotations removed, and nothing else about it +// moves. +// +// The pairing is also the safety check. The rebuilt graph must match the stored +// one object for object (same kind, same branch structure); if describe → build +// does not reproduce the stored graph, the flow does not round-trip through MDL +// and is refused rather than laid out by guesswork. + +// FlowLayoutResult reports what laying out one flow did. +type FlowLayoutResult struct { + Kind string // "microflow" or "nanoflow" + Name string // qualified name + Objects int // objects in the flow + Moved int // objects whose position or size changed + Flows int // sequence flows whose anchors or curve changed + Refused string // why the flow was left alone; empty when it was laid out +} + +// Changed reports whether laying the flow out changes anything. +func (r FlowLayoutResult) Changed() bool { return r.Moved > 0 || r.Flows > 0 } + +// LayoutFlow lays out one microflow or nanoflow (kind "microflow" or +// "nanoflow"). With dryRun, it computes the result and writes nothing. +// +// A flow that cannot be laid out safely is reported in Refused, not as an +// error: a batch over a module should skip it and carry on. +func (e *Executor) LayoutFlow(kind string, name ast.QualifiedName, dryRun bool) (FlowLayoutResult, error) { + return layoutFlow(e.newExecContext(context.Background()), kind, name, dryRun) +} + +// Modules lists the connected project's modules. +func (e *Executor) Modules() ([]*model.Module, error) { + ctx := e.newExecContext(context.Background()) + if !ctx.Connected() { + return nil, mdlerrors.NewNotConnected() + } + return ctx.Backend.ListModules() +} + +// ListFlowNames returns the qualified names of the live microflows and +// nanoflows in a module, sorted. +func (e *Executor) ListFlowNames(moduleName string) (mfs []string, nfs []string, err error) { + ctx := e.newExecContext(context.Background()) + h, err := getHierarchy(ctx) + if err != nil { + return nil, nil, mdlerrors.NewBackend("build hierarchy", err) + } + all, err := ctx.Backend.ListMicroflows() + if err != nil { + return nil, nil, mdlerrors.NewBackend("list microflows", err) + } + for _, mf := range all { + if !mf.Excluded && h.GetModuleName(h.FindModuleID(mf.ContainerID)) == moduleName { + mfs = append(mfs, moduleName+"."+mf.Name) + } + } + allNf, err := ctx.Backend.ListNanoflows() + if err != nil { + return nil, nil, mdlerrors.NewBackend("list nanoflows", err) + } + for _, nf := range allNf { + if !nf.Excluded && h.GetModuleName(h.FindModuleID(nf.ContainerID)) == moduleName { + nfs = append(nfs, moduleName+"."+nf.Name) + } + } + sort.Strings(mfs) + sort.Strings(nfs) + return mfs, nfs, nil +} + +func layoutFlow(ctx *ExecContext, kind string, name ast.QualifiedName, dryRun bool) (FlowLayoutResult, error) { + res := FlowLayoutResult{Kind: kind, Name: name.String()} + if !dryRun && !ctx.ConnectedForWrite() { + return res, mdlerrors.NewNotConnectedWrite() + } + + stored, err := storedFlow(ctx, kind, name) + if err != nil { + return res, err + } + rebuilt, err := rebuildFlowLayout(ctx, kind, name) + if err != nil { + res.Refused = err.Error() + return res, nil + } + + plan, err := pairFlowLayout(stored, rebuilt) + if err != nil { + res.Refused = err.Error() + return res, nil + } + res.Objects = plan.objectCount + + raw, err := ctx.Backend.GetRawUnitBytes(stored.id) + if err != nil { + return res, mdlerrors.NewBackend("read "+kind+" "+name.String(), err) + } + var doc bson.D + if err := bson.Unmarshal(raw, &doc); err != nil { + return res, mdlerrors.NewBackend("parse "+kind+" "+name.String(), err) + } + patched := plan.apply(doc) + res.Moved, res.Flows = plan.movedObjects, plan.changedFlows + if !res.Changed() || dryRun { + return res, nil + } + + out, err := bson.Marshal(patched) + if err != nil { + return res, mdlerrors.NewBackend("encode "+kind+" "+name.String(), err) + } + if err := ctx.Backend.UpdateRawUnit(string(stored.id), out); err != nil { + return res, mdlerrors.NewBackend("write "+kind+" "+name.String(), err) + } + return res, nil +} + +// layoutGraph is the part of a flow the layout pairing needs, the same for a +// microflow and a nanoflow. +type layoutGraph struct { + id model.ID + parameters []*microflows.MicroflowParameter + objects *microflows.MicroflowObjectCollection +} + +func storedFlow(ctx *ExecContext, kind string, name ast.QualifiedName) (*layoutGraph, error) { + h, err := getHierarchy(ctx) + if err != nil { + return nil, mdlerrors.NewBackend("build hierarchy", err) + } + inModule := func(containerID model.ID) bool { + return h.GetModuleName(h.FindModuleID(containerID)) == name.Module + } + switch kind { + case "microflow": + all, err := ctx.Backend.ListMicroflows() + if err != nil { + return nil, mdlerrors.NewBackend("list microflows", err) + } + mf, ok := pickLive(all, + func(m *microflows.Microflow) bool { return m.Name == name.Name && inModule(m.ContainerID) }, + func(m *microflows.Microflow) bool { return m.Excluded }) + if !ok { + return nil, mdlerrors.NewNotFound("microflow", name.String()) + } + return &layoutGraph{id: mf.ID, parameters: mf.Parameters, objects: mf.ObjectCollection}, nil + case "nanoflow": + all, err := ctx.Backend.ListNanoflows() + if err != nil { + return nil, mdlerrors.NewBackend("list nanoflows", err) + } + nf, ok := pickLive(all, + func(n *microflows.Nanoflow) bool { return n.Name == name.Name && inModule(n.ContainerID) }, + func(n *microflows.Nanoflow) bool { return n.Excluded }) + if !ok { + return nil, mdlerrors.NewNotFound("nanoflow", name.String()) + } + return &layoutGraph{id: nf.ID, parameters: nf.Parameters, objects: nf.ObjectCollection}, nil + } + return nil, fmt.Errorf("cannot lay out a %s", kind) +} + +// rebuildFlowLayout describes the stored flow, strips its layout annotations and +// builds it again the way CREATE would — without writing anything. +func rebuildFlowLayout(ctx *ExecContext, kind string, name ast.QualifiedName) (*layoutGraph, error) { + var mdl string + var err error + if kind == "nanoflow" { + mdl, _, err = describeNanoflowToString(ctx, name) + } else { + mdl, _, err = describeMicroflowToString(ctx, name) + } + if err != nil { + return nil, err + } + + prog, errs := visitor.Build(mdl) + if len(errs) > 0 { + return nil, fmt.Errorf("its description does not parse back (%v), so it does not round-trip through MDL", errs[0]) + } + + opts := buildFlowOpts{ResetLayout: true} + for _, stmt := range prog.Statements { + switch s := stmt.(type) { + case *ast.CreateMicroflowStmt: + stripFlowLayout(s) + built, err := buildMicroflowFromStmt(ctx, s, opts) + if err != nil { + return nil, fmt.Errorf("rebuilding it from its description failed: %w", err) + } + mf := built.Microflow + return &layoutGraph{id: mf.ID, parameters: mf.Parameters, objects: mf.ObjectCollection}, nil + case *ast.CreateNanoflowStmt: + stripFlowLayout(s) + built, err := buildNanoflowFromStmt(ctx, s, opts) + if err != nil { + return nil, fmt.Errorf("rebuilding it from its description failed: %w", err) + } + nf := built.Nanoflow + return &layoutGraph{id: nf.ID, parameters: nf.Parameters, objects: nf.ObjectCollection}, nil + } + } + return nil, fmt.Errorf("its description holds no %s definition", kind) +} + +// stripFlowLayout clears every layout annotation in a CREATE MICROFLOW or +// CREATE NANOFLOW statement, so the builder places everything itself. +// +// What goes is exactly what DESCRIBE emits to pin geometry: @position (on +// activities, parameters and notes), @anchor in all its forms, @curve, @merge and +// @start. A note keeps its size, which is its content's box rather than its +// placement; everything else in the statement is left alone. +// +// Reflective because statements nest bodies in a dozen places — branches, loops, +// splits' cases, error handlers — and a hand-written walk that missed one would +// leave that body pinned where it was. +func stripFlowLayout(stmt ast.Statement) { + clearLayoutAnnotations(reflect.ValueOf(stmt), map[uintptr]bool{}) +} + +var ( + activityAnnotationsType = reflect.TypeOf(&ast.ActivityAnnotations{}) + microflowNoteType = reflect.TypeOf(ast.MicroflowAnnotation{}) + microflowParamType = reflect.TypeOf(ast.MicroflowParam{}) +) + +func clearLayoutAnnotations(v reflect.Value, seen map[uintptr]bool) { + switch v.Kind() { + case reflect.Ptr: + if v.IsNil() || seen[v.Pointer()] { + return + } + seen[v.Pointer()] = true + if v.Type() == activityAnnotationsType { + ann := v.Interface().(*ast.ActivityAnnotations) + ann.Position = nil + ann.Anchor = nil + ann.TrueBranchAnchor = nil + ann.FalseBranchAnchor = nil + ann.IteratorAnchor = nil + ann.BodyTailAnchor = nil + ann.Curve = nil + ann.Merge = nil + ann.Start = nil + } + clearLayoutAnnotations(v.Elem(), seen) + case reflect.Interface: + if !v.IsNil() { + clearLayoutAnnotations(v.Elem(), seen) + } + case reflect.Struct: + if v.CanSet() { + switch v.Type() { + case microflowNoteType: + v.FieldByName("Position").Set(reflect.Zero(v.FieldByName("Position").Type())) + case microflowParamType: + v.FieldByName("Position").Set(reflect.Zero(v.FieldByName("Position").Type())) + } + } + t := v.Type() + for i := 0; i < v.NumField(); i++ { + if t.Field(i).IsExported() { + clearLayoutAnnotations(v.Field(i), seen) + } + } + case reflect.Slice, reflect.Array: + for i := 0; i < v.Len(); i++ { + clearLayoutAnnotations(v.Index(i), seen) + } + } +} + +// layoutPlan is the geometry to patch onto the stored flow, keyed by the stored +// element's normalised $ID. +type layoutPlan struct { + objects map[string]objectGeometry + flows map[string]flowGeometry + + objectCount int + movedObjects int + changedFlows int +} + +type objectGeometry struct { + position model.Point + size *model.Size // nil leaves the stored size alone (parameters, notes) +} + +type flowGeometry struct { + originIndex, destinationIndex int + originVector, destVector string +} + +// normID puts an element ID in the form both the typed model and the raw BSON +// reduce to: lower-case hex without dashes. +func normID(id string) string { + return strings.ToLower(strings.ReplaceAll(id, "-", "")) +} + +// pairFlowLayout matches the rebuilt flow to the stored one and returns the +// geometry to carry across. Any stored object the walk cannot pair is a refusal. +func pairFlowLayout(stored, rebuilt *layoutGraph) (*layoutPlan, error) { + if stored.objects == nil || rebuilt.objects == nil { + return nil, fmt.Errorf("it has no flow to lay out") + } + p := &flowPairing{ + stored: indexFlowGraph(stored.objects), + rebuilt: indexFlowGraph(rebuilt.objects), + objects: map[model.ID]model.ID{}, + flows: map[model.ID]model.ID{}, + } + if err := p.walk(); err != nil { + return nil, err + } + + plan := &layoutPlan{objects: map[string]objectGeometry{}, flows: map[string]flowGeometry{}} + for sid, bid := range p.objects { + so, bo := p.stored.object[sid], p.rebuilt.object[bid] + size := objectSize(bo) + g := objectGeometry{position: bo.GetPosition(), size: &size} + if _, isNote := so.(*microflows.Annotation); isNote { + g.size = nil + } + plan.objects[normID(string(sid))] = g + } + for sid, bid := range p.flows { + bf := p.rebuilt.flow[bid] + plan.flows[normID(string(sid))] = flowGeometry{ + originIndex: bf.OriginConnectionIndex, + destinationIndex: bf.DestinationConnectionIndex, + originVector: orZeroVector(bf.OriginControlVector), + destVector: orZeroVector(bf.DestinationControlVector), + } + } + + for _, m := range p.passThrough { + placePassThroughMerge(plan, m, p.rebuilt) + } + + // Parameters pair by name. One the rebuild placed without an annotation + // gets the position the writer derives from its index. + byName := map[string]model.Point{} + for i, bp := range rebuilt.parameters { + pos := microflows.DerivedParameterPosition(i) + if bp.Position != nil { + pos = *bp.Position + } + byName[bp.Name] = pos + } + for _, sp := range stored.parameters { + pos, ok := byName[sp.Name] + if !ok { + return nil, fmt.Errorf("its parameter $%s did not survive the rebuild", sp.Name) + } + plan.objects[normID(string(sp.ID))] = objectGeometry{position: pos} + } + plan.objectCount = len(p.objects) + len(p.passThrough) + len(stored.parameters) + return plan, nil +} + +// placePassThroughMerge puts a merge the rebuild does not have halfway along the +// rebuilt edge it sits on, and splits that edge's anchors between the two stored +// flows around it: the one in keeps the rebuilt flow's origin side, the one out +// keeps its destination side, and they meet at the merge on the side facing +// each. A chain of such merges shares one point, which is rare enough not to +// spread out. +func placePassThroughMerge(plan *layoutPlan, m passThroughMerge, rebuilt *flowGraphIndex) { + bf := rebuilt.flow[m.rebuilt] + from := rebuilt.object[bf.OriginID].GetPosition() + to := rebuilt.object[bf.DestinationID].GetPosition() + + // The edge enters its destination from the left or right (a horizontal + // run) or from the top or bottom (a vertical drop); the merge sits on the + // straight part leading in. + pos := model.Point{X: (from.X + to.X) / 2, Y: to.Y} + if bf.DestinationConnectionIndex == AnchorTop || bf.DestinationConnectionIndex == AnchorBottom { + pos = model.Point{X: to.X, Y: (from.Y + to.Y) / 2} + } + plan.objects[normID(string(m.merge))] = objectGeometry{position: pos, size: &model.Size{Width: MergeSize, Height: MergeSize}} + + plan.flows[normID(string(m.inFlow))] = flowGeometry{ + originIndex: bf.OriginConnectionIndex, + destinationIndex: bf.DestinationConnectionIndex, + originVector: "0;0", + destVector: "0;0", + } + plan.flows[normID(string(m.outFlow))] = flowGeometry{ + originIndex: oppositeAnchor(bf.DestinationConnectionIndex), + destinationIndex: bf.DestinationConnectionIndex, + originVector: "0;0", + destVector: "0;0", + } +} + +func oppositeAnchor(side int) int { + switch side { + case AnchorTop: + return AnchorBottom + case AnchorBottom: + return AnchorTop + case AnchorLeft: + return AnchorRight + } + return AnchorLeft +} + +// layoutRawID reads a stored element's $ID in the typed model's form. The binary +// is a .NET GUID, whose first three groups are little-endian, so its plain hex +// is NOT the UUID string the decoder produced. +func layoutRawID(d bson.D) (string, bool) { + for _, e := range d { + if e.Key != "$ID" { + continue + } + switch v := e.Value.(type) { + case primitive.Binary: + return types.BlobToUUID(v.Data), true + case []byte: + return types.BlobToUUID(v), true + case string: + return v, true + } + } + return "", false +} + +func orZeroVector(v string) string { + if v == "" { + return "0;0" + } + return v +} + +func objectSize(o microflows.MicroflowObject) model.Size { + if s, ok := o.(interface{ GetSize() model.Size }); ok { + return s.GetSize() + } + return model.Size{} +} + +// flowGraphIndex is a flow's objects and edges, flattened across loop bodies. +type flowGraphIndex struct { + object map[model.ID]microflows.MicroflowObject + flow map[model.ID]*microflows.SequenceFlow + outgoing map[model.ID][]*microflows.SequenceFlow + incoming map[model.ID]int + // notesOn maps an object to the notes wired to it; freeNotes are the ones + // wired to nothing. + notesOn map[model.ID][]*microflows.Annotation + freeNotes []*microflows.Annotation + start model.ID + // order is every object in document order, for deterministic reporting. + order []model.ID +} + +func indexFlowGraph(oc *microflows.MicroflowObjectCollection) *flowGraphIndex { + g := &flowGraphIndex{ + object: map[model.ID]microflows.MicroflowObject{}, + flow: map[model.ID]*microflows.SequenceFlow{}, + outgoing: map[model.ID][]*microflows.SequenceFlow{}, + incoming: map[model.ID]int{}, + notesOn: map[model.ID][]*microflows.Annotation{}, + } + var notes []*microflows.Annotation + var annFlows []*microflows.AnnotationFlow + var addCollection func(c *microflows.MicroflowObjectCollection) + addCollection = func(c *microflows.MicroflowObjectCollection) { + if c == nil { + return + } + for _, o := range c.Objects { + g.object[o.GetID()] = o + g.order = append(g.order, o.GetID()) + switch t := o.(type) { + case *microflows.StartEvent: + if g.start == "" { + g.start = t.ID + } + case *microflows.LoopedActivity: + addCollection(t.ObjectCollection) + case *microflows.Annotation: + notes = append(notes, t) + } + } + for _, f := range c.Flows { + g.flow[f.ID] = f + g.outgoing[f.OriginID] = append(g.outgoing[f.OriginID], f) + g.incoming[f.DestinationID]++ + } + annFlows = append(annFlows, c.AnnotationFlows...) + } + addCollection(oc) + + wired := map[model.ID]bool{} + for _, af := range annFlows { + note, target := af.OriginID, af.DestinationID + if _, isNote := g.object[note].(*microflows.Annotation); !isNote { + note, target = target, note + } + n, ok := g.object[note].(*microflows.Annotation) + if !ok { + continue + } + g.notesOn[target] = append(g.notesOn[target], n) + wired[n.ID] = true + } + for _, n := range notes { + if !wired[n.ID] { + g.freeNotes = append(g.freeNotes, n) + } + } + return g +} + +// flowPairing walks the stored and rebuilt graphs in step from their start +// events, pairing each object with its counterpart and each flow with the flow +// that leaves the paired origin on the same branch. +type flowPairing struct { + stored, rebuilt *flowGraphIndex + objects map[model.ID]model.ID // stored → rebuilt + flows map[model.ID]model.ID + queue [][2]model.ID + // passThrough are stored merges the rebuild has no node for — see + // skipPassThroughMerges. + passThrough []passThroughMerge +} + +// passThroughMerge is a stored ExclusiveMerge with one flow in and one flow out. +// It joins nothing, so DESCRIBE leaves it out and the rebuild has no node to +// pair it with. It is placed on the rebuilt edge it sits on instead. +type passThroughMerge struct { + merge model.ID + inFlow model.ID // stored flow into the merge + outFlow model.ID // stored flow out of it + rebuilt model.ID // the rebuilt flow the whole chain stands for +} + +func (p *flowPairing) walk() error { + if p.stored.start == "" || p.rebuilt.start == "" { + return fmt.Errorf("it has no start event") + } + if err := p.pair(p.stored.start, p.rebuilt.start); err != nil { + return err + } + for len(p.queue) > 0 { + next := p.queue[0] + p.queue = p.queue[1:] + if err := p.visit(next[0], next[1]); err != nil { + return err + } + } + + // Notes are not on the control flow: pair the ones wired to a paired object + // by their caption, and the free ones by caption in order. + for sid, bid := range p.objects { + if err := p.pairNotes(p.stored.notesOn[sid], p.rebuilt.notesOn[bid]); err != nil { + return err + } + } + if err := p.pairNotes(p.stored.freeNotes, p.rebuilt.freeNotes); err != nil { + return err + } + + for _, id := range p.stored.order { + if _, ok := p.objects[id]; !ok && !p.isPassThrough(id) { + return fmt.Errorf("its %s did not survive the rebuild, so it does not round-trip through MDL", + describeLayoutObject(p.stored.object[id])) + } + } + if len(p.objects) != len(p.rebuilt.object) { + return fmt.Errorf("rebuilding it from its description made %d objects where it has %d, so it does not round-trip through MDL", + len(p.rebuilt.object), len(p.stored.object)) + } + for id := range p.stored.flow { + if _, ok := p.flows[id]; !ok { + return fmt.Errorf("one of its sequence flows did not survive the rebuild, so it does not round-trip through MDL") + } + } + return nil +} + +func (p *flowPairing) pair(sid, bid model.ID) error { + if prev, ok := p.objects[sid]; ok { + if prev != bid { + return fmt.Errorf("its %s joins branches differently after a rebuild, so it does not round-trip through MDL", + describeLayoutObject(p.stored.object[sid])) + } + return nil + } + so, bo := p.stored.object[sid], p.rebuilt.object[bid] + if so == nil || bo == nil { + return fmt.Errorf("a sequence flow points at an object that is not in the flow") + } + if layoutKind(so) != layoutKind(bo) { + return fmt.Errorf("its %s came back as a %s after a rebuild, so it does not round-trip through MDL", + describeLayoutObject(so), describeLayoutObject(bo)) + } + p.objects[sid] = bid + p.queue = append(p.queue, [2]model.ID{sid, bid}) + return nil +} + +func (p *flowPairing) visit(sid, bid model.ID) error { + // A loop's body is not reached by a flow: its entry is the body object + // nothing flows into. + if sl, ok := p.stored.object[sid].(*microflows.LoopedActivity); ok { + bl := p.rebuilt.object[bid].(*microflows.LoopedActivity) + se, be := loopEntries(sl, p.stored), loopEntries(bl, p.rebuilt) + if len(se) != len(be) || len(se) > 1 { + return fmt.Errorf("its loop body does not round-trip through MDL") + } + if len(se) == 1 { + if err := p.pair(se[0], be[0]); err != nil { + return err + } + } + } + + sOut, bOut := p.stored.outgoing[sid], p.rebuilt.outgoing[bid] + if len(sOut) != len(bOut) { + return fmt.Errorf("its %s has %d outgoing flows after a rebuild where it has %d, so it does not round-trip through MDL", + describeLayoutObject(p.stored.object[sid]), len(bOut), len(sOut)) + } + used := make([]bool, len(bOut)) + for _, sf := range sOut { + key := flowBranchKey(sf) + match := -1 + for i, bf := range bOut { + if !used[i] && flowBranchKey(bf) == key { + match = i + break + } + } + if match < 0 { + return fmt.Errorf("its %s has no %s branch after a rebuild, so it does not round-trip through MDL", + describeLayoutObject(p.stored.object[sid]), key) + } + used[match] = true + bf := bOut[match] + p.flows[sf.ID] = bf.ID + dest := p.skipPassThroughMerges(sf, bf) + if err := p.pair(dest, bf.DestinationID); err != nil { + return err + } + } + return nil +} + +// skipPassThroughMerges follows a stored flow through any merges the rebuild +// does not have, recording each, and returns the object the rebuilt flow's +// destination stands for. +func (p *flowPairing) skipPassThroughMerges(sf, bf *microflows.SequenceFlow) model.ID { + dest := sf.DestinationID + for { + if _, isMerge := p.stored.object[dest].(*microflows.ExclusiveMerge); !isMerge { + return dest + } + if _, alsoMerge := p.rebuilt.object[bf.DestinationID].(*microflows.ExclusiveMerge); alsoMerge { + return dest + } + out := p.stored.outgoing[dest] + if p.stored.incoming[dest] != 1 || len(out) != 1 { + return dest + } + p.passThrough = append(p.passThrough, passThroughMerge{ + merge: dest, inFlow: sf.ID, outFlow: out[0].ID, rebuilt: bf.ID, + }) + p.flows[out[0].ID] = bf.ID + sf = out[0] + dest = sf.DestinationID + } +} + +func (p *flowPairing) isPassThrough(id model.ID) bool { + for _, m := range p.passThrough { + if m.merge == id { + return true + } + } + return false +} + +func (p *flowPairing) pairNotes(stored, rebuilt []*microflows.Annotation) error { + used := make([]bool, len(rebuilt)) + for _, sn := range stored { + if _, done := p.objects[sn.ID]; done { + continue + } + for i, bn := range rebuilt { + if !used[i] && bn.Caption == sn.Caption { + if prev, ok := reverseLookup(p.objects, bn.ID); ok && prev != sn.ID { + continue + } + used[i] = true + p.objects[sn.ID] = bn.ID + break + } + } + } + return nil +} + +func reverseLookup(m map[model.ID]model.ID, v model.ID) (model.ID, bool) { + for k, x := range m { + if x == v { + return k, true + } + } + return "", false +} + +// loopEntries returns the objects in a loop body that no sequence flow enters, +// leaving out notes, which are never on the control flow. +func loopEntries(loop *microflows.LoopedActivity, g *flowGraphIndex) []model.ID { + if loop.ObjectCollection == nil { + return nil + } + var out []model.ID + for _, o := range loop.ObjectCollection.Objects { + if _, isNote := o.(*microflows.Annotation); isNote { + continue + } + if g.incoming[o.GetID()] == 0 { + out = append(out, o.GetID()) + } + } + return out +} + +// flowBranchKey names the branch a sequence flow takes out of its origin: the +// error handler, a split's case, or the one plain flow. +func flowBranchKey(f *microflows.SequenceFlow) string { + prefix := "" + if f.IsErrorHandler { + prefix = "error " + } + switch c := f.CaseValue.(type) { + case nil: + return prefix + "default" + case microflows.NoCase, *microflows.NoCase: + return prefix + "default" + case microflows.EnumerationCase: + return prefix + "'" + c.Value + "'" + case *microflows.EnumerationCase: + return prefix + "'" + c.Value + "'" + case microflows.BooleanCase: + return fmt.Sprintf("%s'%t'", prefix, c.Value) + case *microflows.BooleanCase: + return fmt.Sprintf("%s'%t'", prefix, c.Value) + case microflows.ExpressionCase: + return prefix + "'" + c.Expression + "'" + case *microflows.ExpressionCase: + return prefix + "'" + c.Expression + "'" + case microflows.InheritanceCase: + return prefix + inheritanceKey(c) + case *microflows.InheritanceCase: + return prefix + inheritanceKey(*c) + } + return prefix + fmt.Sprintf("%T", f.CaseValue) +} + +func inheritanceKey(c microflows.InheritanceCase) string { + if c.EntityQualifiedName != "" { + return "'" + c.EntityQualifiedName + "'" + } + return "'" + string(c.EntityID) + "'" +} + +// layoutKind is what two paired objects must agree on: their node type and, for +// an activity, the action it runs. +func layoutKind(o microflows.MicroflowObject) string { + k := fmt.Sprintf("%T", o) + if a, ok := o.(*microflows.ActionActivity); ok && a.Action != nil { + k += "/" + fmt.Sprintf("%T", a.Action) + } + return k +} + +func describeLayoutObject(o microflows.MicroflowObject) string { + if o == nil { + return "object" + } + t := fmt.Sprintf("%T", o) + if i := strings.LastIndex(t, "."); i >= 0 { + t = t[i+1:] + } + if a, ok := o.(*microflows.ActionActivity); ok && a.Action != nil { + at := fmt.Sprintf("%T", a.Action) + if i := strings.LastIndex(at, "."); i >= 0 { + at = at[i+1:] + } + if a.Caption != "" { + return fmt.Sprintf("%s activity '%s'", at, a.Caption) + } + return at + " activity" + } + return t +} + +// apply patches the plan onto a stored unit document and counts what changed. +// Only keys the stored element already has are rewritten, so the patch never +// invents a property the stored document's Mendix version does not have. +func (plan *layoutPlan) apply(doc bson.D) bson.D { + plan.movedObjects, plan.changedFlows = 0, 0 + out, _ := plan.patch(doc).(bson.D) + return out +} + +func (plan *layoutPlan) patch(node any) any { + switch v := node.(type) { + case bson.D: + out := make(bson.D, len(v)) + copy(out, v) + if id, ok := layoutRawID(out); ok { + if g, ok := plan.objects[normID(id)]; ok { + changed := setLayoutString(out, "RelativeMiddlePoint", fmt.Sprintf("%d;%d", g.position.X, g.position.Y)) + if g.size != nil { + changed = setLayoutString(out, "Size", fmt.Sprintf("%d;%d", g.size.Width, g.size.Height)) || changed + } + if changed { + plan.movedObjects++ + } + } else if g, ok := plan.flows[normID(id)]; ok { + if plan.patchFlow(out, g) { + plan.changedFlows++ + } + } + } + for i, e := range out { + out[i] = bson.E{Key: e.Key, Value: plan.patch(e.Value)} + } + return out + case bson.A: + out := make(bson.A, len(v)) + for i, item := range v { + out[i] = plan.patch(item) + } + return out + } + return node +} + +func (plan *layoutPlan) patchFlow(f bson.D, g flowGeometry) bool { + changed := setLayoutInt(f, "OriginConnectionIndex", g.originIndex) + changed = setLayoutInt(f, "DestinationConnectionIndex", g.destinationIndex) || changed + // Mendix 10+ keeps the vectors on a BezierCurve line; older versions on the + // flow itself. + for i, e := range f { + if e.Key != "Line" { + continue + } + if line, ok := e.Value.(bson.D); ok { + line = append(bson.D(nil), line...) + changed = setLayoutString(line, "OriginControlVector", g.originVector) || changed + changed = setLayoutString(line, "DestinationControlVector", g.destVector) || changed + f[i].Value = line + } + } + changed = setLayoutString(f, "OriginBezierVector", g.originVector) || changed + changed = setLayoutString(f, "DestinationBezierVector", g.destVector) || changed + return changed +} + +func setLayoutString(d bson.D, key, value string) bool { + for i, e := range d { + if e.Key != key { + continue + } + if cur, ok := e.Value.(string); ok && cur == value { + return false + } + d[i].Value = value + return true + } + return false +} + +// setLayoutInt rewrites an integer key keeping the stored BSON integer width. +func setLayoutInt(d bson.D, key string, value int) bool { + for i, e := range d { + if e.Key != key { + continue + } + switch cur := e.Value.(type) { + case int32: + if int(cur) == value { + return false + } + d[i].Value = int32(value) + case int64: + if int(cur) == value { + return false + } + d[i].Value = int64(value) + default: + d[i].Value = int32(value) + } + return true + } + return false +} diff --git a/mdl/executor/cmd_microflows_layout_test.go b/mdl/executor/cmd_microflows_layout_test.go new file mode 100644 index 0000000000..82113253a2 --- /dev/null +++ b/mdl/executor/cmd_microflows_layout_test.go @@ -0,0 +1,418 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "fmt" + "reflect" + "regexp" + "sort" + "strings" + "testing" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend" + modelsdkbackend "github.com/mendixlabs/mxcli/mdl/backend/modelsdk" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// The fixture's flows were drawn in Studio Pro, so none of them sits where the +// layout engine would put it — every "it moved" assertion below has that as its +// control, and every "nothing moved" assertion follows a run that did move it. + +func layoutExecutor(t *testing.T) *Executor { + t.Helper() + exec := New(&bytes.Buffer{}) + exec.SetQuiet(true) + exec.SetBackendFactory(func() backend.FullBackend { return modelsdkbackend.New() }) + t.Cleanup(func() { exec.Close() }) + run(t, exec, "CONNECT LOCAL '"+visitor.QuoteString(projectFixture(t))+"'") + return exec +} + +func flowQN(s string) ast.QualifiedName { + mod, name, _ := strings.Cut(s, ".") + return ast.QualifiedName{Module: mod, Name: name} +} + +// rawFlow reads a flow's stored unit as a generic document. +func rawFlow(t *testing.T, exec *Executor, kind, name string) (bson.M, *layoutGraph) { + t.Helper() + g, err := storedFlow(exec.newExecContext(t.Context()), kind, flowQN(name)) + if err != nil { + t.Fatalf("read %s: %v", name, err) + } + raw, err := exec.backend.GetRawUnitBytes(g.id) + if err != nil { + t.Fatalf("read raw %s: %v", name, err) + } + var doc bson.M + if err := bson.Unmarshal(raw, &doc); err != nil { + t.Fatalf("parse %s: %v", name, err) + } + return doc, g +} + +// changedKeys lists the keys of every leaf value that differs between two +// documents of the same shape, and any structural difference as "shape". +func changedKeys(a, b any, key string, out map[string]int) { + switch av := a.(type) { + case bson.M: + bv, ok := b.(bson.M) + if !ok || len(av) != len(bv) { + out["shape@"+key]++ + return + } + for k, x := range av { + y, ok := bv[k] + if !ok { + out["shape@"+k]++ + continue + } + changedKeys(x, y, k, out) + } + case bson.A: + bv, ok := b.(bson.A) + if !ok || len(av) != len(bv) { + out["shape@"+key]++ + return + } + for i := range av { + changedKeys(av[i], bv[i], key, out) + } + default: + if !reflect.DeepEqual(a, b) { + out[key]++ + } + } +} + +// layoutKeys are the only properties a layout may change. +var layoutKeys = map[string]bool{ + "RelativeMiddlePoint": true, + "Size": true, + "OriginConnectionIndex": true, + "DestinationConnectionIndex": true, + "OriginControlVector": true, + "DestinationControlVector": true, + "OriginBezierVector": true, + "DestinationBezierVector": true, +} + +func TestLayoutFlow_ChangesOnlyGeometry(t *testing.T) { + exec := layoutExecutor(t) + for _, tc := range []struct{ kind, name string }{ + {"microflow", "Administration.ChangeMyPassword"}, + {"nanoflow", "FeedbackModule.ACT_Feedback_UploadImage"}, + } { + t.Run(tc.name, func(t *testing.T) { + before, _ := rawFlow(t, exec, tc.kind, tc.name) + res, err := exec.LayoutFlow(tc.kind, flowQN(tc.name), false) + if err != nil { + t.Fatal(err) + } + if res.Refused != "" { + t.Fatalf("refused: %s", res.Refused) + } + if !res.Changed() { + t.Fatal("a Studio Pro-drawn flow reported nothing to move — the control failed") + } + after, _ := rawFlow(t, exec, tc.kind, tc.name) + + diff := map[string]int{} + changedKeys(before, after, "", diff) + if diff["RelativeMiddlePoint"] == 0 { + t.Errorf("no position changed on disk; diff: %v", diff) + } + for k, n := range diff { + if !layoutKeys[k] { + t.Errorf("layout changed %d %q value(s); only geometry may change", n, k) + } + } + }) + } +} + +func TestLayoutFlow_SecondRunChangesNothing(t *testing.T) { + exec := layoutExecutor(t) + name := flowQN("Administration.ChangeMyPassword") + + first, err := exec.LayoutFlow("microflow", name, false) + if err != nil || first.Refused != "" { + t.Fatalf("first run: %v %s", err, first.Refused) + } + if !first.Changed() { + t.Fatal("control: the first run must move the Studio Pro layout") + } + before, _ := rawFlow(t, exec, "microflow", name.String()) + + second, err := exec.LayoutFlow("microflow", name, false) + if err != nil { + t.Fatal(err) + } + if second.Changed() { + t.Errorf("second run moved %d objects and %d flows", second.Moved, second.Flows) + } + after, _ := rawFlow(t, exec, "microflow", name.String()) + if diff := map[string]int{}; func() bool { changedKeys(before, after, "", diff); return len(diff) > 0 }() { + t.Errorf("second run changed the stored flow: %v", diff) + } +} + +func TestLayoutFlow_DryRunWritesNothing(t *testing.T) { + exec := layoutExecutor(t) + name := flowQN("Administration.ChangePassword") + before, _ := rawFlow(t, exec, "microflow", name.String()) + + res, err := exec.LayoutFlow("microflow", name, true) + if err != nil || res.Refused != "" { + t.Fatalf("%v %s", err, res.Refused) + } + if !res.Changed() { + t.Fatal("control: a dry run must still report what would move") + } + after, _ := rawFlow(t, exec, "microflow", name.String()) + diff := map[string]int{} + changedKeys(before, after, "", diff) + if len(diff) > 0 { + t.Errorf("dry run wrote: %v", diff) + } +} + +// layoutAnnotationLine is a DESCRIBE line that only pins geometry. +var layoutAnnotationLine = regexp.MustCompile(`^\s*@(position|anchor|curve|merge|start)\b`) + +// notePosition is the placement inside an @annotation line; its size stays. +var notePosition = regexp.MustCompile(`,?\s*position:\s*\(\s*-?\d+\s*,\s*-?\d+\s*\)`) + +// TestLayoutFlow_MatchesCreate is the invariant the command exists for: a flow +// laid out in place looks exactly like the same flow created from MDL without +// layout annotations. The copy is made from the DESCRIBE text with those lines +// deleted — independently of stripFlowLayout — and executed as a new microflow. +func TestLayoutFlow_MatchesCreate(t *testing.T) { + exec := layoutExecutor(t) + for _, name := range []string{ + "Administration.ChangeMyPassword", // hand-placed start event (#951) must be reset too + "Administration.SaveNewAccount", + "FeedbackModule.SUB_Feedback_Sanitize", + } { + t.Run(name, func(t *testing.T) { + ctx := exec.newExecContext(t.Context()) + mdl, _, err := describeMicroflowToString(ctx, flowQN(name)) + if err != nil { + t.Fatal(err) + } + var kept []string + for _, line := range strings.Split(mdl, "\n") { + if strings.HasPrefix(strings.TrimSpace(line), "grant ") { + continue + } + if layoutAnnotationLine.MatchString(line) { + continue + } + if strings.Contains(line, "@annotation(") { + line = notePosition.ReplaceAllString(line, "") + } + kept = append(kept, line) + } + copyName := name + "_LayoutCopy" + script := strings.Replace(strings.Join(kept, "\n"), "microflow "+name+" ", "microflow "+copyName+" ", 1) + if !strings.Contains(script, copyName) { + t.Fatalf("could not rename the described microflow:\n%s", script) + } + run(t, exec, script) + + res, err := exec.LayoutFlow("microflow", flowQN(name), false) + if err != nil || res.Refused != "" { + t.Fatalf("%v %s", err, res.Refused) + } + + got := flowGeometryFingerprint(t, exec, name) + want := flowGeometryFingerprint(t, exec, copyName) + if strings.Join(got, "\n") != strings.Join(want, "\n") { + t.Errorf("laid out in place:\n %s\ncreated without annotations:\n %s", + strings.Join(got, "\n "), strings.Join(want, "\n ")) + } + }) + } +} + +// flowGeometryFingerprint is every object's kind, position and size, sorted, so +// two flows with the same layout compare equal whatever their IDs. +func flowGeometryFingerprint(t *testing.T, exec *Executor, name string) []string { + t.Helper() + _, g := rawFlow(t, exec, "microflow", name) + var out []string + idx := indexFlowGraph(g.objects) + for _, id := range idx.order { + o := idx.object[id] + out = append(out, fmt.Sprintf("%s @%v %v", layoutKind(o), o.GetPosition(), objectSize(o))) + } + sort.Strings(out) + return out +} + +// A merge with one flow in and one out joins nothing, so DESCRIBE leaves it out +// and the rebuild has no node for it. It must not make the flow unlayoutable. +func TestLayoutFlow_PassThroughMerge(t *testing.T) { + exec := layoutExecutor(t) + name := flowQN("FeedbackModule.ACT_SubmitFeedback") + _, g := rawFlow(t, exec, "nanoflow", name.String()) + idx := indexFlowGraph(g.objects) + var merge microflows.MicroflowObject + for _, o := range idx.object { + if _, ok := o.(*microflows.ExclusiveMerge); ok && idx.incoming[o.GetID()] == 1 { + merge = o + } + } + if merge == nil { + t.Fatal("fixture no longer has a pass-through merge") + } + + res, err := exec.LayoutFlow("nanoflow", name, false) + if err != nil { + t.Fatal(err) + } + if res.Refused != "" { + t.Fatalf("refused: %s", res.Refused) + } + _, after := rawFlow(t, exec, "nanoflow", name.String()) + moved := indexFlowGraph(after.objects).object[merge.GetID()] + if moved.GetPosition() == merge.GetPosition() { + t.Errorf("the pass-through merge stayed at %v while everything around it moved", merge.GetPosition()) + } +} + +// A flow whose description rebuilds into a different graph is left alone. +func TestLayoutFlow_RefusesWhatDoesNotRoundTrip(t *testing.T) { + exec := layoutExecutor(t) + name := flowQN("FeedbackModule.VAL_Feedback") // branches share merges MDL rebuilds differently + before, _ := rawFlow(t, exec, "microflow", name.String()) + + res, err := exec.LayoutFlow("microflow", name, false) + if err != nil { + t.Fatal(err) + } + if res.Refused == "" { + t.Fatal("laid out a flow that does not round-trip through MDL") + } + if !strings.Contains(res.Refused, "round-trip") { + t.Errorf("refusal does not say why: %s", res.Refused) + } + after, _ := rawFlow(t, exec, "microflow", name.String()) + diff := map[string]int{} + changedKeys(before, after, "", diff) + if len(diff) > 0 { + t.Errorf("a refused flow was written: %v", diff) + } +} + +// stripFlowLayout must reach every nested body — a missed one would leave that +// body pinned where it was. +func TestStripFlowLayout_ReachesNestedBodies(t *testing.T) { + src := `create microflow M.F ( + @position(1, 2) + $P: String +) +begin + @start(0, 0) + @position(10, 10) + @anchor(from: bottom, to: top) + @curve(from: (1, 1), to: (2, 2)) + @annotation(text: 'note', position: (5, 5), size: (100, 40)) + declare $L List of M.E = empty; + @position(20, 20) + loop $I in $L begin + @position(30, 30) + if $P = 'x' then + @position(40, 40) + log info node 'n' 'a'; + else + @position(50, 50) + log info node 'n' 'b'; + end if; + end loop; + @position(60, 60) + @merge(65, 65) + if $P = 'y' then + @position(70, 70) + $R = call microflow M.G() on error { + @position(80, 80) + log info node 'n' 'c'; + }; + end if; +end;` + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse: %v", errs) + } + stmt := prog.Statements[0].(*ast.CreateMicroflowStmt) + + annotations := 0 + var check func(v reflect.Value, strip bool) int + check = func(v reflect.Value, _ bool) int { + n := 0 + switch v.Kind() { + case reflect.Ptr, reflect.Interface: + if v.IsNil() { + return 0 + } + if ann, ok := v.Interface().(*ast.ActivityAnnotations); ok { + annotations++ + for _, f := range []any{ann.Position, ann.Anchor, ann.Curve, ann.Merge, ann.Start} { + if !reflect.ValueOf(f).IsNil() { + n++ + } + } + for _, note := range append(ann.Notes, ann.FreeNotes...) { + if note.Position != nil { + n++ + } + } + } + return n + check(v.Elem(), false) + case reflect.Struct: + if p, ok := v.Interface().(ast.MicroflowParam); ok && p.Position != nil { + n++ + } + for i := 0; i < v.NumField(); i++ { + if v.Type().Field(i).IsExported() { + n += check(v.Field(i), false) + } + } + case reflect.Slice: + for i := 0; i < v.Len(); i++ { + n += check(v.Index(i), false) + } + } + return n + } + + if before := check(reflect.ValueOf(stmt), false); before < 12 { + t.Fatalf("control: expected the fixture to carry layout annotations throughout, found %d", before) + } + annotations = 0 + stripFlowLayout(stmt) + if left := check(reflect.ValueOf(stmt), false); left != 0 { + t.Errorf("%d layout annotations survived the strip", left) + } + if annotations == 0 { + t.Fatal("walked no annotations at all") + } + + // Content survives: the note keeps its text and size. + var note *ast.MicroflowAnnotation + for _, s := range stmt.Body { + if ann := ast.StatementAnnotations(s); ann != nil && len(ann.Notes)+len(ann.FreeNotes) > 0 { + all := append(ann.Notes, ann.FreeNotes...) + note = &all[0] + } + } + if note == nil || note.Text != "note" || note.Size == nil { + t.Errorf("the strip damaged the note: %+v", note) + } +} From d6dd95e1a0c5ffcf0c5476ff0a021ada3cea142e Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 24 Sep 2026 23:46:25 +0000 Subject: [PATCH 04/47] docs: document mxcli layout flows User manual page for the flow layout command, linked from the microflow @position section, and a pointer in the write-microflows skill so an agent re-arranges a flow with the command instead of rewriting it without its @position lines. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012qfvyasDvxj5Zqrn4bxczi --- .../skills/mendix/write-microflows/SKILL.md | 2 +- docs-site/src/SUMMARY.md | 1 + docs-site/src/language/microflow-structure.md | 5 ++ docs-site/src/tools/flow-layout.md | 75 +++++++++++++++++++ 4 files changed, 82 insertions(+), 1 deletion(-) create mode 100644 docs-site/src/tools/flow-layout.md diff --git a/.claude/skills/mendix/write-microflows/SKILL.md b/.claude/skills/mendix/write-microflows/SKILL.md index 28be44b05f..1035ebab0a 100644 --- a/.claude/skills/mendix/write-microflows/SKILL.md +++ b/.claude/skills/mendix/write-microflows/SKILL.md @@ -593,7 +593,7 @@ $var/Module.AssociationName/attribute -- Chained ### Annotation Pattern ```mdl -@position(200, 200) +@position(200, 200) -- optional: omit it and mxcli lays the flow out; to re-arrange an existing flow run `mxcli layout flows` @caption 'Persist order' @color Green @annotation 'Note about the next activity' diff --git a/docs-site/src/SUMMARY.md b/docs-site/src/SUMMARY.md index b6ef406e30..7d31ab83d5 100644 --- a/docs-site/src/SUMMARY.md +++ b/docs-site/src/SUMMARY.md @@ -121,6 +121,7 @@ - [Default Styling](tools/theme.md) - [Domain Model Layout](tools/domain-model-layout.md) +- [Microflow and Nanoflow Layout](tools/flow-layout.md) - [Code Navigation](tools/code-navigation.md) - [SHOW CALLERS / CALLEES](tools/callers-callees.md) - [SHOW REFERENCES / IMPACT](tools/references-impact.md) diff --git a/docs-site/src/language/microflow-structure.md b/docs-site/src/language/microflow-structure.md index 2e85e8d9d4..1d6570a94d 100644 --- a/docs-site/src/language/microflow-structure.md +++ b/docs-site/src/language/microflow-structure.md @@ -147,6 +147,11 @@ cross. A statement that carries `@position` is never moved, and becomes the star the row for the statements after it — so either place everything or nothing: a few hand-placed statements are not measured against what is laid out around them. +To re-arrange a flow that already exists — one drawn in Studio Pro, or one whose +positions no longer fit after edits — run `mxcli layout flows`. It applies this +same layout to the stored flow and changes nothing but coordinates; see +[Microflow and Nanoflow Layout](../tools/flow-layout.md). + ### Start event The start event has no statement of its own, so `@start` goes on the **first** diff --git a/docs-site/src/tools/flow-layout.md b/docs-site/src/tools/flow-layout.md new file mode 100644 index 0000000000..3c9a15f31f --- /dev/null +++ b/docs-site/src/tools/flow-layout.md @@ -0,0 +1,75 @@ +# Microflow and Nanoflow Layout + +`mxcli layout flows` re-arranges existing microflows and nanoflows on their +canvas. It is the flow counterpart of [`mxcli layout`](domain-model-layout.md), +and it is how you "reset the layout" of a flow: there is no clause on +`CREATE MICROFLOW` for that, because layout is not part of what a flow *does*. + +```bash +mxcli layout flows -p app.mpr Sales.ACT_Order_Submit # one flow +mxcli layout flows -p app.mpr --module Sales --dry-run # list what would move +mxcli layout flows -p app.mpr --module Sales # every flow in a module +``` + +## The same layout as `CREATE` + +There is one layout engine for flows, and this command uses it. A flow laid out +here ends up exactly where the same flow would be if you created it from MDL +with no `@position`: main path left to right, guard branches in the lane below, +long rows wrapped, loops sized to their bodies. See +[Microflow Structure](../language/microflow-structure.md#position) for the rules. + +The flow is described to MDL, every layout annotation (`@position`, `@anchor`, +`@curve`, `@merge`, `@start`, and the placement of `@annotation` notes) is +dropped, and it is built again as `CREATE` would build it. Only the geometry of +that build is kept. + +## Only positions change + +The result is patched onto the stored flow rather than written in its place, so +nothing but layout changes: + +- positions and sizes of activities, events, splits, merges and loops; +- positions of parameters and notes (a note keeps its size); +- the connection sides and curves of sequence flows. + +Element IDs, captions, expressions and every property MDL cannot express are +left exactly as stored. That is also why this works on flows drawn in Studio +Pro. + +## Flows it will not touch + +The rebuilt flow has to match the stored one object for object. If it does +not — the flow uses something MDL cannot yet express the same way, such as +several branches sharing one merge — the flow is skipped with the reason, and +the rest of the batch carries on: + +``` +FeedbackModule.VAL_Feedback: skipped — its ExclusiveMerge came back as a +ValidationFeedbackAction activity after a rebuild, so it does not round-trip +through MDL +``` + +A merge with one flow in and one flow out joins nothing, so it is not a reason +to skip: it is placed on the edge it sits on. + +## It replaces positions you set by hand + +Every flow it touches is re-arranged, including any you arranged yourself. Use +`--dry-run` first. Marketplace modules and `System` are never touched unless +you pass `--include-marketplace`, whether you name a module or a single flow. + +Running it again changes nothing: a second run reports `already laid out` and +writes nothing. + +## Flags + +| Flag | Meaning | +|---|---| +| `Module.Flow ...` | Lay out these microflows or nanoflows. | +| `--module ` | Lay out every microflow and nanoflow in this module (repeatable). | +| `--dry-run` | Report what would move, write nothing. | +| `--include-marketplace` | Also lay out flows in Marketplace modules. A module update replaces them, so this is normally pointless. | + +Either name flows or pass `--module`; laying out every flow in a project is too +broad to do by default. From c23af78999e03581663151c1e6779f50826cb35a Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 24 Sep 2026 23:51:11 +0000 Subject: [PATCH 05/47] docs(proposal): OData client expression slots for first-class expressions The previous revision said REST/OData had no expression slots. The consumed OData client has four (HttpUsername, HttpPassword, ClientCertificate, header values), and 10-odata-examples.mdl writes every literal as '''admin'''. - add the slots to the inventory and slice 3, with a worked before/after of the 10-odata-examples.mdl FullConfigAPI block - slice 0b: formatExprValue returns an already-quoted stored value unchanged, so describe -> exec turns 'admin' into admin (measured against the real function); ClientCertificate and header keys are printed unescaped - ProxyHost/Port/Username/Password are ByNameRef constants, not expressions; out of scope - open question 5 measured: the shipped odata-data-sharing skill writes HttpUsername: 'MxAdmin', which stores the identifier MxAdmin today; the example writes the triple-quoted form. Each option breaks one spelling; recommend flipping the meaning plus a check that detects the legacy form Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../PROPOSAL_first_class_expressions.md | 172 +++++++++++++++++- 1 file changed, 166 insertions(+), 6 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_first_class_expressions.md b/docs/11-proposals/PROPOSAL_first_class_expressions.md index a5cac1f968..5d9edb7e8b 100644 --- a/docs/11-proposals/PROPOSAL_first_class_expressions.md +++ b/docs/11-proposals/PROPOSAL_first_class_expressions.md @@ -2,7 +2,7 @@ title: First-class expressions for expression-typed MDL properties status: draft date: 2026-09-08 -revised: 2026-09-23 +revised: 2026-09-24 related: - https://github.com/mendixlabs/mxcli/issues/750 - PROPOSAL_expression_type_checking.md @@ -174,11 +174,16 @@ Two of #750's targets are **not** expression slots and drop out: (`MDLDomainModel.g4`, `attributeConstraint`) — Mendix computes the value with a microflow, and no expression is stored. Attribute `default` already takes `literal | expression`. -- **REST/OData "filter and mapping expressions".** No such slot exists in +- **REST "filter and mapping expressions".** No such slot exists in `MDLService.g4`: REST `Path:` / `Body: template` are `{param}` text templates - with their own escaping, and OData/REST mappings bind attributes by name. If a - real expression slot turns up there it joins slice 3; nothing is designed for - it speculatively. + with their own escaping, and REST mappings bind attributes by name. + +*(Revised 2026-09-24.)* An earlier revision said the same of OData; that was +wrong. The **consumed OData client** has four expression slots — +`HttpUsername`, `HttpPassword`, `ClientCertificate` and every `headers (…)` +value — and `mdl-examples/doctype-tests/10-odata-examples.mdl` (level 8.2, +`FullConfigAPI`) shows the cost: every literal is written `'''admin'''`. They +join slice 3, and their describer has a live round-trip bug (§6.2, slice 0b). ## 4. What this unlocks @@ -235,6 +240,35 @@ proposals compose rather than compete. the form and the executor decides by the declared kind — but it is the one place this change widens a generic rule, so it wants a maintainer decision. +5. **Flip the meaning of a quoted OData credential/header (§6.4 option a)?** + *Measured 2026-09-24* — grep of `mdl-examples/`, `.claude/skills/` and + `docs-site/src/` for OData `Http*` and `headers` values: + + | Where | Spelling | Stores today | Under (a) | + |---|---|---|---| + | `mdl-examples/doctype-tests/10-odata-examples.mdl` (5 values) | triple-quoted, `'''admin'''` | string literal `'admin'` — correct | a string whose text is `'admin'`, quote characters included — **wrong** | + | `.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md` (4 clients) | single-quoted, `HttpUsername: 'MxAdmin'`, `HttpPassword: '1'` | identifier `MxAdmin`, integer `1` — **wrong** | string literals `'MxAdmin'`, `'1'` — correct | + + The create path copies the value through unchanged (`cmd_odata.go`, + `Username: stmt.HttpUsername`), so the shipped skill — the text agents copy + from — teaches a spelling that stores an expression Mendix cannot use as a + credential (CE0117 or a type error expected; not yet measured with + `mx check`). + + So both spellings are in use and **each option breaks one of them**: (c) + leaves the skill's spelling silently wrong; (a) silently puts quote + characters into every triple-quoted credential. What makes (a) still the + better choice is that its failure is *detectable*: under (a), a value in + these four slots whose text itself begins and ends with `'` is almost + certainly the legacy form, and `check` can flag it (a real password that + starts and ends with a quote character is the only false positive). Under + (c) the wrong spelling is indistinguishable from a deliberate identifier + expression. Proposal: (a) plus that check, as an error for one release, then + a warning. + + Independent of the decision: the skill's four clients are a live defect and + should be fixed now, in the triple-quoted form that is correct today. + ## 6. Implementation plan ### 6.1 Slot inventory @@ -247,10 +281,20 @@ proposals compose rather than compete. | `ALTER PAGE SET DynamicClasses = … ON w` | expression | `alterPageAssignment` (`MDLParser.g4`) | n/a | | page/snippet `Variables: { $v: T = '…' }` | expression | `variableDeclaration: VARIABLE COLON dataType EQUALS STRING_LITERAL` | `cmd_pages_describe.go` — `mdlQuote(defaultVal)` | | workflow / user-task `due date '…'` | expression | `DUE DATE_TYPE STRING_LITERAL` (`MDLWorkflow.g4`) | `cmd_workflows.go` — `mdlQuoted(DueDate)` | +| OData client `HttpUsername:` / `HttpPassword:` | expression | `odataPropertyAssignment` → `odataPropertyValue` (`MDLService.g4`) | `cmd_odata.go` — `formatExprValue` (**round-trip bug**, slice 0b) | +| OData client `ClientCertificate:` | expression | as above | `cmd_odata.go` — raw `'%s'`, **unescaped** (slice 0b) | +| OData client `headers ( 'K': … )` value | expression | `odataHeaderEntry: STRING_LITERAL COLON odataPropertyValue` | `cmd_odata.go` — `formatExprValue`; the key is printed raw `'%s'` | | user task `targeting users/groups xpath '…'` | XPath | `TARGETING … XPATH STRING_LITERAL` | `cmd_workflows.go` — `mdlQuoted(us.XPath)` | | offline `sync … where` | XPath | `WHERE (xpathConstraint \| STRING_LITERAL)` | **done** | | widget `Visible:` / `Editable:` | XPath-shaped | `xpathConstraint` | **done** | +**Not expressions, despite appearances:** OData client `ProxyHost`, +`ProxyPort`, `ProxyUsername` and `ProxyPassword` are `ByNameRef`s to a constant +(`modelsdk/gen/rest/types.go`, `ConsumedODataService.proxyHost`), not +expression strings. The comment in `10-odata-examples.mdl` blaming "the BSON +shape" describes a by-name reference being written as a string. That is a +separate bug, and a first-class expression would not fix it. + Explicitly not yet in: workflow `timer '…'`, `decide by veto '…'`, `fallback '…'` and `description '…'`. Verify the stored kind of each against the reflection data before adding one — a timer delay and a veto outcome are not @@ -283,6 +327,36 @@ property (`expressionWidgetProps` in `validate_widgets.go`, plus `mdl/executor/`; prove it by reverting the check. Append a finding to `.claude/skills/fix-issue/findings/.jsonl`. +**Slice 0b — bug: OData client `describe` loses a quote level.** + +`formatExprValue` (`cmd_odata.go`) returns a stored value unchanged when it +already starts and ends with `'`, on the theory that it is "already a quoted +Mendix expression string literal". But the visitor *unquotes* the MDL string +(`odataValueText` → `unquoteString`), so the MDL text must carry one more level +of quoting than the stored expression. Measured on `8f08e229` by feeding the +real `formatExprValue` output back through the same unquoting: + +| stored expression | `describe` emits | re-`exec` stores | round-trips | +|---|---|---|---| +| `'admin'` | `'admin'` | `admin` | **no** | +| `'it''s'` | `'it''s'` | `it's` | **no** | +| `@Mod.C` | `'@Mod.C'` | `@Mod.C` | yes | +| `'a' + @Mod.C` | `'''a'' + @Mod.C'` | `'a' + @Mod.C` | yes | + +The failing rows are the *common* case — exactly the literals the example file +writes as `'''admin'''`. After one describe → exec cycle the credential is the +bare identifier `admin`, which Mendix parses as an expression and rejects +(CE0117 expected; not yet measured with `mx check`). `ClientCertificate` is +worse: it is printed as a raw `'%s'`, so a stored `'my-cert'` becomes +`''my-cert''`, which does not re-parse as one string. Header keys are printed +raw too. + +Fix: drop the fast path and always `mdlQuote` (escape every `'`). Test first: a +describe → parse → store round trip for each row above plus `ClientCertificate` +and a header, where the rows that pass today are the control. This is +independent of the feature and should ship first; slice 3 then replaces the +quoted output with the bare form. + **Slice 1 — XPath family: `targeting … xpath [ … ]`.** | File | Change | @@ -312,6 +386,10 @@ property (`expressionWidgetProps` in `validate_widgets.go`, plus | `mdl/grammar/domains/MDLPage.g4` | `variableDeclaration: VARIABLE COLON dataType EQUALS (STRING_LITERAL \| expression)` — a lone `STRING_LITERAL` keeps its legacy meaning (§6.3) | | `mdl/grammar/domains/MDLWorkflow.g4` | `DUE DATE_TYPE (STRING_LITERAL \| expression)` in workflow and user-task clauses | | `mdl/grammar/MDLParser.g4` | `alterPageAssignment`: expression alternative, validated against the property's type | +| `mdl/grammar/domains/MDLService.g4` | `odataPropertyValue`: add `expression` **last**, after `AT qualifiedName` and `qualifiedName`, so `@Mod.C` and `microflow M.F` keep their meaning in non-expression properties; same for the `odataHeaderEntry` value | +| `mdl/visitor/visitor_odata.go` | for `HttpUsername` / `HttpPassword` / `ClientCertificate` / header values only: parse the value as `expression` and store it rendered, so `'admin'` stores `'admin'` (§6.4 option a; under option c a `STRING_LITERAL` keeps today's meaning instead). The `*IsLiteral` flags (read by `resolveCredential` for the design-time `$metadata` fetch) become "is a single string-literal expression" | +| `mdl/executor/cmd_odata.go` | `describe` emits the bare form for these slots | +| `mdl-examples/doctype-tests/10-odata-examples.mdl` | rewrite level 8.2 in the bare form (§6.4); keep one quoted case as the compatibility test | | matching visitors, `cmd_pages_describe.go`, `cmd_workflows.go` | as slice 2 | ### 6.3 The one semantic trap: a quoted value keeps meaning "expression text" @@ -331,7 +409,89 @@ That is today's behaviour; the plan does not make it worse, and the skills showing the bare form is what steers people off it. Recorded here so no reviewer "fixes" the grammar by making a `STRING_LITERAL` a string value — that would silently re-interpret every existing script. The ordering in slice 2 -(`propertyValueV3` before `expression`) is what enforces it. +(`propertyValueV3` before `expression`) is what enforces it. The one proposed +exception, argued separately because its trade-off is the opposite, is the +OData client's four slots (§6.4). + +### 6.4 Worked example: the OData client + +`10-odata-examples.mdl`, level 8.2, today: + +```mdl +create odata client OdTest.FullConfigAPI ( + ... + -- HttpUsername/HttpPassword/ClientCertificate are Mendix expression fields: + -- the stored value must be a Mendix expression string, so a string literal + -- needs single quotes inside the MDL string (use doubled '' to escape). + HttpUsername: '''admin''', + HttpPassword: '''secret''', + ClientCertificate: '''my-cert''', + ErrorHandlingMicroflow: microflow OdTest.HandleError +) +-- Header values are Mendix expression fields too; wrap literal values in +-- single quotes (escaped as doubled '') so the BSON stores a valid +-- string-literal expression rather than a bare identifier. +headers ( + 'X-Api-Key': '''abc123''', + 'Accept': '''application/json''' +); +``` + +After slice 3, under option (a) below — the same stored bytes as today's +`'''admin'''` spelling, and no explanatory comments needed: + +```mdl +create odata client OdTest.FullConfigAPI ( + ... + HttpUsername: 'admin', + HttpPassword: 'secret', + ClientCertificate: 'my-cert', + ErrorHandlingMicroflow: microflow OdTest.HandleError +) +headers ( + 'X-Api-Key': 'abc123', + 'Accept': 'application/json' +); +``` + +and a compound value, which today has to be written +`'''Key '' + @OdTest.ApiKey'`, becomes `'X-Api-Key': 'Key ' + @OdTest.ApiKey`. + +This is the one slot family where the §6.3 trap bites hardest, because the +first-class form and the legacy form **look the same and mean different +things**: `HttpUsername: 'admin'` today stores the expression `admin` (an +identifier), and a user reading the rewritten example will expect it to store +the string `'admin'`. The two cannot both hold. Options, for the maintainer: + +- **(a) Flip the meaning in these four slots**: a `STRING_LITERAL` becomes a + Mendix string literal, so `'admin'` stores `'admin'`. Readable, and matches + what everyone who ever wrote `'''admin'''` meant — but it silently changes the + stored value for any existing script that relied on `'admin'` storing + `admin`, and it changes today's quoted spelling of a compound value: a single + literal whose text is `'a' + @Mod.C` would store a string containing a plus + sign instead of the concatenation. Needs a check-time warning on the ambiguous + forms for one release (§5, question 5), and the examples file rewritten in the + same PR. +- **(b) Keep the meaning, and write the bare form with a marker**, e.g. + `HttpUsername: expr 'admin'` — no ambiguity, but it adds a keyword and gives + up most of the readability gain in the case that matters most. +- **(c) Keep the meaning and do nothing extra**: the example stays + `'''admin'''` for literals and the gain is only for compound expressions. + +There is precedent inside MDL for (a). The consumed **REST** client already +spells a header the way (a) would — `headers: ('Accept' = 'application/json')` +in `06-rest-client-examples.mdl` — because `restClientHeaderItem` parses the +value into `Value` / `Prefix` / `Variable` rather than taking expression text. +So today the same HTTP header is written `'Accept' = 'application/json'` on a +REST client and `'Accept': '''application/json'''` on an OData client. Option +(a) removes that inconsistency; (c) keeps it. + +Recommendation: **(a)**, scoped to these four OData slots. Unlike +`dynamicclasses` — where a plain class name is rare and an `if` is the norm — +an OData credential or header is almost always a literal or a constant, so the +quoted form *is* the common case and has to read correctly. The trade-off is a +real behaviour change — both spellings are in use today — and is measured in +open question 5. ## 7. Test plan From 6ff7404ffb8ce9086a8bb3c2beb63373f28046a1 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 24 Sep 2026 23:59:21 +0000 Subject: [PATCH 06/47] docs(proposal): `constant Mod.C` for by-name constant references @Mod.C is expression syntax, but MDL also uses it for slots that are ByNameRefs to a constant. Once expression slots take @Mod.C bare, the same text cannot also mean a reference. Spell references by target kind, as `microflow Mod.F` already does: `ProxyHost: constant Mod.C`. Adds 6.5 with the inventory of every @-spelled slot classified by its metamodel type: OData ServiceUrl is an expression (stays @); OData proxy, database connection and REST client credentials are references. The OData proxy path stores "@Mod.C" verbatim into the ByNameRef; the REST and database paths strip it. Records that the CE0117 claim in 10-odata-examples.mdl for the unprefixed form needs a Studio Pro BSON reference before the proxy half is built. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../PROPOSAL_first_class_expressions.md | 68 ++++++++++++++++++- 1 file changed, 66 insertions(+), 2 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_first_class_expressions.md b/docs/11-proposals/PROPOSAL_first_class_expressions.md index 5d9edb7e8b..2feb5b313f 100644 --- a/docs/11-proposals/PROPOSAL_first_class_expressions.md +++ b/docs/11-proposals/PROPOSAL_first_class_expressions.md @@ -292,8 +292,9 @@ proposals compose rather than compete. `ProxyPort`, `ProxyUsername` and `ProxyPassword` are `ByNameRef`s to a constant (`modelsdk/gen/rest/types.go`, `ConsumedODataService.proxyHost`), not expression strings. The comment in `10-odata-examples.mdl` blaming "the BSON -shape" describes a by-name reference being written as a string. That is a -separate bug, and a first-class expression would not fix it. +shape" describes a by-name reference being written with an expression's `@`. +A first-class expression would not fix it; §6.5 gives these slots their own +spelling, `constant Mod.C`. Explicitly not yet in: workflow `timer '…'`, `decide by veto '…'`, `fallback '…'` and `description '…'`. Verify the stored kind of each against the @@ -493,6 +494,69 @@ quoted form *is* the common case and has to read correctly. The trade-off is a real behaviour change — both spellings are in use today — and is measured in open question 5. +### 6.5 The other half of the boundary: `constant Mod.C` for named-constant references + +*(Added 2026-09-24.)* `@Mod.C` is Mendix **expression** syntax — it is how an +expression reads a constant's value, and it is valid only where an expression +is. MDL currently also uses it in places that are **not** expressions but +by-name references to a constant document, where the stored value is the +qualified name `Mod.C` and nothing is ever evaluated. First-class expressions +make that overloading untenable: once an expression slot takes +`ServiceUrl: @Mod.C` bare, the same four characters must not mean "a reference" +two lines further down. + +The rule: **spell a reference by its target kind, the way microflow references +already are.** + +```mdl +create odata client OdTest.FullConfigAPI ( + ServiceUrl: @OdTest.ServiceUrl, -- expression: reads the value + ErrorHandlingMicroflow: microflow OdTest.HandleError, -- reference (exists today) + ProxyHost: constant OdTest.ProxyHost, -- reference (new) + ProxyPort: constant OdTest.ProxyPort +) +``` + +`@` stays expression-only; `microflow`, `nanoflow`, `page`, `constant` name +what a by-name slot points at. The kind keyword also gives `check --references` +the target type to resolve against, which a bare qualified name does not. + +**Slot inventory.** Every `@`-spelled slot, classified by its metamodel type +(`modelsdk/gen/`): + +| Slot | Metamodel | Kind | Today | Status | +|---|---|---|---|---| +| OData client `ServiceUrl` | `ConsumedODataService.serviceUrl` `Primitive[string]` | expression | `@Mod.C`, also `'@Mod.C'` | correct; stays `@` | +| OData client `ProxyHost` / `ProxyPort` / `ProxyUsername` / `ProxyPassword` | `ByNameRef` (`rest/types.go`) | reference | `@Mod.C` → stored `"@Mod.C"` verbatim by `addStrIf` (`odata_write.go`) | **broken** — `@` kept in the name | +| database connection `connection string` / `username` / `password` | `ByNameRef` → `Constants$Constant` (`databaseconnector/types.go`) | reference | `@Mod.C`; visitor strips `@`, sets `*IsRef` | works; gains `constant` spelling | +| REST client `Username:` / `Password:` (and other constant-capable properties) | `Rest$ConstantValue.value` `ByNameRef` | reference | `@Mod.C`, legacy `$Mod.C`; visitor rewrites both to `$Mod.C` | works; two spellings already, `constant` becomes the canonical one | +| expression `atomicExpression` | — | expression | `@Mod.C` | correct; this is what `@` means | + +**Plan** (its own slice, 4 — independent of slices 0–3; the proxy fix alone +could ship as a bug): + +| File | Change | +|---|---| +| `mdl/grammar/domains/MDLService.g4` | `odataPropertyValue`, `restClientProperty`, `databaseConnectionOption`: add `CONSTANT qualifiedName` beside the existing `AT qualifiedName` (the `CONSTANT` token exists) | +| `mdl/visitor/visitor_odata.go` | `constant Mod.C` → `Mod.C` for the four proxy slots; `@Mod.C` there is stripped the same way (fixes the stored `"@Mod.C"`) — accepted for compatibility | +| `mdl/visitor/visitor_rest.go`, `visitor_dbconnection.go` | `constant` branch producing the same value as today's `@` branch | +| `mdl/executor/cmd_odata.go`, REST and database-connection describers | emit `constant Mod.C` for by-name constant slots | +| `mdl/executor/` (reference validation) | `constant X` must resolve to a constant; `constant` in an expression slot, or `@` in a reference slot, is a check-time hint naming the other spelling | +| `mdl-examples/doctype-tests/10-odata-examples.mdl` | replace the "omit proxy, BSON shape isn't well-defined" comment with a working `ProxyHost: constant …` case | + +**Compatibility.** `@Mod.C` keeps parsing in all three reference families, +permanently, as §3.2 does for quoted expressions; only `describe` changes. In +the REST client `$Mod.C` stays accepted too. + +**Needs a Studio Pro reference before the proxy half is built.** The metamodel +says `ProxyHost` stores the qualified name `Mod.C`, which is what +`ProxyHost: Mod.C` (no `@`) already writes today — yet the comment in +`10-odata-examples.mdl` reports that form fails with CE0117. Either the comment +is stale, or something else is required (a `ProxyType` other than +`DefaultProxy`, a constant of a particular type for the port, or a different +field). Per CLAUDE.md this is settled by a Studio Pro–configured client with a +custom proxy, dumped with `mxcli bson dump`, not by guessing. + ## 7. Test plan Every test follows CLAUDE.md "Working Rules": written first, proven by reverting From 9a032150fe7c16afdbadf943f01182a125dba35c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 00:05:52 +0000 Subject: [PATCH 07/47] docs(proposal): measure OData proxy constants against a Studio Pro reference Decoded Odata.Bug1073 from ako/TestApp@37e0cc0 (custom proxy configured in Studio Pro) and compared it with clients mxcli writes from the describe output: - by-name proxy slots store the bare qualified name with ProxyType Override; the unprefixed MDL form already matches byte-for-byte, so the CE0117 comment in 10-odata-examples.mdl is stale - `ProxyHost: @X` stores "@X" verbatim: the live bug `constant X` fixes - describe -> exec turns HttpAuthenticationUserName 'abc' into abc (slice 0b reproduced on real data) and blanks CustomLocation when OverrideLocation is false - the constant's type is user-chosen (String/Integer/Long), so check validates only that it is a constant Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../PROPOSAL_first_class_expressions.md | 60 +++++++++++++++---- 1 file changed, 49 insertions(+), 11 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_first_class_expressions.md b/docs/11-proposals/PROPOSAL_first_class_expressions.md index 2feb5b313f..bcbbfcdb11 100644 --- a/docs/11-proposals/PROPOSAL_first_class_expressions.md +++ b/docs/11-proposals/PROPOSAL_first_class_expressions.md @@ -2,7 +2,7 @@ title: First-class expressions for expression-typed MDL properties status: draft date: 2026-09-08 -revised: 2026-09-24 +revised: 2026-09-25 related: - https://github.com/mendixlabs/mxcli/issues/750 - PROPOSAL_expression_type_checking.md @@ -352,6 +352,9 @@ worse: it is printed as a raw `'%s'`, so a stored `'my-cert'` becomes `''my-cert''`, which does not re-parse as one string. Header keys are printed raw too. +Reproduced on a Studio Pro–authored client in §6.5 (`HttpAuthenticationUserName` +`'abc'` → `abc`), which also shows `CustomLocation` blanked by the round trip. + Fix: drop the fast path and always `mdlQuote` (escape every `'`). Test first: a describe → parse → store round trip for each row above plus `ClientCertificate` and a header, where the rows that pass today are the control. This is @@ -541,21 +544,56 @@ could ship as a bug): | `mdl/visitor/visitor_odata.go` | `constant Mod.C` → `Mod.C` for the four proxy slots; `@Mod.C` there is stripped the same way (fixes the stored `"@Mod.C"`) — accepted for compatibility | | `mdl/visitor/visitor_rest.go`, `visitor_dbconnection.go` | `constant` branch producing the same value as today's `@` branch | | `mdl/executor/cmd_odata.go`, REST and database-connection describers | emit `constant Mod.C` for by-name constant slots | -| `mdl/executor/` (reference validation) | `constant X` must resolve to a constant; `constant` in an expression slot, or `@` in a reference slot, is a check-time hint naming the other spelling | -| `mdl-examples/doctype-tests/10-odata-examples.mdl` | replace the "omit proxy, BSON shape isn't well-defined" comment with a working `ProxyHost: constant …` case | +| `mdl/executor/` (reference validation) | `constant X` must resolve to a constant of any type (below); any `Proxy*` set requires `ProxyType: Override`; `constant` in an expression slot, or `@` in a reference slot, is a check-time hint naming the other spelling | +| `mdl-examples/doctype-tests/10-odata-examples.mdl` | replace the "omit proxy, BSON shape isn't well-defined" comment with a working `ProxyType: Override, ProxyHost: constant …` case, mirroring `Odata.Bug1073` | **Compatibility.** `@Mod.C` keeps parsing in all three reference families, permanently, as §3.2 does for quoted expressions; only `describe` changes. In the REST client `$Mod.C` stays accepted too. -**Needs a Studio Pro reference before the proxy half is built.** The metamodel -says `ProxyHost` stores the qualified name `Mod.C`, which is what -`ProxyHost: Mod.C` (no `@`) already writes today — yet the comment in -`10-odata-examples.mdl` reports that form fails with CE0117. Either the comment -is stale, or something else is required (a `ProxyType` other than -`DefaultProxy`, a constant of a particular type for the port, or a different -field). Per CLAUDE.md this is settled by a Studio Pro–configured client with a -custom proxy, dumped with `mxcli bson dump`, not by guessing. +**Measured against a Studio Pro reference (2026-09-25).** `ako/TestApp` +commit `37e0cc0` ("proxy example odata") holds `Odata.Bug1073`, an OData client +configured in Studio Pro with a custom proxy, each field bound to a constant. +Its `Rest$ConsumedODataService` unit, decoded, next to two clients written by +mxcli (`8f08e229`+) into a copy of the project — `RT` from the unchanged +`describe` output, `AT` from the same with `ProxyHost: @…` and +`HttpUsername: '''abc'''`: + +| Field | Studio Pro | `RT` (describe → exec) | `AT` | +|---|---|---|---| +| `ProxyType` | `Override` | `Override` | `Override` | +| `ProxyHost` | `Odata.Bug1073_ProxyHost` | `Odata.Bug1073_ProxyHost` ✓ | `@Odata.Bug1073_ProxyHost` ✗ | +| `ProxyPort` / `ProxyUsername` / `ProxyPassword` | qualified name | same ✓ | same ✓ | +| `HttpConfiguration.HttpAuthenticationUserName` | `'abc'` | `abc` ✗ | `'abc'` ✓ | +| `HttpConfiguration.HttpAuthenticationPassword` | `@Clients.OrdersRestClient_password` | same ✓ | same ✓ | +| `HttpConfiguration.CustomLocation` (with `OverrideLocation: false`) | `@Odata.Bug1073_Location` | `""` ✗ | `""` ✗ | + +What this settles: + +1. **A by-name constant slot stores the bare qualified name, and the + unprefixed MDL form already writes it byte-for-byte.** The CE0117 claim in + `10-odata-examples.mdl`'s comment for the unprefixed form is stale; the + likelier cause at the time was `ProxyType` left at `DefaultProxy`. Studio + Pro sets `Override` whenever a custom proxy is configured, so `check` should + require `ProxyType: Override` when any `Proxy*` reference is set. +2. **`@` in a reference slot is the live bug**: it is stored verbatim, and the + name no longer resolves. The `constant` keyword fixes it; accepting `@` + there means stripping it, as the database-connection visitor does. +3. **Slice 0b reproduces on Studio Pro data**, not only on synthetic input: + `HttpUsername: 'abc'`, exactly as `describe` prints it, re-stores `abc`. +4. **A further describe loss:** Studio Pro keeps a `CustomLocation` + expression while `OverrideLocation` is false, and `describe` omits + `ServiceUrl` in that state, so a round trip blanks it. Harmless at runtime + (it is not used) but it is churn under ADR-0008, and the value reappears + the moment someone ticks "override" in Studio Pro. Belongs with slice 0b. + +**The constant's type is the user's choice.** Studio Pro lets the constant +behind a proxy field be String, Integer or Long — the reference uses a String +`'232'` for the port, but an Integer would do. So `constant X` is validated +only for *being a constant*; mxcli does not constrain its type. If Mendix +rejects a type in a given slot, `mx check` reports it against the real +version's rules, the same division of labour as xpathFunctionName's comment +describes. ## 7. Test plan From 513fdfe0152338f8fee3a1f4aa1bfc292f3601f0 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 00:06:36 +0000 Subject: [PATCH 08/47] docs(proposal): record Integer/Long proxy-port constants as measured ako/TestApp@11a8fca changes the proxy port constant from String to Integer with no Studio Pro errors (Long reported clean too); the client still stores the bare constant name. `constant X` stays type-agnostic. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../PROPOSAL_first_class_expressions.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/11-proposals/PROPOSAL_first_class_expressions.md b/docs/11-proposals/PROPOSAL_first_class_expressions.md index bcbbfcdb11..e59375ddb9 100644 --- a/docs/11-proposals/PROPOSAL_first_class_expressions.md +++ b/docs/11-proposals/PROPOSAL_first_class_expressions.md @@ -587,13 +587,15 @@ What this settles: (it is not used) but it is churn under ADR-0008, and the value reappears the moment someone ticks "override" in Studio Pro. Belongs with slice 0b. -**The constant's type is the user's choice.** Studio Pro lets the constant -behind a proxy field be String, Integer or Long — the reference uses a String -`'232'` for the port, but an Integer would do. So `constant X` is validated -only for *being a constant*; mxcli does not constrain its type. If Mendix -rejects a type in a given slot, `mx check` reports it against the real -version's rules, the same division of labour as xpathFunctionName's comment -describes. +**The constant's type is the user's choice — measured.** In `ako/TestApp` +commit `11a8fca` ("Proxy port") the port constant `Odata.Bug1073_ProxyPort` was +changed from String `'232'` to Integer `232`; Studio Pro reports no errors +(Long was also tried and reported clean), and the client's `ProxyPort` still +stores the bare name `Odata.Bug1073_ProxyPort` — the reference is identical +whatever the constant's type. So `constant X` is validated only for *being a +constant*; mxcli does not constrain its type. If Mendix rejects a type in some +slot, `mx check` reports it against the real version's rules, the same +division of labour as xpathFunctionName's comment describes. ## 7. Test plan From 26880a0477e8de273c1e26bffdf5724233f2a949 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 00:22:01 +0000 Subject: [PATCH 09/47] fix(odata): describe odata client re-quotes stored expressions A client configured in Studio Pro with a literal user name stores the expression `'abc'` (quotes included). DESCRIBE printed `HttpUsername: 'abc'`, and executing that output stored `abc` - an identifier, not a string. Measured on a Studio Pro-authored client (ako/TestApp@37e0cc0, Odata.Bug1073) by decoding the mxunit before and after a describe -> exec round trip. formatExprValue passed any value that already started and ended with a quote through unchanged. The visitor unquotes the MDL string, so the text always needs one more level of quoting than the stored expression. It now always uses mdlQuote, the inverse of unquoteString. ClientCertificate, header keys, Version, MetadataUrl and Folder were printed as a raw '%s' and did not re-parse when they held a quote; they are escaped too. Tests parse the describe output with the real visitor and compare what a re-exec would store against the stored value. Before the fix: HttpUsername: stored "'abc'", re-exec of describe output stores "abc" with the `@Module.Const` password (which already round-tripped) passing as the control; the escaping test failed to parse at all. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../fix-issue/findings/mdl-executor.jsonl | 1 + ...a-client-describe-requotes-expressions.mdl | 44 +++++++ mdl/executor/cmd_odata.go | 28 ++--- .../cmd_odata_client_describe_quoting_test.go | 107 ++++++++++++++++++ 4 files changed, 167 insertions(+), 13 deletions(-) create mode 100644 mdl-examples/bug-tests/odata-client-describe-requotes-expressions.mdl create mode 100644 mdl/executor/cmd_odata_client_describe_quoting_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index f340793d9f..55a346a719 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -674,3 +674,4 @@ {"area":"mdl/executor","date":"2026-09-22","symptom":"`UPDATE WIDGETS` prints a per-property `Warning: Failed to set …` for every assignment and then reports `Updated 2 widget(s)`, plus `Note: Run 'refresh catalog full force' to update the catalog with changes`, and exits 0. `describe styling` afterwards shows nothing was written","cause":"`updated++` sat OUTSIDE the assignment loop and was unconditional, so the counter meant \"this widget was found\" and was reported as \"Updated\". The same counter gated `mutator.Save()`, so a container whose every assignment failed was still saved","file":"`mdl/executor/cmd_widgets.go` (`updateOutcome`, `updateWidgetsInContainer`, `execUpdateWidgets` summary)","insight":"**A success counter incremented in the wrong loop is invisible to every test that only checks the happy path** — the failures were already being printed correctly one line above the lie. Split the outcome into the three things that actually happen (changed / matched-but-unwritable / in-catalog-but-not-in-document) rather than adding a boolean: rounding the third into either of the others is how a stale catalog reads as success. **Bound the severity before writing it up**: the rebuilt document was semantically identical, so ADR-0008 elision skipped the write — measured, no `mprcontents/` unit changed mtime and `mx check` stayed at 0 errors, making this a reporting defect and not a data one. Worth saying, because \"claims success after failing\" otherwise reads as corruption. **The DRY RUN had the same defect one step earlier and is the worse half**, since the syntax help tells you to run it first: it printed `Would set …` without attempting anything. Fixed by running the assignments against `pagemutator.Probe()` — the discardable copy `mxcli check` already uses for ALTER PAGE SET — so the preview reports `Cannot set`. Reuse that seam rather than re-deriving what a setter accepts; a preview that re-implements the rule drifts from it in exactly the direction that hurts","refs":["ako/mxcli#520","ako/mxcli#515"]} {"area":"mdl/executor","date":"2026-09-22","symptom":"`alter page … set '' = on ` dead-ended — `set` reaches first-class properties and the stored widget's PLUGGABLE property bag, and a design property lives in `Appearance.DesignProperties`. The only spelling that worked was `alter styling`, a second statement for the same operation","cause":"No resolution from a STORED widget to its theme-registry key, so `set` could not tell a design property from a mistyped pluggable one and had to assume the latter","file":"`mdl/backend/pagemutator/probe.go` (`WidgetStorageType`); `mdl/executor/design_property_routing.go` (new); `cmd_alter_page.go` (`applySetPropertyMutator`); `mdl/backend/pagemutator/mutator.go` (the now-stale error message)","insight":"**The resolver the routing needed already existed with zero callers.** `bsonTypeToDesignPropsKey` ($Type → theme key) had never been referenced, so it had never been validated against anything; ako/mxcli#509 deliberately avoided standing up a third consumer of the concept before something needed it, and this was that something. **Do not assert the two key maps are consistent — they are not, and both directions have measured reasons.** $Type-only: `DataGrid`/`Gallery` are the NATIVE widgets, which the MDL keywords no longer produce (`datagrid`→Data grid 2's id via pluggableKeywordIDs), so the stored path resolves MORE than the inline one. Keyword-only: `header`/`footer` map to \"Header\"/\"Footer\" but MDL builds BOTH as `Forms$DivContainer`, and Atlas declares no such groups — so the inline design-property validation for a header widget misses and skips the widget silently, the same shape pluggableKeywordIDs records for combobox/gallery/image. A test that pins both exclusive SETS with their reasons is the useful shape; a consistency assertion fails on correct code. **Route only on a positive theme declaration for THIS widget's type** — routing on \"the theme says nothing, so it must be a design property\" turns a typo into a silently-written design property. **Prove the two statements are the same operation on bytes, not on reasoning**: write via `alter styling`, then run the `alter page` form and count rewritten units — 0 means elision found them semantically equal. `Altered page` is ALTER PAGE's fixed verb and is NOT the elision verb, so it proves nothing. Knock-on: the #1135 error message named `alter styling` as the route, which became stale the moment `set` learned the route — and a test asserted that wording, so it had to be inverted like the others","refs":["ako/mxcli#515","ako/mxcli#509","ako/mxcli#511","mendixlabs/mxcli#1135"]} {"area":"mdl/executor","date":"2026-09-22","symptom":"No way to set a design property across pages — \"every data grid compact and striped\" was one statement per page, and the bulk command that looked right (`update widgets`) writes only the pluggable property bag","cause":"ALTER PAGE's design-property SET (the singular half of ako/mxcli#515) had no plural sibling; MDL's only bulk page statement was `ALTER PAGES … SET LAYOUT`","file":"`mdl/grammar/MDLParser.g4` (`alterPagesStylingStatement`); `mdl/ast/ast_alter_page.go`; `mdl/visitor/visitor_alter_page.go`; `mdl/executor/cmd_alter_pages_styling.go` (new)","insight":"**The selector is the whole design problem, and a name cannot be it**: a widget name is unique only within its page (measured — `actionButton1` in 30 units of a blank project), so the predicate has to be a widget TYPE. Name it by the **MDL keyword**, resolved through the existing `pluggableKeywordIDs`, not by a `LIKE` over the stored id: `WidgetType LIKE '%datagrid%'` matches 20 widgets in 6 containers on a blank project because it sweeps in DatagridTextFilter/DateFilter/DropdownFilter, which do not carry the grid's design properties. **Reuse three things instead of growing a fourth of each** — `findMatchingWidgets` (the catalog query), the per-widget routing decision from the singular form, and `updateOutcome` from ako/mxcli#520 so a sweep that matches and writes nothing exits non-zero instead of claiming success. **Two ANTLR traps, both positional**: the rule has two `identifierOrKeyword` slots (optional module, WHERE value) returned as ONE list, so reading them positionally without checking `ctx.IN()` scopes a project-wide sweep to a module named after a widget type; and the sibling `ALTER PAGES … SET LAYOUT` shares the same prefix, so a test that the layout form still parses as itself is not optional. `ensureCatalog(ctx, true)` must be called before `findMatchingWidgets` or it nil-panics — a cold catalog otherwise reads as \"no such widgets\"","refs":["ako/mxcli#515","ako/mxcli#520"]} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "`describe odata client` on a Studio Pro-authored client printed `HttpUsername: 'abc'`; executing that output stored the expression `abc` (an identifier) where Studio Pro had stored `'abc'` (a string literal). ClientCertificate, header keys, Version, MetadataUrl and Folder were printed as a raw '%s' and did not re-parse when they held a quote", "cause": "formatExprValue returned any stored value that already started and ended with a quote unchanged, on the theory it was 'already a quoted Mendix expression string literal'. The visitor unquotes the MDL string, so the MDL text always needs one more level of quoting than the stored expression; the fast path removed exactly one level for exactly the common case (a literal credential or header)", "file": "`mdl/executor/cmd_odata.go` (`formatExprValue`, `outputConsumedODataServiceMDL`)", "insight": "**For an expression-typed slot, the stored text is the payload, not a display form** - quote it like any other string, never by inspecting its first and last character. A heuristic that recognises 'already quoted' is wrong precisely when the stored expression is itself a string literal, which is the most common case. The test that catches it parses the describe output with the real visitor and compares the parsed value to the stored one (exec(describe(x)) == x), with a value that already round-tripped (`@Module.Const`) as the control; asserting on describe text alone would have passed. The reference was decoded straight from a Studio Pro mxunit (ako/TestApp Odata.Bug1073). The describe output is now correct but reads `'''abc'''` - that readability cost is what PROPOSAL_first_class_expressions.md addresses, not this fix. Tests `cmd_odata_client_describe_quoting_test.go`", "refs": ["mendixlabs/mxcli#750"]} diff --git a/mdl-examples/bug-tests/odata-client-describe-requotes-expressions.mdl b/mdl-examples/bug-tests/odata-client-describe-requotes-expressions.mdl new file mode 100644 index 0000000000..f7d5352b17 --- /dev/null +++ b/mdl-examples/bug-tests/odata-client-describe-requotes-expressions.mdl @@ -0,0 +1,44 @@ +-- Bug: DESCRIBE ODATA CLIENT lost a quote level on a literal credential. +-- +-- Symptom: a client configured in Studio Pro with a literal user name stores the +-- Mendix expression `'abc'` (quotes included) in +-- HttpConfiguration.HttpAuthenticationUserName. DESCRIBE printed +-- `HttpUsername: 'abc'`; executing that output stored `abc` — an identifier, not +-- a string. Measured on a Studio Pro-authored client (ako/TestApp@37e0cc0, +-- Odata.Bug1073), decoded before and after a describe -> exec round trip. +-- +-- Cause: formatExprValue returned any value that already started and ended with +-- a quote unchanged, as "already a quoted Mendix expression". But the visitor +-- unquotes the MDL string, so the MDL text needs one more level of quoting than +-- the stored expression. ClientCertificate, header keys, Version, MetadataUrl and +-- Folder were printed as a raw '%s' and did not re-parse at all when they held a +-- quote. +-- +-- Fix: always mdlQuote — the inverse of the visitor's unquoteString. +-- +-- This is the form DESCRIBE now prints for that client. Each value re-stores the +-- expression Studio Pro stored: 'abc', @Module.Const, 'Key ' + @Module.Const. + +create constant OdQuote.ApiLocation +type string +default 'https://api.example.com/odata/v4/'; + +create constant OdQuote.ApiKey +type string +default 'secret'; +/ + +create odata client OdQuote.QuotedApi ( + ODataVersion: OData4, + MetadataUrl: 'https://api.example.com/odata/v4/$metadata', + ServiceUrl: '@OdQuote.ApiLocation', + UseAuthentication: Yes, + HttpUsername: '''abc''', + HttpPassword: '@OdQuote.ApiKey', + ClientCertificate: '''my-cert''' +) +headers ( + 'X-Api-Key': '''Key '' + @OdQuote.ApiKey', + 'X-O''Key': '''it''''s''' +); +/ diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index 85ebee85ac..4eb97e184e 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -137,16 +137,16 @@ func outputConsumedODataServiceMDL(ctx *ExecContext, svc *model.ConsumedODataSer var props []string if folderPath != "" { - props = append(props, fmt.Sprintf(" Folder: '%s'", folderPath)) + props = append(props, fmt.Sprintf(" Folder: %s", mdlQuote(folderPath))) } if svc.Version != "" { - props = append(props, fmt.Sprintf(" Version: '%s'", svc.Version)) + props = append(props, fmt.Sprintf(" Version: %s", mdlQuote(svc.Version))) } if svc.ODataVersion != "" { props = append(props, fmt.Sprintf(" ODataVersion: %s", svc.ODataVersion)) } if svc.MetadataUrl != "" { - props = append(props, fmt.Sprintf(" MetadataUrl: '%s'", svc.MetadataUrl)) + props = append(props, fmt.Sprintf(" MetadataUrl: %s", mdlQuote(svc.MetadataUrl))) } if svc.TimeoutExpression != "" { props = append(props, fmt.Sprintf(" Timeout: %s", svc.TimeoutExpression)) @@ -170,7 +170,7 @@ func outputConsumedODataServiceMDL(ctx *ExecContext, svc *model.ConsumedODataSer } } if cfg.ClientCertificate != "" { - props = append(props, fmt.Sprintf(" ClientCertificate: '%s'", cfg.ClientCertificate)) + props = append(props, fmt.Sprintf(" ClientCertificate: %s", formatExprValue(cfg.ClientCertificate))) } } @@ -212,7 +212,7 @@ func outputConsumedODataServiceMDL(ctx *ExecContext, svc *model.ConsumedODataSer if i == len(cfg.HeaderEntries)-1 { comma = "" } - fmt.Fprintf(ctx.Output, " '%s': %s%s\n", h.Key, formatExprValue(h.Value), comma) + fmt.Fprintf(ctx.Output, " %s: %s%s\n", mdlQuote(h.Key), formatExprValue(h.Value), comma) } fmt.Fprintln(ctx.Output, ");") } else { @@ -1783,15 +1783,17 @@ func validateODataClientExists(ctx *ExecContext, ref ast.QualifiedName) error { return mdlerrors.NewNotFoundMsg("odata client", ref.String(), fmt.Sprintf("odata client not found: %s", ref)) } -// formatExprValue formats a Mendix expression value for MDL output. -// If the value is already a quoted string literal (starts/ends with '), it's output as-is. -// Otherwise, it's wrapped in single quotes for round-trip compatibility. +// formatExprValue formats a stored Mendix expression value for MDL output. +// +// It always quotes, even a value that already starts and ends with a quote. The +// stored text IS the expression, and the visitor unquotes the MDL string, so +// Studio Pro's literal credential `'abc'` has to print with its own quotes +// doubled inside a second pair. Passing an already-quoted value through +// unchanged made a re-exec of DESCRIBE store `abc` — an identifier, not a +// string (ako/TestApp Odata.Bug1073). mdlQuote is the inverse of the visitor's +// unquoteString, backslashes included. func formatExprValue(val string) string { - if len(val) >= 2 && val[0] == '\'' && val[len(val)-1] == '\'' { - return val // Already a quoted Mendix expression string literal - } - // Wrap in quotes, escaping internal single quotes - return "'" + strings.ReplaceAll(val, "'", "''") + "'" + return mdlQuote(val) } // extractMicroflowRef strips a leading "microflow " keyword (any case) from a diff --git a/mdl/executor/cmd_odata_client_describe_quoting_test.go b/mdl/executor/cmd_odata_client_describe_quoting_test.go new file mode 100644 index 0000000000..c657e749e9 --- /dev/null +++ b/mdl/executor/cmd_odata_client_describe_quoting_test.go @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/model" +) + +// DESCRIBE ODATA CLIENT must print each stored value so that executing the +// output stores the same value again. HttpUsername/HttpPassword/header values +// hold Mendix EXPRESSIONS, and Studio Pro stores a literal credential as the +// expression `'abc'` — quotes included. formatExprValue passed any value that +// already started and ended with a quote through unchanged, so describe printed +// `HttpUsername: 'abc'`, the visitor unquoted it, and re-executing stored `abc`: +// an identifier, not a string. Measured on a Studio Pro-authored client +// (ako/TestApp@37e0cc0, Odata.Bug1073). ClientCertificate, header keys and the +// plain string properties were printed as a raw '%s', unescaped. +// describeAndReparse runs DESCRIBE on stored and parses the output with the real +// visitor, returning what a re-exec would store. +func describeAndReparse(t *testing.T, stored *model.ConsumedODataService, folder string) (*ast.CreateODataClientStmt, string) { + t.Helper() + var out bytes.Buffer + ctx := &ExecContext{Output: &out} + if err := outputConsumedODataServiceMDL(ctx, stored, "Odata", folder); err != nil { + t.Fatalf("describe: %v", err) + } + prog := parseMDL(t, out.String()) + for _, s := range prog.Statements { + if c, ok := s.(*ast.CreateODataClientStmt); ok { + return c, out.String() + } + } + t.Fatalf("describe output has no create odata client statement:\n%s", out.String()) + return nil, "" +} + +// The reported shape, exactly as Studio Pro stores it. +func TestDescribeODataClient_LiteralCredentialSurvivesReExec(t *testing.T) { + stored := &model.ConsumedODataService{ + Name: "Bug1073", + ODataVersion: "OData4", + HttpConfiguration: &model.HttpConfiguration{ + UseAuthentication: true, + Username: "'abc'", + // A constant reference already round-tripped: the control. + Password: "@Clients.OrdersRestClient_password", + }, + } + got, out := describeAndReparse(t, stored, "") + if got.HttpUsername != "'abc'" { + t.Errorf("HttpUsername: stored %q, re-exec of describe output stores %q\n%s", "'abc'", got.HttpUsername, out) + } + if got.HttpPassword != stored.HttpConfiguration.Password { + t.Errorf("HttpPassword (control): stored %q, re-exec stores %q\n%s", stored.HttpConfiguration.Password, got.HttpPassword, out) + } +} + +// Every other value the describer prints, with the characters that need escaping. +func TestDescribeODataClient_StoredValuesSurviveReExec(t *testing.T) { + stored := &model.ConsumedODataService{ + Name: "Bug1073", + Version: "1.0'b", + ODataVersion: "OData4", + MetadataUrl: "file:///tmp/o'reilly/$metadata.xml", + HttpConfiguration: &model.HttpConfiguration{ + OverrideLocation: true, + CustomLocation: "@Odata.Bug1073_Location", + ClientCertificate: "'my-cert'", + HeaderEntries: []*model.HttpHeaderEntry{ + {Key: "X-Api-Key", Value: "'Key ' + @Odata.ApiKey"}, + {Key: "X-O'Key", Value: "'it''s'"}, + {Key: "Accept", Value: `'a\b'`}, + // Written by an older mxcli from `'abc'`: not a valid string + // expression, but describe must reproduce it, not repair it. + {Key: "X-Legacy", Value: "abc"}, + }, + }, + } + got, out := describeAndReparse(t, stored, "Api/O'Clients") + + cfg := stored.HttpConfiguration + for _, c := range []struct{ field, want, got string }{ + {"Version", stored.Version, got.Version}, + {"MetadataUrl", stored.MetadataUrl, got.MetadataUrl}, + {"Folder", "Api/O'Clients", got.Folder}, + {"ServiceUrl", cfg.CustomLocation, got.ServiceUrl}, + {"ClientCertificate", cfg.ClientCertificate, got.ClientCertificate}, + } { + if c.got != c.want { + t.Errorf("%s: stored %q, re-exec of describe output stores %q\n%s", c.field, c.want, c.got, out) + } + } + + if len(got.Headers) != len(cfg.HeaderEntries) { + t.Fatalf("headers: stored %d, re-exec stores %d\n%s", len(cfg.HeaderEntries), len(got.Headers), out) + } + for i, h := range cfg.HeaderEntries { + if got.Headers[i].Key != h.Key || got.Headers[i].Value != h.Value { + t.Errorf("header %d: stored %q: %q, re-exec stores %q: %q", + i, h.Key, h.Value, got.Headers[i].Key, got.Headers[i].Value) + } + } +} From dad9be6d77f2bc028770902c1e5b79df39eba095 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 00:22:36 +0000 Subject: [PATCH 10/47] fix(odata): store proxy constant references without the @ prefix ProxyHost / ProxyPort / ProxyUsername / ProxyPassword are BY_NAME references to a constant. Studio Pro stores the bare qualified name with ProxyType Override (ako/TestApp@37e0cc0, Odata.Bug1073). MDL writes a constant as `@Module.Const`, and the create, create-or-modify and alter paths copied that text into the reference unchanged, so "@Module.Const" named no constant and the proxy resolved to nothing. extractConstantRef strips the `@` at every assignment site, as extractMicroflowRef already strips `microflow ` (#573). The bare, `@` and quoted-`@` spellings now all store the bare name. Tests parse real MDL and capture the value handed to the backend, on create and alter. Before the fix: ProxyHost written as @MyModule.ProxyHost: stored "@MyModule.ProxyHost" with the bare spelling passing as the control. End to end on a copy of ako/TestApp, `ProxyHost: @Odata.Bug1073_ProxyHost` now stores "Odata.Bug1073_ProxyHost", byte-identical to Studio Pro. Also corrects the comment in 10-odata-examples.mdl, which said the bare form fails with CE0117; the Studio Pro reference shows it is the form Mendix stores. The constant's type is not validated: Studio Pro accepts a String or an Integer port (ako/TestApp@11a8fca). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../odata-client-proxy-constant-at-prefix.mdl | 52 ++++++++ .../doctype-tests/10-odata-examples.mdl | 9 +- mdl/executor/cmd_odata.go | 35 ++++-- mdl/executor/cmd_odata_proxy_constant_test.go | 116 ++++++++++++++++++ 5 files changed, 196 insertions(+), 17 deletions(-) create mode 100644 mdl-examples/bug-tests/odata-client-proxy-constant-at-prefix.mdl create mode 100644 mdl/executor/cmd_odata_proxy_constant_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 55a346a719..729dfa4a97 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -675,3 +675,4 @@ {"area":"mdl/executor","date":"2026-09-22","symptom":"`alter page … set '' = on ` dead-ended — `set` reaches first-class properties and the stored widget's PLUGGABLE property bag, and a design property lives in `Appearance.DesignProperties`. The only spelling that worked was `alter styling`, a second statement for the same operation","cause":"No resolution from a STORED widget to its theme-registry key, so `set` could not tell a design property from a mistyped pluggable one and had to assume the latter","file":"`mdl/backend/pagemutator/probe.go` (`WidgetStorageType`); `mdl/executor/design_property_routing.go` (new); `cmd_alter_page.go` (`applySetPropertyMutator`); `mdl/backend/pagemutator/mutator.go` (the now-stale error message)","insight":"**The resolver the routing needed already existed with zero callers.** `bsonTypeToDesignPropsKey` ($Type → theme key) had never been referenced, so it had never been validated against anything; ako/mxcli#509 deliberately avoided standing up a third consumer of the concept before something needed it, and this was that something. **Do not assert the two key maps are consistent — they are not, and both directions have measured reasons.** $Type-only: `DataGrid`/`Gallery` are the NATIVE widgets, which the MDL keywords no longer produce (`datagrid`→Data grid 2's id via pluggableKeywordIDs), so the stored path resolves MORE than the inline one. Keyword-only: `header`/`footer` map to \"Header\"/\"Footer\" but MDL builds BOTH as `Forms$DivContainer`, and Atlas declares no such groups — so the inline design-property validation for a header widget misses and skips the widget silently, the same shape pluggableKeywordIDs records for combobox/gallery/image. A test that pins both exclusive SETS with their reasons is the useful shape; a consistency assertion fails on correct code. **Route only on a positive theme declaration for THIS widget's type** — routing on \"the theme says nothing, so it must be a design property\" turns a typo into a silently-written design property. **Prove the two statements are the same operation on bytes, not on reasoning**: write via `alter styling`, then run the `alter page` form and count rewritten units — 0 means elision found them semantically equal. `Altered page` is ALTER PAGE's fixed verb and is NOT the elision verb, so it proves nothing. Knock-on: the #1135 error message named `alter styling` as the route, which became stale the moment `set` learned the route — and a test asserted that wording, so it had to be inverted like the others","refs":["ako/mxcli#515","ako/mxcli#509","ako/mxcli#511","mendixlabs/mxcli#1135"]} {"area":"mdl/executor","date":"2026-09-22","symptom":"No way to set a design property across pages — \"every data grid compact and striped\" was one statement per page, and the bulk command that looked right (`update widgets`) writes only the pluggable property bag","cause":"ALTER PAGE's design-property SET (the singular half of ako/mxcli#515) had no plural sibling; MDL's only bulk page statement was `ALTER PAGES … SET LAYOUT`","file":"`mdl/grammar/MDLParser.g4` (`alterPagesStylingStatement`); `mdl/ast/ast_alter_page.go`; `mdl/visitor/visitor_alter_page.go`; `mdl/executor/cmd_alter_pages_styling.go` (new)","insight":"**The selector is the whole design problem, and a name cannot be it**: a widget name is unique only within its page (measured — `actionButton1` in 30 units of a blank project), so the predicate has to be a widget TYPE. Name it by the **MDL keyword**, resolved through the existing `pluggableKeywordIDs`, not by a `LIKE` over the stored id: `WidgetType LIKE '%datagrid%'` matches 20 widgets in 6 containers on a blank project because it sweeps in DatagridTextFilter/DateFilter/DropdownFilter, which do not carry the grid's design properties. **Reuse three things instead of growing a fourth of each** — `findMatchingWidgets` (the catalog query), the per-widget routing decision from the singular form, and `updateOutcome` from ako/mxcli#520 so a sweep that matches and writes nothing exits non-zero instead of claiming success. **Two ANTLR traps, both positional**: the rule has two `identifierOrKeyword` slots (optional module, WHERE value) returned as ONE list, so reading them positionally without checking `ctx.IN()` scopes a project-wide sweep to a module named after a widget type; and the sibling `ALTER PAGES … SET LAYOUT` shares the same prefix, so a test that the layout form still parses as itself is not optional. `ensureCatalog(ctx, true)` must be called before `findMatchingWidgets` or it nil-panics — a cold catalog otherwise reads as \"no such widgets\"","refs":["ako/mxcli#515","ako/mxcli#520"]} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`describe odata client` on a Studio Pro-authored client printed `HttpUsername: 'abc'`; executing that output stored the expression `abc` (an identifier) where Studio Pro had stored `'abc'` (a string literal). ClientCertificate, header keys, Version, MetadataUrl and Folder were printed as a raw '%s' and did not re-parse when they held a quote", "cause": "formatExprValue returned any stored value that already started and ended with a quote unchanged, on the theory it was 'already a quoted Mendix expression string literal'. The visitor unquotes the MDL string, so the MDL text always needs one more level of quoting than the stored expression; the fast path removed exactly one level for exactly the common case (a literal credential or header)", "file": "`mdl/executor/cmd_odata.go` (`formatExprValue`, `outputConsumedODataServiceMDL`)", "insight": "**For an expression-typed slot, the stored text is the payload, not a display form** - quote it like any other string, never by inspecting its first and last character. A heuristic that recognises 'already quoted' is wrong precisely when the stored expression is itself a string literal, which is the most common case. The test that catches it parses the describe output with the real visitor and compares the parsed value to the stored one (exec(describe(x)) == x), with a value that already round-tripped (`@Module.Const`) as the control; asserting on describe text alone would have passed. The reference was decoded straight from a Studio Pro mxunit (ako/TestApp Odata.Bug1073). The describe output is now correct but reads `'''abc'''` - that readability cost is what PROPOSAL_first_class_expressions.md addresses, not this fix. Tests `cmd_odata_client_describe_quoting_test.go`", "refs": ["mendixlabs/mxcli#750"]} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "`create odata client ... (ProxyHost: @Module.Const, ...)` (and `alter odata client ... set ProxyHost = @Module.Const`) stored the proxy reference as \"@Module.Const\", which names no constant; the proxy silently resolved to nothing", "cause": "ProxyHost/ProxyPort/ProxyUsername/ProxyPassword are BY_NAME references to a constant, stored as the bare qualified name. The visitor turns `@Module.Const` into the text \"@Module.Const\" (right for expression slots such as ServiceUrl) and the create, create-or-modify and alter paths all copied it into the reference unchanged", "file": "`mdl/executor/cmd_odata.go` (`extractConstantRef`, `createODataClient`, `alterODataClient`)", "insight": "**`@Module.Const` is expression syntax; a BY_NAME constant slot wants the bare name** - classify each slot by its metamodel type (`ByNameRef` vs `Primitive[string]` in modelsdk/gen) before deciding what a spelling means. Same shape as #573's `microflow ` prefix, and fixed the same way (a strip helper at every assignment site: create, create-or-modify, alter). The comment in 10-odata-examples.mdl that blamed 'the BSON shape' and claimed the bare form gave CE0117 was wrong: decoding a Studio Pro client with a custom proxy (ako/TestApp@37e0cc0) showed the bare form already matches byte-for-byte, with ProxyType Override. The constant's type is the author's choice (String or Integer port both accepted, ako/TestApp@11a8fca), so do not validate it. Tests `cmd_odata_proxy_constant_test.go`, bare spelling as control", "refs": ["mendixlabs/mxcli#750"]} diff --git a/mdl-examples/bug-tests/odata-client-proxy-constant-at-prefix.mdl b/mdl-examples/bug-tests/odata-client-proxy-constant-at-prefix.mdl new file mode 100644 index 0000000000..14ac43e3df --- /dev/null +++ b/mdl-examples/bug-tests/odata-client-proxy-constant-at-prefix.mdl @@ -0,0 +1,52 @@ +-- Bug: an OData client's proxy constant written as `@Module.Const` was stored +-- with the `@`. +-- +-- Symptom: `ProxyHost: @Module.Const` (MDL's spelling of a constant everywhere +-- else) stored the reference as "@Module.Const", which names no constant, so the +-- proxy silently resolved to nothing. +-- +-- ProxyHost / ProxyPort / ProxyUsername / ProxyPassword are BY_NAME references to +-- a constant. Studio Pro stores the bare qualified name, with ProxyType Override +-- (measured: ako/TestApp@37e0cc0, Odata.Bug1073 — `ProxyHost: +-- "Odata.Bug1073_ProxyHost"`). The constant's type is the author's choice: the +-- same client accepts an Integer port constant (ako/TestApp@11a8fca). +-- +-- Fix: extractConstantRef strips the `@` on CREATE and ALTER, as +-- extractMicroflowRef already strips `microflow ` (#573). The bare, `@` and +-- quoted-`@` spellings below all store the same bare name. + +create constant OdProxy.ProxyHost +type string +default 'proxy.example.com'; + +create constant OdProxy.ProxyPort +type integer +default 8080; + +create constant OdProxy.ProxyUsername +type string +default 'proxyuser'; + +create constant OdProxy.ProxyPassword +type string +default 'proxypass'; + +create constant OdProxy.ApiLocation +type string +default 'https://api.example.com/odata/v4/'; +/ + +create odata client OdProxy.ProxiedApi ( + ODataVersion: OData4, + MetadataUrl: 'https://api.example.com/odata/v4/$metadata', + ServiceUrl: '@OdProxy.ApiLocation', + ProxyType: Override, + ProxyHost: @OdProxy.ProxyHost, + ProxyPort: '@OdProxy.ProxyPort', + ProxyUsername: OdProxy.ProxyUsername, + ProxyPassword: @OdProxy.ProxyPassword +); +/ + +alter odata client OdProxy.ProxiedApi set ProxyHost = @OdProxy.ProxyHost; +/ diff --git a/mdl-examples/doctype-tests/10-odata-examples.mdl b/mdl-examples/doctype-tests/10-odata-examples.mdl index 528d5f5ffc..0f1c413cd9 100644 --- a/mdl-examples/doctype-tests/10-odata-examples.mdl +++ b/mdl-examples/doctype-tests/10-odata-examples.mdl @@ -305,11 +305,10 @@ create odata client OdTest.FullConfigAPI ( HttpPassword: '''secret''', ClientCertificate: '''my-cert''', ErrorHandlingMicroflow: microflow OdTest.HandleError - -- ProxyHost and ProxyPort are constant references but the BSON shape Mendix - -- expects isn't well-defined yet; passing them as `@OdTest.X` BSON-stores - -- `@OdTest.X` and Mendix can't resolve the literal name, passing them - -- unprefixed makes Mendix parse the value as an expression (CE0117). For - -- now, omit; configure proxy via project Constants if needed. + -- ProxyHost / ProxyPort / ProxyUsername / ProxyPassword are by-name + -- references to a constant, and need `ProxyType: Override` alongside them — + -- the shape Studio Pro stores for a custom proxy. `OdTest.X` and `@OdTest.X` + -- both store the bare name; see bug-tests/odata-client-proxy-constant-at-prefix.mdl. ) -- Header values are Mendix expression fields too; wrap literal values in -- single quotes (escaped as doubled '') so the BSON stores a valid diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index 4eb97e184e..dd2df60757 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -1013,16 +1013,16 @@ func createODataClient(ctx *ExecContext, stmt *ast.CreateODataClientStmt) error svc.ErrorHandlingMicroflow = extractMicroflowRef(stmt.ErrorHandlingMicroflow) } if stmt.ProxyHost != "" { - svc.ProxyHost = stmt.ProxyHost + svc.ProxyHost = extractConstantRef(stmt.ProxyHost) } if stmt.ProxyPort != "" { - svc.ProxyPort = stmt.ProxyPort + svc.ProxyPort = extractConstantRef(stmt.ProxyPort) } if stmt.ProxyUsername != "" { - svc.ProxyUsername = stmt.ProxyUsername + svc.ProxyUsername = extractConstantRef(stmt.ProxyUsername) } if stmt.ProxyPassword != "" { - svc.ProxyPassword = stmt.ProxyPassword + svc.ProxyPassword = extractConstantRef(stmt.ProxyPassword) } // Update HTTP configuration if stmt.ServiceUrl != "" || stmt.UseAuthentication || stmt.HttpUsername != "" || @@ -1121,10 +1121,10 @@ func createODataClient(ctx *ExecContext, stmt *ast.CreateODataClientStmt) error ConfigurationMicroflow: extractMicroflowRef(stmt.ConfigurationMicroflow), HeadersMicroflow: extractMicroflowRef(stmt.HeadersMicroflow), ErrorHandlingMicroflow: extractMicroflowRef(stmt.ErrorHandlingMicroflow), - ProxyHost: stmt.ProxyHost, - ProxyPort: stmt.ProxyPort, - ProxyUsername: stmt.ProxyUsername, - ProxyPassword: stmt.ProxyPassword, + ProxyHost: extractConstantRef(stmt.ProxyHost), + ProxyPort: extractConstantRef(stmt.ProxyPort), + ProxyUsername: extractConstantRef(stmt.ProxyUsername), + ProxyPassword: extractConstantRef(stmt.ProxyPassword), } // Build HTTP configuration if any HTTP-level properties are set @@ -1330,13 +1330,13 @@ func alterODataClient(ctx *ExecContext, stmt *ast.AlterODataClientStmt) error { case "errorhandlingmicroflow": svc.ErrorHandlingMicroflow = extractMicroflowRef(strVal) case "proxyhost": - svc.ProxyHost = strVal + svc.ProxyHost = extractConstantRef(strVal) case "proxyport": - svc.ProxyPort = strVal + svc.ProxyPort = extractConstantRef(strVal) case "proxyusername": - svc.ProxyUsername = strVal + svc.ProxyUsername = extractConstantRef(strVal) case "proxypassword": - svc.ProxyPassword = strVal + svc.ProxyPassword = extractConstantRef(strVal) default: return mdlerrors.NewUnsupported(fmt.Sprintf("unknown OData client property: %s", key)) } @@ -1796,6 +1796,17 @@ func formatExprValue(val string) string { return mdlQuote(val) } +// extractConstantRef strips a leading "@" from a constant reference. The proxy +// properties are BY_NAME references to a constant, and Studio Pro stores the bare +// qualified name (measured: ako/TestApp Odata.Bug1073, `ProxyHost: +// "Odata.Bug1073_ProxyHost"`). `@Module.Const` is MDL's spelling of a constant +// everywhere else, and it was written through verbatim — `"@Module.Const"` names +// nothing, so the proxy resolved to no constant. Accepts the bare, `@` and +// quoted-`@` spellings alike. +func extractConstantRef(ref string) string { + return strings.TrimPrefix(ref, "@") +} + // extractMicroflowRef strips a leading "microflow " keyword (any case) from a // microflow reference string. The visitor emits uppercase `"MICROFLOW " + qn` // for `microflow Module.Name` property values (see visitor_odata.go); both diff --git a/mdl/executor/cmd_odata_proxy_constant_test.go b/mdl/executor/cmd_odata_proxy_constant_test.go new file mode 100644 index 0000000000..df17c1f59c --- /dev/null +++ b/mdl/executor/cmd_odata_proxy_constant_test.go @@ -0,0 +1,116 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" +) + +// ProxyHost / ProxyPort / ProxyUsername / ProxyPassword are BY_NAME references to +// a constant: Studio Pro stores the bare qualified name `Odata.Bug1073_ProxyHost` +// (measured, ako/TestApp@37e0cc0). `@Module.Const` is how MDL spells a constant +// everywhere else, but it was written into the reference verbatim — `"@Odata.X"` +// names nothing, so the proxy silently resolves to no constant. Same class as +// #573's "MICROFLOW " prefix: the value handed to the backend must be the bare +// name, whatever spelling the author used. + +var proxyConstantSpellings = []struct { + name, mdl string +}{ + {"at", "@MyModule.ProxyHost"}, + {"quoted at", "'@MyModule.ProxyHost'"}, + // The bare name already stored correctly: the control. + {"bare", "MyModule.ProxyHost"}, +} + +func TestCreateODataClient_ProxyConstantStoredAsBareName(t *testing.T) { + for _, sp := range proxyConstantSpellings { + t.Run(sp.name, func(t *testing.T) { + mod := mkModule("MyModule") + var captured *model.ConsumedODataService + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListConstantsFunc: func() ([]*model.Constant, error) { return nil, nil }, + ListConsumedODataServicesFunc: func() ([]*model.ConsumedODataService, error) { + return nil, nil + }, + CreateConsumedODataServiceFunc: func(svc *model.ConsumedODataService) error { + captured = svc + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(mkHierarchy(mod))) + + prog := parseMDL(t, `create odata client MyModule.Api ( + ODataVersion: OData4, + MetadataUrl: 'https://example.com/odata/$metadata', + ProxyType: Override, + ProxyHost: `+sp.mdl+`, + ProxyPort: `+sp.mdl+`, + ProxyUsername: `+sp.mdl+`, + ProxyPassword: `+sp.mdl+` +);`) + stmt := prog.Statements[0].(*ast.CreateODataClientStmt) + _ = createODataClient(ctx, stmt) // the $metadata fetch may warn; the write is what is under test + + if captured == nil { + t.Fatal("CreateConsumedODataService was not called") + } + for field, got := range map[string]string{ + "ProxyHost": captured.ProxyHost, + "ProxyPort": captured.ProxyPort, + "ProxyUsername": captured.ProxyUsername, + "ProxyPassword": captured.ProxyPassword, + } { + if got != "MyModule.ProxyHost" { + t.Errorf("%s written as %s: stored %q, want the bare constant name %q", + field, sp.mdl, got, "MyModule.ProxyHost") + } + } + }) + } +} + +func TestAlterODataClient_ProxyConstantStoredAsBareName(t *testing.T) { + for _, sp := range proxyConstantSpellings { + t.Run(sp.name, func(t *testing.T) { + mod := mkModule("MyModule") + svc := &model.ConsumedODataService{ + BaseElement: model.BaseElement{ID: nextID("cos")}, + ContainerID: mod.ID, + Name: "Api", + } + h := mkHierarchy(mod) + withContainer(h, svc.ContainerID, mod.ID) + var updated *model.ConsumedODataService + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListConsumedODataServicesFunc: func() ([]*model.ConsumedODataService, error) { + return []*model.ConsumedODataService{svc}, nil + }, + UpdateConsumedODataServiceFunc: func(s *model.ConsumedODataService) error { + updated = s + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + prog := parseMDL(t, `alter odata client MyModule.Api set ProxyHost = `+sp.mdl+`;`) + assertNoError(t, alterODataClient(ctx, prog.Statements[0].(*ast.AlterODataClientStmt))) + + if updated == nil { + t.Fatal("UpdateConsumedODataService was not called") + } + if updated.ProxyHost != "MyModule.ProxyHost" { + t.Errorf("ProxyHost set to %s: stored %q, want the bare constant name %q", + sp.mdl, updated.ProxyHost, "MyModule.ProxyHost") + } + }) + } +} From 6a48ed72ce9608c189d9776c5e29bc53bef964ed Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 00:31:28 +0000 Subject: [PATCH 11/47] fix(odata): design-time fetch sends a string-literal credential's content HttpUsername / HttpPassword / header values are Mendix expressions. Studio Pro stores a literal credential as the string literal `'MxAdmin'`, and MDL spells that `HttpUsername: '''MxAdmin'''`. The design-time $metadata fetch sent the stored text verbatim, quotes included: a 401 and an empty client for the spelling that is correct at runtime. resolveCredential now evaluates a value that is exactly one Mendix string literal to its content (a doubled quote is one quote). Any other expression starting with a quote (`'Key ' + @M.C`) cannot be evaluated at design time and is reported unresolved instead of being sent as text. The existing spellings (bare literal, `@M.C`, `'@M.C'`, dotted literals) are unchanged and remain the controls. Before the fix: got "'MxAdmin'", want "MxAdmin" fetch sends "'MxAdmin'" / "'1'", want MxAdmin / 1 Reproduced again as the control by disabling the new branch. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../fix-issue/findings/mdl-executor.jsonl | 1 + mdl/executor/cmd_odata.go | 28 ++++++++++++++ mdl/executor/cmd_odata_metadata_auth_test.go | 37 +++++++++++++++++++ 3 files changed, 66 insertions(+) diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 729dfa4a97..aafeafd535 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -676,3 +676,4 @@ {"area":"mdl/executor","date":"2026-09-22","symptom":"No way to set a design property across pages — \"every data grid compact and striped\" was one statement per page, and the bulk command that looked right (`update widgets`) writes only the pluggable property bag","cause":"ALTER PAGE's design-property SET (the singular half of ako/mxcli#515) had no plural sibling; MDL's only bulk page statement was `ALTER PAGES … SET LAYOUT`","file":"`mdl/grammar/MDLParser.g4` (`alterPagesStylingStatement`); `mdl/ast/ast_alter_page.go`; `mdl/visitor/visitor_alter_page.go`; `mdl/executor/cmd_alter_pages_styling.go` (new)","insight":"**The selector is the whole design problem, and a name cannot be it**: a widget name is unique only within its page (measured — `actionButton1` in 30 units of a blank project), so the predicate has to be a widget TYPE. Name it by the **MDL keyword**, resolved through the existing `pluggableKeywordIDs`, not by a `LIKE` over the stored id: `WidgetType LIKE '%datagrid%'` matches 20 widgets in 6 containers on a blank project because it sweeps in DatagridTextFilter/DateFilter/DropdownFilter, which do not carry the grid's design properties. **Reuse three things instead of growing a fourth of each** — `findMatchingWidgets` (the catalog query), the per-widget routing decision from the singular form, and `updateOutcome` from ako/mxcli#520 so a sweep that matches and writes nothing exits non-zero instead of claiming success. **Two ANTLR traps, both positional**: the rule has two `identifierOrKeyword` slots (optional module, WHERE value) returned as ONE list, so reading them positionally without checking `ctx.IN()` scopes a project-wide sweep to a module named after a widget type; and the sibling `ALTER PAGES … SET LAYOUT` shares the same prefix, so a test that the layout form still parses as itself is not optional. `ensureCatalog(ctx, true)` must be called before `findMatchingWidgets` or it nil-panics — a cold catalog otherwise reads as \"no such widgets\"","refs":["ako/mxcli#515","ako/mxcli#520"]} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`describe odata client` on a Studio Pro-authored client printed `HttpUsername: 'abc'`; executing that output stored the expression `abc` (an identifier) where Studio Pro had stored `'abc'` (a string literal). ClientCertificate, header keys, Version, MetadataUrl and Folder were printed as a raw '%s' and did not re-parse when they held a quote", "cause": "formatExprValue returned any stored value that already started and ended with a quote unchanged, on the theory it was 'already a quoted Mendix expression string literal'. The visitor unquotes the MDL string, so the MDL text always needs one more level of quoting than the stored expression; the fast path removed exactly one level for exactly the common case (a literal credential or header)", "file": "`mdl/executor/cmd_odata.go` (`formatExprValue`, `outputConsumedODataServiceMDL`)", "insight": "**For an expression-typed slot, the stored text is the payload, not a display form** - quote it like any other string, never by inspecting its first and last character. A heuristic that recognises 'already quoted' is wrong precisely when the stored expression is itself a string literal, which is the most common case. The test that catches it parses the describe output with the real visitor and compares the parsed value to the stored one (exec(describe(x)) == x), with a value that already round-tripped (`@Module.Const`) as the control; asserting on describe text alone would have passed. The reference was decoded straight from a Studio Pro mxunit (ako/TestApp Odata.Bug1073). The describe output is now correct but reads `'''abc'''` - that readability cost is what PROPOSAL_first_class_expressions.md addresses, not this fix. Tests `cmd_odata_client_describe_quoting_test.go`", "refs": ["mendixlabs/mxcli#750"]} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`create odata client ... (ProxyHost: @Module.Const, ...)` (and `alter odata client ... set ProxyHost = @Module.Const`) stored the proxy reference as \"@Module.Const\", which names no constant; the proxy silently resolved to nothing", "cause": "ProxyHost/ProxyPort/ProxyUsername/ProxyPassword are BY_NAME references to a constant, stored as the bare qualified name. The visitor turns `@Module.Const` into the text \"@Module.Const\" (right for expression slots such as ServiceUrl) and the create, create-or-modify and alter paths all copied it into the reference unchanged", "file": "`mdl/executor/cmd_odata.go` (`extractConstantRef`, `createODataClient`, `alterODataClient`)", "insight": "**`@Module.Const` is expression syntax; a BY_NAME constant slot wants the bare name** - classify each slot by its metamodel type (`ByNameRef` vs `Primitive[string]` in modelsdk/gen) before deciding what a spelling means. Same shape as #573's `microflow ` prefix, and fixed the same way (a strip helper at every assignment site: create, create-or-modify, alter). The comment in 10-odata-examples.mdl that blamed 'the BSON shape' and claimed the bare form gave CE0117 was wrong: decoding a Studio Pro client with a custom proxy (ako/TestApp@37e0cc0) showed the bare form already matches byte-for-byte, with ProxyType Override. The constant's type is the author's choice (String or Integer port both accepted, ako/TestApp@11a8fca), so do not validate it. Tests `cmd_odata_proxy_constant_test.go`, bare spelling as control", "refs": ["mendixlabs/mxcli#750"]} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "`create odata client ... HttpUsername: '''MxAdmin'''` - the spelling that stores Studio Pro's string-literal expression `'MxAdmin'` - sent `'MxAdmin'`, quotes included, as the username on the design-time $metadata fetch: a 401, then an empty client. Fixing the odata-data-sharing skill's `HttpUsername: 'MxAdmin'` (which stores the identifier MxAdmin) would have traded a runtime defect for this one", "cause": "resolveCredential treated every quoted MDL value as a plain literal and sent the stored text. The stored text is a Mendix EXPRESSION; its tests had been written around `HttpUsername: 'f1api'`, the spelling that is wrong at runtime, so the correct spelling was never exercised", "file": "`mdl/executor/cmd_odata.go` (`resolveCredential`, `mendixStringLiteral`)", "insight": "**The design-time fetch has to evaluate the expression the runtime will evaluate** - for a single string literal that means its content, and anything compound (`'Key ' + @M.C`) is reported unresolved rather than sent as text. Tests written around an input spelling encode that spelling's meaning: the #23 tests used the runtime-wrong `'f1api'`, so they could not see this. Before changing what a skill or example teaches, run the new spelling through every consumer of the value (here: storage AND the metadata fetch), not only the one being fixed. Control: with the new branch disabled the end-to-end test reports `fetch sends \"'MxAdmin'\" / \"'1'\"`. Tests `cmd_odata_metadata_auth_test.go`", "refs": ["#23", "mendixlabs/mxcli#750"]} diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index dd2df60757..99517b6bc8 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -2123,12 +2123,40 @@ func resolveCredential(value string, isLiteral bool, consts map[string]string) ( v, found := consts[strings.ToLower(ref)] return v, found && v != "" } + // The value is a Mendix expression. A string literal — Studio Pro's own + // spelling of a literal credential, `'MxAdmin'` — sends its content; any + // other expression that starts with a quote (`'Key ' + @M.C`) cannot be + // evaluated here and is reported unresolved rather than sent as text. + if strings.HasPrefix(value, "'") { + s, ok := mendixStringLiteral(value) + return s, ok && s != "" + } if isLiteral { return value, true } return "", false } +// mendixStringLiteral reports whether expr is exactly one Mendix string literal +// (`'…'`, a quote inside doubled) and returns its content. +func mendixStringLiteral(expr string) (string, bool) { + if len(expr) < 2 || expr[0] != '\'' || expr[len(expr)-1] != '\'' { + return "", false + } + inner := expr[1 : len(expr)-1] + var b strings.Builder + for i := 0; i < len(inner); i++ { + if inner[i] == '\'' { + if i+1 >= len(inner) || inner[i+1] != '\'' { + return "", false // a lone quote ends the literal early: not one literal + } + i++ + } + b.WriteByte(inner[i]) + } + return b.String(), true +} + // constantReference reports whether a property value names a constant, and which // one. A leading @ marks a reference in either spelling; an unquoted qualified // name is one too, since a bare Module.Name cannot be a credential. diff --git a/mdl/executor/cmd_odata_metadata_auth_test.go b/mdl/executor/cmd_odata_metadata_auth_test.go index ec11fe78bb..c1ebebcafc 100644 --- a/mdl/executor/cmd_odata_metadata_auth_test.go +++ b/mdl/executor/cmd_odata_metadata_auth_test.go @@ -131,6 +131,17 @@ func TestResolveCredential(t *testing.T) { // A literal that happens to contain a dot is still a literal — passwords // contain dots, and that must not be read as a reference. {"a dotted literal stays a literal", "s3.cret", true, "s3.cret", true}, + // The value is a Mendix EXPRESSION. Studio Pro stores a literal + // credential as the string literal `'MxAdmin'`, quotes included, and so + // does MDL's `HttpUsername: '''MxAdmin'''`. The fetch must send its + // content, not the quotes — sending `'MxAdmin'` is a 401 against the + // very service the odata-data-sharing walkthrough imports from. + {"a string-literal expression sends its content", "'MxAdmin'", true, "MxAdmin", true}, + {"a doubled quote inside it is one quote", "'it''s'", true, "it's", true}, + {"an empty string literal is an empty credential", "''", true, "", false}, + // A compound expression cannot be evaluated here. Sending its text + // looks like it tried; reporting it unresolved says what happened. + {"a compound expression is unresolved", "'Key ' + @M.ApiUser", true, "", false}, } for _, tc := range cases { @@ -145,3 +156,29 @@ func TestResolveCredential(t *testing.T) { }) } } + +// The spelling the odata-data-sharing skill now teaches, end to end: parse the +// MDL, then build the credentials the design-time fetch will send. The stored +// value must be the expression `'MxAdmin'` (what Studio Pro stores) and the +// fetch must send `MxAdmin` — before the fix, fixing the skill traded a runtime +// defect for a 401 at design time. +func TestMetadataAuth_StringLiteralCredentialFromMDL(t *testing.T) { + prog := parseMDL(t, `create odata client M.Api ( + ODataVersion: OData4, + MetadataUrl: 'http://localhost:8080/odata/api/v1/$metadata', + UseAuthentication: Yes, + HttpUsername: '''MxAdmin''', + HttpPassword: '''1''' +);`) + stmt := prog.Statements[0].(*ast.CreateODataClientStmt) + if stmt.HttpUsername != "'MxAdmin'" { + t.Fatalf("stored HttpUsername = %q, want the string-literal expression %q", stmt.HttpUsername, "'MxAdmin'") + } + auth := metadataAuthFromStmt(&ExecContext{}, stmt) + if auth.Username != "MxAdmin" || auth.Password != "1" { + t.Errorf("fetch sends %q / %q, want MxAdmin / 1", auth.Username, auth.Password) + } + if len(auth.Unresolved) != 0 { + t.Errorf("Unresolved = %v, want none", auth.Unresolved) + } +} From dcd6f71eb7b06575415753c3731ba5a6ccbc9151 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 00:31:28 +0000 Subject: [PATCH 12/47] docs(skill): odata-data-sharing walkthrough credentials as string literals The four consumer clients wrote `HttpUsername: 'MxAdmin'` and `HttpPassword: '1'`. These properties hold a Mendix expression, so that stored the identifier `MxAdmin` and the integer `1` - not credentials. Studio Pro stores a literal credential as the string literal `'MxAdmin'`, which MDL spells `'''MxAdmin'''`. Verified by executing the walkthrough's client into a copy of ako/TestApp and decoding the unit: HttpAuthenticationUserName is now "'MxAdmin'" and HttpAuthenticationPassword "'1'". The design-time $metadata fetch sends MxAdmin / 1 for this spelling since the previous commit. A comment on the first client says why the quotes are doubled and that a constant (`@Module.Const`) needs none. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../reference/walkthroughs.md | 20 +++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md b/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md index 73f370eb1b..fa9d201496 100644 --- a/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md +++ b/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md @@ -186,8 +186,12 @@ create odata client ProductClient.ProductDataApiClient ( timeout: 300, ServiceUrl: '@ProductClient.ProductDataApiLocation', UseAuthentication: Yes, - HttpUsername: 'MxAdmin', - HttpPassword: '1' + -- HttpUsername/HttpPassword hold a Mendix EXPRESSION. A literal credential + -- is a string literal inside the MDL string, so its quotes are doubled; + -- 'MxAdmin' alone would store the identifier MxAdmin. A constant needs no + -- extra quotes: HttpPassword: @ProductClient.ApiPassword + HttpUsername: '''MxAdmin''', + HttpPassword: '''1''' ); -- OData client with local file - relative path (offline development) @@ -198,8 +202,8 @@ CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( Timeout: 300, ServiceUrl: '@ProductClient.ProductDataApiLocation', UseAuthentication: Yes, - HttpUsername: 'MxAdmin', - HttpPassword: '1' + HttpUsername: '''MxAdmin''', + HttpPassword: '''1''' ); -- OData client with local file - relative path without ./ @@ -209,8 +213,8 @@ CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( Timeout: 300, ServiceUrl: '@ProductClient.ProductDataApiLocation', UseAuthentication: Yes, - HttpUsername: 'MxAdmin', - HttpPassword: '1' + HttpUsername: '''MxAdmin''', + HttpPassword: '''1''' ); -- OData client with local file - absolute file:// URI @@ -220,8 +224,8 @@ CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( Timeout: 300, ServiceUrl: '@ProductClient.ProductDataApiLocation', UseAuthentication: Yes, - HttpUsername: 'MxAdmin', - HttpPassword: '1' + HttpUsername: '''MxAdmin''', + HttpPassword: '''1''' ); -- External entities (mapped from published service) From 3baa709763efb82ce0ec71ded771d9a66680bba9 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 00:35:16 +0000 Subject: [PATCH 13/47] fix: DESCRIBE WIDGET and widget list see installed widgets without widget init (#663) On a fresh clone (.mxcli/ is gitignored) DESCRIBE WIDGET called an installed widget unknown and `widget list -p` showed 9 definitions, because both read only .mxcli/widgets/*.def.json. The page builder and LoadWidgetRegistry already generate those from the installed .mpk first (mendixlabs/mxcli#1135). Factor that into LoadProjectWidgetDefinitions and use it from all three. The not-found error now only suggests forms that work: the quoted widget id (the unquoted one is a parse error), deduped keywords, and -p when no project is open. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + cmd/mxcli/cmd_widget.go | 10 +-- ...escribe-widget-installed-uninitialized.mdl | 21 +++++ mdl/executor/validate_widgets.go | 54 +++++++------ mdl/executor/widget_describe.go | 35 ++++++--- .../widget_describe_uninitialized_test.go | 78 +++++++++++++++++++ 6 files changed, 161 insertions(+), 38 deletions(-) create mode 100644 mdl-examples/bug-tests/663-describe-widget-installed-uninitialized.mdl create mode 100644 mdl/executor/widget_describe_uninitialized_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index b095f0c388..194f28d0d6 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -691,3 +691,4 @@ {"area":"mdl/executor","date":"2026-09-24","symptom":"`mxcli check -p … --references` passes a view entity whose association-path join is written `join s/Mod.A_B/System.UserRole AS r` (uppercase AS) even when a pass-through column from `r` declares the wrong string length; mxbuild then fails with CE6770 \"View Entity is out of sync with the OQL Query.\" Lowercase `as` reports MDL031 correctly.","cause":"extractAliasMap matched the path join case-insensitively ((?i)…(?:as\\s+)?) but then recovered the path from match[0] with strings.TrimSuffix(path, \"as\") — case-sensitive — so with `AS` the path kept a trailing ` AS`, the end-anchored lastEntity regex failed, and the alias was never mapped. Every column from that alias silently went without type inference.","file":"mdl/executor/oql_type_inference.go","insight":"A (?i) regex followed by string surgery on the whole match reintroduces case sensitivity by the back door: capture every piece you need as its own group instead of trimming it back out. The tell is a single control table varying only the case of one keyword — every other keyword in upper case was harmless, which points straight at code that handles that one token outside the regex. An unresolved alias is silent (the checker skips unknown types rather than reporting), so the symptom is a check that PASSES; test at extractAliasMap directly, and control with the unfixed binary on a real project (it printed `Check passed!` for AS, the error for as). DESCRIBE prints AS in upper case, so round-tripped OQL hits this by default.","refs":["#652"],"ce":["CE6770"],"rules":["MDL031"]} {"area":"mdl/executor","date":"2026-09-24","symptom":"`alter page M.P { insert into dv { use fragment SaveCancelFooter } }` passed `check` and failed in exec: \"failed to insert: failed to build widgets: failed to build widget SaveCancelFooter: unsupported widget type: USE_FRAGMENT\". Same for REPLACE, and for a fragment nested inside an inserted container.","cause":"Fragment/building-block expansion (pageBuilder.expandFragments) was called only on the CREATE PAGE / snippet / layout paths. ALTER PAGE's applyInsertWidgetMutator / applyReplaceWidgetMutator passed op.Widgets straight to buildWidgetsFromAST, whose pageBuilder even carried ctx.Fragments — the registry was wired, the expansion call was not. Second defect found on the way: cloneWidget copied only Type/Name/Properties/Children, dropping Specialization and TypeIsGeneric, so a cloned `template for` lost its routing.","file":"`mdl/executor/cmd_alter_page.go` (expandAlterFragments, called first in the INSERT and REPLACE mutators), `mdl/executor/cmd_pages_builder_v3.go` (cloneWidget copies the whole struct)","insight":"Expand before ANYTHING inspects the widget list, not just before the build: the duplicate-name check, allColumns and allListViewTemplates all read op.Widgets, and seeing the sentinel they check the fragment's name instead of its widgets' names. When a registry is threaded into a builder, grep for the call that consumes it (`expandFragments`), not for the field — the field being set on every pageBuilder is what made the ALTER path look covered. A field-by-field clone is a silent-drop hazard; `c := *w` then deep-copy the reference fields.","refs":["#572"]} {"area":"mdl/executor","date":"2026-09-24","symptom":"`describe layout Atlas_Core.Phone_BottomBar` emitted `-- Forms$SimpleMenuBar (simpleMenuBar1) -- NOT re-executable: mxcli cannot author this widget, so re-running this script would drop it`, so a phone layout written in MDL had no bottom bar and a describe -> exec copy of Atlas's lost it","cause":"No keyword, builder, writer or describer for Forms$SimpleMenuBar, although modelsdk/gen already had SimpleMenuBar and MenuDocumentSource. And the widget's whole point is WHICH menu document it renders (a Forms$MenuDocumentSource in MenuSource), a source kind no menu widget could express — menubar/navigationtree always wrote a Forms$NavigationSource, so one pointed at a menu document described as `menubar m` and replayed onto the Responsive profile. Layouts were also never reference-checked, so a new `Menu:` typo would have passed --references and hit CE1613 at build","file":"mdl/grammar/MDLLexer.g4 + domains/MDLPage.g4 + domains/MDLSettings.g4 (SIMPLEMENUBAR), mdl/executor/cmd_pages_builder_v3.go (buildSimpleMenuBarV3, menuSourceV3), mdl/backend/modelsdk/widget_write.go (menuSourceToGen), mdl/executor/cmd_pages_describe_parse.go + cmd_pages_describe_output.go, mdl/executor/helpers.go + validate.go (menu refs, CreateLayoutStmt), sdk/pages/pages_widgets_advanced.go","insight":"**Measure the stored shape before designing the syntax** — dumping the widget from a blank project (`mxcli new --version 11.14.0`, `bson dump --type layout`) showed the source was a menu DOCUMENT, not a profile, which is what made this a source-kind feature rather than a one-keyword copy of `menubar`. Treat MenuSource as one slot with two subtypes on all three menu widgets (one helper each side), or the siblings keep silently rewriting a menu-document source to Responsive. The control that justified the reference check: `menu: Bug573.No_Such_Menu` -> `mxcli check --references` passed, `mx check` CE1613 'The selected menu ... no longer exists.' at Simple menu bar. MDL has no `show menus`, so the not-found message lists the menus that exist. Adding reference validation to CreateLayoutStmt is new coverage for EVERY widget reference in a layout, not just menus","refs":["ako/mxcli#573"],"ce":["CE1613"]} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "On a fresh clone, `DESCRIBE WIDGET fieldset` reports `unknown widget \"fieldset\" — use an MDL keyword (barcodescanner, …) or a full widget id (com.mendix.widget…)` although Fieldset is installed in widgets/, page authoring accepts `fieldset` and DESCRIBE PAGE emits it. `mxcli widget list -p` shows 9 definitions. Both suggested remedies fail: `fieldset` IS the keyword, and the unquoted id is a parse error.", "cause": "DescribeWidget and `widget list` called only `LoadUserDefinitions`, which reads `.mxcli/widgets/*.def.json` — gitignored, so absent on every clone. The page builder and (since mendixlabs/mxcli#1135) LoadWidgetRegistry call RefreshStaleWidgetDefinitions first; these two readers were missed by that fix.", "file": "mdl/executor/validate_widgets.go (LoadProjectWidgetDefinitions, shared), mdl/executor/widget_describe.go (DescribeWidget, widgetNotFoundError), cmd/mxcli/cmd_widget.go (runWidgetList)", "insight": "**Second instance of the same class as mendixlabs/mxcli#1135 — the first fix patched one reader, not the call pattern.** `grep -rn LoadUserDefinitions` lists every reader of the project widget registry; each one that is not preceded by RefreshStaleWidgetDefinitions is this bug. The fix is a single LoadProjectWidgetDefinitions that does both, so the next reader cannot pick half. Still open: `cmd/mxcli/lsp_completion.go` (LSP completions) calls LoadUserDefinitions directly. **Reproduce cold or it is invisible**: `rm -rf .mxcli/widgets` first — any earlier exec/check in the same project self-heals it (the reporter's earlier sweeps read 43 keywords for exactly this reason). `testdata/expr-checker/widgets/com.mendix.widget.web.Fieldset.mpk` copied into a temp dir is the whole fixture; never use the checked-in project, because refresh writes .def.json into it. **Every remedy an error names must be run**: the old message's 'full widget id' is only valid QUOTED in DESCRIBE WIDGET (`'com.mendix.widget.web.fieldset.Fieldset'`), and the keyword list must be deduped (registry `datagrid` + builtin alias `DATAGRID` printed it twice).", "refs": ["ako/mxcli#663", "mendixlabs/mxcli#1135"]} diff --git a/cmd/mxcli/cmd_widget.go b/cmd/mxcli/cmd_widget.go index 57639a1eed..0a0dcbcfe8 100644 --- a/cmd/mxcli/cmd_widget.go +++ b/cmd/mxcli/cmd_widget.go @@ -230,12 +230,12 @@ func runWidgetList(cmd *cobra.Command, args []string) error { return fmt.Errorf("failed to create widget registry: %w", err) } - // Load user definitions if project path available + // Load the project's widgets, generating their definitions from the + // installed .mpk first — `.mxcli/` is gitignored, so a fresh clone has none + // (ako/mxcli#663). projectPath, _ := cmd.Flags().GetString("project") - if projectPath != "" { - if err := registry.LoadUserDefinitions(projectPath); err != nil { - log.Printf("warning: loading user widget definitions: %v", err) - } + if err := executor.LoadProjectWidgetDefinitions(registry, projectPath); err != nil { + log.Printf("warning: loading user widget definitions: %v", err) } defs := registry.All() diff --git a/mdl-examples/bug-tests/663-describe-widget-installed-uninitialized.mdl b/mdl-examples/bug-tests/663-describe-widget-installed-uninitialized.mdl new file mode 100644 index 0000000000..85129befcc --- /dev/null +++ b/mdl-examples/bug-tests/663-describe-widget-installed-uninitialized.mdl @@ -0,0 +1,21 @@ +-- ako/mxcli#663: DESCRIBE WIDGET called an installed widget "unknown". +-- +-- On a fresh clone (no .mxcli/ — it is gitignored) with Fieldset installed in +-- widgets/, this reported: +-- +-- Error: unknown widget "fieldset" — use an MDL keyword (barcodescanner, ...) +-- or a full widget id (com.mendix.widget…). +-- +-- DESCRIBE WIDGET now generates the project's widget definitions from its +-- installed .mpk before resolving the name, as exec and check already did. +-- +-- To reproduce, run against a project with Fieldset installed after +-- `rm -rf .mxcli/widgets`: +-- mxcli exec mdl-examples/bug-tests/663-describe-widget-installed-uninitialized.mdl -p app.mpr +-- +-- Both spellings must describe the widget. The id form has to be QUOTED — +-- the old error suggested it unquoted, which is a parse error. + +describe widget fieldset; + +describe widget 'com.mendix.widget.web.fieldset.Fieldset'; diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index 93bb98cc84..51626bd0e1 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -61,28 +61,7 @@ func LoadWidgetRegistry(projectPath string) *WidgetRegistry { return nil } if projectPath != "" { - // Generate the project's .def.json files from its installed .mpk when - // they are missing or behind this build, exactly as the page builder - // does before it reads them (cmd_pages_builder.go). Without this the - // validator and the builder read DIFFERENT registries, and the - // difference pointed the wrong way: on a project that had never run - // `mxcli widget init`, `check -p --references` reported every installed - // widget as "not a widget in this project" while `exec --no-check` - // wrote the page and generated the definitions on its way past - // (mendixlabs/mxcli#1135). check is meant to be the strict gate and - // exec the thing that runs; here it was inverted, and the script it - // blocked was one describe had just emitted. - // - // The self-healing is what made it read as flaky: the first exec writes - // the definitions and every check after it passes. - // - // Best-effort. A project whose definitions cannot be written — read-only - // checkout, no widgets/ at all — gets the registry it got before, which - // is strictly better than failing the check over a cache. - if _, err := RefreshStaleWidgetDefinitions(projectPath); err != nil { - log.Printf("warning: updating widget definitions: %v", err) - } - _ = registry.LoadUserDefinitions(projectPath) + _ = LoadProjectWidgetDefinitions(registry, projectPath) registry.projectPath = projectPath // The validator and DESCRIBE WIDGET must agree about which properties a // widget has; they read different sources, so the definition is topped up @@ -93,6 +72,37 @@ func LoadWidgetRegistry(projectPath string) *WidgetRegistry { return registry } +// LoadProjectWidgetDefinitions loads a project's widget definitions into +// registry, first generating `.mxcli/widgets/*.def.json` from the project's +// installed .mpk when they are missing or behind this build — exactly as the +// page builder does before it reads them (cmd_pages_builder.go). +// +// Every reader of the project's widgets must go through here. `.mxcli/` is +// gitignored, so a clone, a CI container or a new machine never has the +// definitions; a reader that only calls LoadUserDefinitions knows the nine +// embedded widgets and calls every installed one unknown. That happened twice: +// `check -p --references` reported installed widgets as "not a widget in this +// project" while exec wrote the page (mendixlabs/mxcli#1135), and DESCRIBE +// WIDGET / `widget list` called `fieldset` unknown while page authoring +// accepted it and DESCRIBE PAGE emitted it (ako/mxcli#663). +// +// The self-healing is what made both read as flaky: the first exec writes the +// definitions and every reader after it is right. +// +// Best-effort. A project whose definitions cannot be written — read-only +// checkout, no widgets/ at all — gets the registry it got before, which is +// strictly better than failing over a cache. The returned error is +// LoadUserDefinitions' (a malformed .def.json), for callers that report it. +func LoadProjectWidgetDefinitions(registry *WidgetRegistry, projectPath string) error { + if registry == nil || projectPath == "" { + return nil + } + if _, err := RefreshStaleWidgetDefinitions(projectPath); err != nil { + log.Printf("warning: updating widget definitions: %v", err) + } + return registry.LoadUserDefinitions(projectPath) +} + // ValidateWidgetPropertiesForStatement runs widget property validation on a // single statement using a pre-loaded registry. Returns no violations for // statements that don't carry pluggable widgets (everything except diff --git a/mdl/executor/widget_describe.go b/mdl/executor/widget_describe.go index 3f95962b33..ad763b7f6c 100644 --- a/mdl/executor/widget_describe.go +++ b/mdl/executor/widget_describe.go @@ -37,13 +37,11 @@ func DescribeWidget(arg, projectPath string) (*WidgetDescription, error) { if err != nil { return nil, mdlerrors.NewBackend("widget registry init", err) } - if projectPath != "" { - _ = registry.LoadUserDefinitions(projectPath) - } + _ = LoadProjectWidgetDefinitions(registry, projectPath) widgetID, def := resolveWidgetTarget(registry, arg) if widgetID == "" { - return nil, widgetNotFoundError(registry, arg) + return nil, widgetNotFoundError(registry, arg, projectPath) } desc := WidgetDescription{WidgetID: widgetID} @@ -236,19 +234,34 @@ var builtinWidgetAliases = map[string]string{ } // widgetNotFoundError builds a helpful error listing the known MDL names. -func widgetNotFoundError(registry *WidgetRegistry, arg string) error { +// +// Every remedy it names must work (ako/mxcli#663). It used to suggest "a full +// widget id (com.mendix.widget…)", which is a parse error in DESCRIBE WIDGET — +// the id has to be quoted there — and it listed only the embedded keywords of a +// project whose installed widgets had simply not been loaded. +func widgetNotFoundError(registry *WidgetRegistry, arg, projectPath string) error { + seen := map[string]bool{} var names []string - for _, d := range registry.All() { - if d.MDLName != "" { - names = append(names, d.MDLName) + add := func(n string) { + if n != "" && !seen[n] { + seen[n] = true + names = append(names, n) } } + for _, d := range registry.All() { + add(d.MDLName) + } for alias := range builtinWidgetAliases { - names = append(names, strings.ToLower(alias)) + add(strings.ToLower(alias)) } sort.Strings(names) - return fmt.Errorf("unknown widget %q — use an MDL keyword (%s) or a full widget id (com.mendix.widget…). Run `mxcli widget list` to see all", - arg, strings.Join(names, ", ")) + scope := "installed in this project's widgets/" + if projectPath == "" { + scope = "built into mxcli (no project is open, so a project's installed widgets are not known; pass -p )" + } + return fmt.Errorf("unknown widget %q: no widget by that name is %s. Known MDL keywords: %s. "+ + "A widget can also be named by its quoted id ('com.mendix.widget.web.combobox.Combobox'); `mxcli widget list` shows the ids", + arg, scope, strings.Join(names, ", ")) } // projectDirOf returns the directory containing widgets/ for a project path diff --git a/mdl/executor/widget_describe_uninitialized_test.go b/mdl/executor/widget_describe_uninitialized_test.go new file mode 100644 index 0000000000..bf94ca31cb --- /dev/null +++ b/mdl/executor/widget_describe_uninitialized_test.go @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +// projectWithInstalledFieldset is a fresh clone: Fieldset's .mpk is in widgets/ +// and .mxcli/ — gitignored — does not exist. +func projectWithInstalledFieldset(t *testing.T) string { + t.Helper() + dir := t.TempDir() + if err := os.MkdirAll(filepath.Join(dir, "widgets"), 0o755); err != nil { + t.Fatal(err) + } + const mpk = "com.mendix.widget.web.Fieldset.mpk" + data, err := os.ReadFile(filepath.Join("..", "..", "testdata", "expr-checker", "widgets", mpk)) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "widgets", mpk), data, 0o644); err != nil { + t.Fatal(err) + } + return filepath.Join(dir, "App.mpr") +} + +// ako/mxcli#663. On a project that never ran `mxcli widget init`: +// +// $ mxcli -p App.mpr -c "DESCRIBE WIDGET fieldset" +// Error: unknown widget "fieldset" — use an MDL keyword (barcodescanner, … +// +// while page authoring accepted `fieldset` and DESCRIBE PAGE emitted it. The +// page builder and the validator both generate the project's definitions from +// its installed .mpk before reading them; DescribeWidget read only whatever was +// already in .mxcli/widgets/, so it knew the nine embedded widgets. +func TestDescribeWidget_InstalledWidgetNeedsNoWidgetInit(t *testing.T) { + desc, err := DescribeWidget("fieldset", projectWithInstalledFieldset(t)) + if err != nil { + t.Fatalf("DESCRIBE WIDGET fieldset on an installed-but-uninitialised project: %v", err) + } + if desc.WidgetID != "com.mendix.widget.web.fieldset.Fieldset" { + t.Errorf("WidgetID = %q, want com.mendix.widget.web.fieldset.Fieldset", desc.WidgetID) + } + if desc.MDLName != "fieldset" { + t.Errorf("MDLName = %q, want fieldset", desc.MDLName) + } +} + +// Control: in the same project a name that is genuinely not installed must still +// be an error — and the error must not send the reader down a path that cannot +// succeed. The old message suggested an unquoted widget id, which is a parse +// error in DESCRIBE WIDGET; the quoted form is the one the grammar accepts. +func TestDescribeWidget_UnknownWidgetErrorNamesAWorkingForm(t *testing.T) { + _, err := DescribeWidget("fieldsett", projectWithInstalledFieldset(t)) + if err == nil { + t.Fatal("want an error for a widget that is not installed, got none") + } + msg := err.Error() + if !strings.Contains(msg, "fieldset") || !strings.Contains(msg, "'com.mendix.widget.web.") { + t.Errorf("error should list the installed keyword and a quoted widget id, got: %s", msg) + } +} + +// Without a project only the embedded widgets are known, and the error should +// say that a project is what brings the rest in. +func TestDescribeWidget_UnknownWidgetWithoutProjectPointsAtTheProject(t *testing.T) { + _, err := DescribeWidget("fieldset", "") + if err == nil { + t.Fatal("want an error with no project, got none") + } + if !strings.Contains(err.Error(), "-p") { + t.Errorf("error should say a project (-p) is needed for installed widgets, got: %s", err) + } +} From 618c6004eaf4807ad200fc3b3fe4b730590fb1e7 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 00:44:23 +0000 Subject: [PATCH 14/47] fix: qualify inherited associations with their declaring module (#662) DESCRIBE PAGE on Administration.Account_New emitted `Attribute: UserRoles`, and exec wrote it back as `Administration.UserRoles`: [CE1613] "The selected association 'Administration.UserRoles' no longer exists." Administration.Account extends System.User, which declares the association. A bare association name was qualified with an entity's MODULE rather than looked up. Before f0d1aea80 (issuetracker #19) the combobox used the option list's module, which happened to be right here. That commit switched to the page entity's module, which is wrong for any association inherited from another module. Bisected: f0d1aea80^ writes 0 combobox CE1613s on this page, f0d1aea80 writes 3. - resolveAssociationPathIn now resolves a bare name to the association with that name on the context entity or a generalization (nearest first), and qualifies it with its declaring module. Unknown or ambiguous names keep the previous guess. All four call sites go through it. - resolveAssociationAttributePath resolves each hop from the entity that hop starts at, not from the path's first entity. - DESCRIBE keeps the module qualifier on a combobox or dropdown-filter association declared outside the context entity's module. Round trip of all 17 pages of an 11.13.0 project with Administration: unfixed 5 errors (as reported), fixed 1: the CE0642 split out as #664. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + ...inherited-association-module-qualifier.mdl | 59 +++++++ .../cmd_pages_builder_inherited_assoc_test.go | 156 ++++++++++++++++++ mdl/executor/cmd_pages_builder_input.go | 89 +++++++++- mdl/executor/cmd_pages_builder_v3.go | 4 +- mdl/executor/cmd_pages_describe_parse.go | 4 +- mdl/executor/cmd_pages_describe_pluggable.go | 30 +++- .../cmd_pages_describe_pluggable_test.go | 30 ++++ 8 files changed, 368 insertions(+), 5 deletions(-) create mode 100644 mdl-examples/bug-tests/662-inherited-association-module-qualifier.mdl create mode 100644 mdl/executor/cmd_pages_builder_inherited_assoc_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index b095f0c388..c93ac269db 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -691,3 +691,4 @@ {"area":"mdl/executor","date":"2026-09-24","symptom":"`mxcli check -p … --references` passes a view entity whose association-path join is written `join s/Mod.A_B/System.UserRole AS r` (uppercase AS) even when a pass-through column from `r` declares the wrong string length; mxbuild then fails with CE6770 \"View Entity is out of sync with the OQL Query.\" Lowercase `as` reports MDL031 correctly.","cause":"extractAliasMap matched the path join case-insensitively ((?i)…(?:as\\s+)?) but then recovered the path from match[0] with strings.TrimSuffix(path, \"as\") — case-sensitive — so with `AS` the path kept a trailing ` AS`, the end-anchored lastEntity regex failed, and the alias was never mapped. Every column from that alias silently went without type inference.","file":"mdl/executor/oql_type_inference.go","insight":"A (?i) regex followed by string surgery on the whole match reintroduces case sensitivity by the back door: capture every piece you need as its own group instead of trimming it back out. The tell is a single control table varying only the case of one keyword — every other keyword in upper case was harmless, which points straight at code that handles that one token outside the regex. An unresolved alias is silent (the checker skips unknown types rather than reporting), so the symptom is a check that PASSES; test at extractAliasMap directly, and control with the unfixed binary on a real project (it printed `Check passed!` for AS, the error for as). DESCRIBE prints AS in upper case, so round-tripped OQL hits this by default.","refs":["#652"],"ce":["CE6770"],"rules":["MDL031"]} {"area":"mdl/executor","date":"2026-09-24","symptom":"`alter page M.P { insert into dv { use fragment SaveCancelFooter } }` passed `check` and failed in exec: \"failed to insert: failed to build widgets: failed to build widget SaveCancelFooter: unsupported widget type: USE_FRAGMENT\". Same for REPLACE, and for a fragment nested inside an inserted container.","cause":"Fragment/building-block expansion (pageBuilder.expandFragments) was called only on the CREATE PAGE / snippet / layout paths. ALTER PAGE's applyInsertWidgetMutator / applyReplaceWidgetMutator passed op.Widgets straight to buildWidgetsFromAST, whose pageBuilder even carried ctx.Fragments — the registry was wired, the expansion call was not. Second defect found on the way: cloneWidget copied only Type/Name/Properties/Children, dropping Specialization and TypeIsGeneric, so a cloned `template for` lost its routing.","file":"`mdl/executor/cmd_alter_page.go` (expandAlterFragments, called first in the INSERT and REPLACE mutators), `mdl/executor/cmd_pages_builder_v3.go` (cloneWidget copies the whole struct)","insight":"Expand before ANYTHING inspects the widget list, not just before the build: the duplicate-name check, allColumns and allListViewTemplates all read op.Widgets, and seeing the sentinel they check the fragment's name instead of its widgets' names. When a registry is threaded into a builder, grep for the call that consumes it (`expandFragments`), not for the field — the field being set on every pageBuilder is what made the ALTER path look covered. A field-by-field clone is a silent-drop hazard; `c := *w` then deep-copy the reference fields.","refs":["#572"]} {"area":"mdl/executor","date":"2026-09-24","symptom":"`describe layout Atlas_Core.Phone_BottomBar` emitted `-- Forms$SimpleMenuBar (simpleMenuBar1) -- NOT re-executable: mxcli cannot author this widget, so re-running this script would drop it`, so a phone layout written in MDL had no bottom bar and a describe -> exec copy of Atlas's lost it","cause":"No keyword, builder, writer or describer for Forms$SimpleMenuBar, although modelsdk/gen already had SimpleMenuBar and MenuDocumentSource. And the widget's whole point is WHICH menu document it renders (a Forms$MenuDocumentSource in MenuSource), a source kind no menu widget could express — menubar/navigationtree always wrote a Forms$NavigationSource, so one pointed at a menu document described as `menubar m` and replayed onto the Responsive profile. Layouts were also never reference-checked, so a new `Menu:` typo would have passed --references and hit CE1613 at build","file":"mdl/grammar/MDLLexer.g4 + domains/MDLPage.g4 + domains/MDLSettings.g4 (SIMPLEMENUBAR), mdl/executor/cmd_pages_builder_v3.go (buildSimpleMenuBarV3, menuSourceV3), mdl/backend/modelsdk/widget_write.go (menuSourceToGen), mdl/executor/cmd_pages_describe_parse.go + cmd_pages_describe_output.go, mdl/executor/helpers.go + validate.go (menu refs, CreateLayoutStmt), sdk/pages/pages_widgets_advanced.go","insight":"**Measure the stored shape before designing the syntax** — dumping the widget from a blank project (`mxcli new --version 11.14.0`, `bson dump --type layout`) showed the source was a menu DOCUMENT, not a profile, which is what made this a source-kind feature rather than a one-keyword copy of `menubar`. Treat MenuSource as one slot with two subtypes on all three menu widgets (one helper each side), or the siblings keep silently rewriting a menu-document source to Responsive. The control that justified the reference check: `menu: Bug573.No_Such_Menu` -> `mxcli check --references` passed, `mx check` CE1613 'The selected menu ... no longer exists.' at Simple menu bar. MDL has no `show menus`, so the not-found message lists the menus that exist. Adding reference validation to CreateLayoutStmt is new coverage for EVERY widget reference in a layout, not just menus","refs":["ako/mxcli#573"],"ce":["CE1613"]} +{"area": "mdl/executor", "symptom": "`DESCRIBE PAGE Administration.Account_New` → `exec` → mx check: **CE1613** \"The selected association 'Administration.UserRoles' no longer exists\" (also User_Language, User_TimeZone, and a DataGrid2 column `Administration.Account.UserRoles/Name`). `check --references` and `exec` both report success; describe → exec → describe is byte-identical", "cause": "A bare association name was qualified with the MODULE of an entity instead of looked up. Administration.Account extends System.User, which declares UserRoles, so the page entity's module named a nonexistent association. The describer always emitted the bare name (shortAttributeName, since 41d01f01); the regression was f0d1aea80 (issuetracker #19), which switched the combobox writer from the option list's module (right here by coincidence) to the page entity's. resolveAssociationAttributePath also qualified every hop against the path's START entity", "file": "`mdl/executor/cmd_pages_builder_input.go` (`resolveAssociationPathIn` → `declaredAssociationQN`), `cmd_pages_builder_v3.go` (`resolveAssociationAttributePath` per-hop context), `cmd_pages_describe_pluggable.go` (`associationRefForContext`)", "insight": "**Two wrong heuristics each fixed the other's case**: 'module of the option list' broke issuetracker #19, 'module of the context entity' broke #662 — both guess a module from an entity name where the model can be asked. Resolve by lookup: the association with that name having an end on the context entity or a generalization, nearest first, qualified with its DECLARING module; ambiguous or unknown keeps the old guess so the validator reports the author's spelling. To find which change regressed it, build the suspect commit and its parent side by side and diff `mx check` on a copy of the project — describe output was identical on all three builds, so the writer changed, not the describer. The bisect subject must be an INHERITED association from a DIFFERENT module whose option list lives in the declaring module; a same-module fixture passes both old rules. A full-project round trip (every page, describe → exec → mx check) found the DataGrid2 `UserRoles/Name` column the issue did not name — the resolver has four call sites, fix it once there", "refs": ["ako/mxcli#662", "issuetracker #19", "ako/mxcli#664"], "ce": ["CE1613"], "date": "2026-09-25"} diff --git a/mdl-examples/bug-tests/662-inherited-association-module-qualifier.mdl b/mdl-examples/bug-tests/662-inherited-association-module-qualifier.mdl new file mode 100644 index 0000000000..89328439d4 --- /dev/null +++ b/mdl-examples/bug-tests/662-inherited-association-module-qualifier.mdl @@ -0,0 +1,59 @@ +-- ============================================================================ +-- ako/mxcli#662: an association inherited through a generalization was +-- written into the page entity's module +-- ============================================================================ +-- +-- Symptom (describe → exec on the stock Administration.Account_New, where +-- Administration.Account extends System.User): +-- [error] [CE1613] "The selected association 'Administration.UserRoles' no longer exists." at Combo box 'comboBox1' +-- [error] [CE1613] "The selected association 'Administration.User_Language' no longer exists." at Combo box 'comboBox3' +-- [error] [CE1613] "The selected association 'Administration.User_TimeZone' no longer exists." at Combo box 'comboBox2' +-- with `check --references` and `exec` both reporting success. +-- +-- Cause: a bare association name was qualified with the module of the context +-- entity. UserRoles is declared on System.User, so the right module is System. +-- (Before f0d1aea80 it was qualified with the option list's module, which was +-- right here only because System.UserRole happens to live in System too.) +-- +-- Fix: a bare name resolves to the association with that name on the context +-- entity or one of its generalizations, qualified with its declaring module; +-- each hop of an association path resolves from the entity that hop starts at. +-- DESCRIBE now keeps the qualifier whenever the association is declared outside +-- the context entity's module. +-- +-- Verify: `mx check` must be clean after exec, and again after a +-- describe → exec round trip. Comparing describe output is NOT a test — the +-- bare name round-trips byte-identically while the model is broken. +-- ============================================================================ + +create entity MyFirstModule.Member extends System.User ( + FullName: String(200) +); +/ + +create or replace page MyFirstModule.Member_Edit +( Title: 'Member', Layout: Atlas_Core.Atlas_Default, Params: { $Member: MyFirstModule.Member } ) +{ + dataview dv (datasource: $Member) { + textbox txtName (label: 'Full name', attribute: FullName) + -- Bare names, declared on System.User: must be written as System.*. + combobox cmbRoles ( + label: 'User role(s)', + attribute: UserRoles, + datasource: database System.UserRole, + CaptionAttribute: Name + ) + combobox cmbLanguage ( + label: 'Language', + attribute: User_Language, + datasource: database System.Language, + CaptionAttribute: Description + ) + -- Association path whose hop is inherited. + textbox txtLangCode (label: 'Language code', attribute: User_Language/Code) + } +} +/ + +-- Expect `attribute: System.UserRoles` / `System.User_Language` on the comboboxes. +describe page MyFirstModule.Member_Edit; diff --git a/mdl/executor/cmd_pages_builder_inherited_assoc_test.go b/mdl/executor/cmd_pages_builder_inherited_assoc_test.go new file mode 100644 index 0000000000..e9c3c6c69b --- /dev/null +++ b/mdl/executor/cmd_pages_builder_inherited_assoc_test.go @@ -0,0 +1,156 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" +) + +// ako/mxcli#662: a bare association name is qualified with the module that +// DECLARES it, found by walking the context entity's generalization chain — not +// with the module of whichever entity happens to be the context. +// +// Administration.Account extends System.User, and UserRoles is declared on +// System.User. `DESCRIBE PAGE Administration.Account_New` emits +// `combobox (Attribute: UserRoles, DataSource: database from System.UserRole)`, +// and exec qualified the bare name with the page entity's module, writing +// `Administration.UserRoles`: +// +// [CE1613] "The selected association 'Administration.UserRoles' no longer exists." +// +// Before f0d1aea80 (issuetracker #19) the name was qualified with the OPTION +// LIST's module and came out right here only by accident — System.UserRole +// happens to live where UserRoles is declared. Both rules guess from a module +// name; the fixture below holds cases each of them gets wrong. +func pageInheritedAssocFixture(entityContext string) *pageBuilder { + const ( + sysID = model.ID("mod-system") + adminID = model.ID("mod-admin") + itID = model.ID("mod-it") + userID = model.ID("e-user") + roleID = model.ID("e-userrole") + langID = model.ID("e-language") + accountID = model.ID("e-account") + issueID = model.ID("e-issue") + noteID = model.ID("e-note") + ) + return &pageBuilder{ + entityContext: entityContext, + paramEntityNames: map[string]string{}, + widgetScope: map[string]model.ID{}, + execCache: &executorCache{ + hierarchy: &ContainerHierarchy{moduleNames: map[model.ID]string{ + sysID: "System", + adminID: "Administration", + itID: "IT", + }}, + domainModels: []*domainmodel.DomainModel{ + { + ContainerID: sysID, + Entities: []*domainmodel.Entity{ + {BaseElement: model.BaseElement{ID: userID}, Name: "User"}, + {BaseElement: model.BaseElement{ID: roleID}, Name: "UserRole"}, + {BaseElement: model.BaseElement{ID: langID}, Name: "Language"}, + }, + Associations: []*domainmodel.Association{ + {Name: "UserRoles", ParentID: userID, ChildID: roleID, Type: domainmodel.AssociationTypeReferenceSet}, + {Name: "User_Language", ParentID: userID, ChildID: langID, Type: domainmodel.AssociationTypeReference}, + }, + }, + { + ContainerID: adminID, + Entities: []*domainmodel.Entity{ + {BaseElement: model.BaseElement{ID: accountID}, Name: "Account", GeneralizationRef: "System.User"}, + }, + }, + { + ContainerID: itID, + Entities: []*domainmodel.Entity{ + {BaseElement: model.BaseElement{ID: issueID}, Name: "Issue"}, + {BaseElement: model.BaseElement{ID: noteID}, Name: "Note"}, + }, + Associations: []*domainmodel.Association{ + {Name: "Note_Issue", ParentID: noteID, ChildID: issueID, Type: domainmodel.AssociationTypeReference}, + }, + CrossAssociations: []*domainmodel.CrossModuleAssociation{ + {Name: "Issue_Assignee", ParentID: issueID, ChildRef: "System.User", Type: domainmodel.AssociationTypeReference}, + // Declared in IT, navigated FROM the System side. + {Name: "Issue_Reporter", ParentID: issueID, ChildRef: "System.User", Type: domainmodel.AssociationTypeReference}, + }, + }, + }, + }, + } +} + +func TestResolveAssociationPathIn_DeclaringModule(t *testing.T) { + tests := []struct { + name, assoc, context, want string + }{ + {"inherited from a System parent (#662)", "UserRoles", "Administration.Account", "System.UserRoles"}, + {"inherited, second association (#662)", "User_Language", "Administration.Account", "System.User_Language"}, + {"declared on the context itself", "UserRoles", "System.User", "System.UserRoles"}, + {"cross-module FROM end (issuetracker #19)", "Issue_Assignee", "IT.Issue", "IT.Issue_Assignee"}, + {"reverse navigation from the other module's end", "Issue_Reporter", "Administration.Account", "IT.Issue_Reporter"}, + {"same-module TO end", "Note_Issue", "IT.Issue", "IT.Note_Issue"}, + // Unknown to the model: keep the historical guess rather than refuse, + // so the reference validator reports it by the name the author wrote. + {"unknown name falls back to context module", "Nope", "Administration.Account", "Administration.Nope"}, + {"already qualified is untouched", "Other.UserRoles", "Administration.Account", "Other.UserRoles"}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + pb := pageInheritedAssocFixture(tc.context) + if got := pb.resolveAssociationPathIn(tc.assoc, tc.context); got != tc.want { + t.Errorf("resolveAssociationPathIn(%q, %q) = %q, want %q", tc.assoc, tc.context, got, tc.want) + } + }) + } +} + +// The reported shape end to end through the widget engine: a ComboBox whose +// DataSource mapping has already moved entityContext to its option list, inside +// a data view over the specialization. +func TestResolveMapping_Association_InheritedFromGeneralization(t *testing.T) { + pb := pageInheritedAssocFixture("System.UserRole") // moved by the DataSource mapping + engine := &PluggableWidgetEngine{pageBuilder: pb, outerEntityContext: "Administration.Account"} + + mapping := PropertyMapping{PropertyKey: "attributeAssociation", Source: "Association", Operation: "association"} + w := &ast.WidgetV3{Properties: map[string]any{"Attribute": "UserRoles"}} + + ctx, err := engine.resolveMapping(mapping, w) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + if want := "System.UserRoles"; ctx.AssocPath != want { + t.Errorf("AssocPath = %q, want %q (Administration.UserRoles is CE1613 \"no longer exists\")", ctx.AssocPath, want) + } +} + +// A multi-hop attribute path qualifies EACH hop against the entity that hop +// starts from, not against the path's starting entity. +func TestResolveAssociationAttributePath_HopQualifiedPerStep(t *testing.T) { + pb := pageInheritedAssocFixture("IT.Note") + finalQN, steps, ok := pb.resolveAssociationAttributePath("Note_Issue/Issue_Assignee/UserRoles/Name") + if !ok { + t.Fatalf("path dropped (ok=false) — the third hop starts at System.User, and qualifying it against the path's start (IT.Note) gives IT.UserRoles") + } + if len(steps) != 3 || steps[0].Association != "IT.Note_Issue" || + steps[1].Association != "IT.Issue_Assignee" || steps[2].Association != "System.UserRoles" { + t.Errorf("steps = %+v", steps) + } + if finalQN != "System.UserRole.Name" { + t.Errorf("finalQN = %q, want System.UserRole.Name", finalQN) + } + + // Inherited association on the first hop. + pb = pageInheritedAssocFixture("Administration.Account") + _, steps, ok = pb.resolveAssociationAttributePath("User_Language/Code") + if !ok || len(steps) != 1 || steps[0].Association != "System.User_Language" { + t.Errorf("inherited hop: ok=%v steps=%+v, want System.User_Language", ok, steps) + } +} diff --git a/mdl/executor/cmd_pages_builder_input.go b/mdl/executor/cmd_pages_builder_input.go index 7d6fc7b2a6..b3dc8decf3 100644 --- a/mdl/executor/cmd_pages_builder_input.go +++ b/mdl/executor/cmd_pages_builder_input.go @@ -257,7 +257,12 @@ func (pb *pageBuilder) resolveAssociationPathIn(assocName, entityContext string) if strings.Contains(assocName, ".") { return assocName } - // Extract module name from entity context (e.g., "PgTest.Order" → "PgTest") + if qn, ok := pb.declaredAssociationQN(assocName, entityContext); ok { + return qn + } + // Not in the model (or no model loaded): guess the context entity's module + // (e.g., "PgTest.Order" → "PgTest"), so a misspelling is reported by the + // name the author wrote. if entityContext != "" { parts := strings.SplitN(entityContext, ".", 2) if len(parts) >= 1 { @@ -267,6 +272,88 @@ func (pb *pageBuilder) resolveAssociationPathIn(assocName, entityContext string) return assocName } +// declaredAssociationQN finds the association a bare name means from +// entityContext: the one with that name that has an end on the context entity +// or one of its generalizations, qualified with the module that DECLARES it. +// +// An association's module is where it is declared, which is neither the +// context entity's module nor the option list's. Qualifying with the first +// broke inherited associations (ako/mxcli#662): Administration.Account extends +// System.User, so `UserRoles` became `Administration.UserRoles` and mxbuild +// failed CE1613 "The selected association … no longer exists". Qualifying with +// the second was issuetracker #19. Each was right only where the two modules +// happened to coincide. +// +// The chain is walked nearest-first, so a specialization's own association +// wins over a same-named one on an ancestor. A name matching more than one +// association at the same level is ambiguous; ok is false and the caller keeps +// its fallback rather than picking one. +func (pb *pageBuilder) declaredAssociationQN(assocName, entityContext string) (string, bool) { + if entityContext == "" { + return "", false + } + // Without a model (a unit test building widgets in isolation) there is + // nothing to look up; leave the name to the caller's fallback. + if pb.backend == nil && (pb.execCache == nil || pb.execCache.domainModels == nil) { + return "", false + } + dms, err := pb.getDomainModels() + if err != nil { + return "", false + } + h, err := pb.getHierarchy() + if err != nil { + return "", false + } + parents, err := pb.entityGeneralizations() + if err != nil { + return "", false + } + + entityQN := make(map[model.ID]string) + for _, dm := range dms { + mod := h.GetModuleName(dm.ContainerID) + for _, e := range dm.Entities { + entityQN[e.ID] = mod + "." + e.Name + } + } + // Every association with this name, keyed by the entities at its ends. + type candidate struct{ qn, from, to string } + var candidates []candidate + for _, dm := range dms { + mod := h.GetModuleName(dm.ContainerID) + for _, a := range dm.Associations { + if a.Name == assocName { + candidates = append(candidates, candidate{mod + "." + a.Name, entityQN[a.ParentID], entityQN[a.ChildID]}) + } + } + for _, ca := range dm.CrossAssociations { + if ca.Name == assocName { + candidates = append(candidates, candidate{mod + "." + ca.Name, entityQN[ca.ParentID], ca.ChildRef}) + } + } + } + + seen := map[string]bool{} + for cur := entityContext; cur != "" && !seen[cur]; cur = parents[cur] { + seen[cur] = true + found := "" + for _, c := range candidates { + if c.from != cur && c.to != cur { + continue + } + if found != "" && found != c.qn { + return "", false + } + found = c.qn + } + if found != "" { + return found, true + } + } + return "", false +} + // resolveSnippetRef resolves a snippet qualified name to its ID. func (pb *pageBuilder) resolveSnippetRef(snippetRef string) (model.ID, error) { if snippetRef == "" { diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index b93b31a958..9d9d8708d3 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -2086,7 +2086,9 @@ func (pb *pageBuilder) resolveAssociationAttributePath(attrRef string) (finalQN steps = make([]pages.AttributeRefStep, 0, len(segs)-1) for _, seg := range segs[:len(segs)-1] { - assocQN := pb.resolveAssociationPath(seg) + // Against the entity THIS hop starts from — qualifying every hop with + // the path's start named a later hop into the wrong module. (#662) + assocQN := pb.resolveAssociationPathIn(seg, current) dest, ok := pb.associationDestination(assocQN, current) if !ok { return "", nil, false diff --git a/mdl/executor/cmd_pages_describe_parse.go b/mdl/executor/cmd_pages_describe_parse.go index 817e3d89f7..4b388b2471 100644 --- a/mdl/executor/cmd_pages_describe_parse.go +++ b/mdl/executor/cmd_pages_describe_parse.go @@ -386,7 +386,7 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s if widget.RenderMode == "combobox" { widget.DataSource = extractComboBoxDataSource(ctx, w) if widget.DataSource != nil { - widget.Content = extractCustomWidgetPropertyAssociation(ctx, w, "attributeAssociation") + widget.Content = associationRefForContext(extractCustomWidgetPropertyAssociationQN(ctx, w, "attributeAssociation"), inheritedCtx) widget.CaptionAttribute = extractCustomWidgetPropertyAttributeRef(ctx, w, "optionsSourceAssociationCaptionAttribute") } // The on-change action, in BOTH modes — the def maps `onChangeEvent` @@ -408,7 +408,7 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s if widget.RenderMode == "dropdownfilter" && extractCustomWidgetPropertyString(ctx, w, "baseType") == "ref" { widget.DataSource = extractCustomWidgetPropertyDataSource(ctx, w, "refOptions") - widget.Content = extractCustomWidgetPropertyAssociation(ctx, w, "refEntity") + widget.Content = associationRefForContext(extractCustomWidgetPropertyAssociationQN(ctx, w, "refEntity"), inheritedCtx) widget.CaptionAttribute = extractCustomWidgetPropertyAttributeRef(ctx, w, "refCaption") } // For DataGrid2, also extract datasource, columns, CONTROLBAR widgets, paging, and selection diff --git a/mdl/executor/cmd_pages_describe_pluggable.go b/mdl/executor/cmd_pages_describe_pluggable.go index 74bcc92114..ecbded2054 100644 --- a/mdl/executor/cmd_pages_describe_pluggable.go +++ b/mdl/executor/cmd_pages_describe_pluggable.go @@ -908,6 +908,34 @@ func extractCustomWidgetPropertyAttributeRef(ctx *ExecContext, w map[string]any, // This is the symmetric counterpart of extractCustomWidgetPropertyAttributeRef, // handling the EntityRef storage format instead of AttributeRef. func extractCustomWidgetPropertyAssociation(ctx *ExecContext, w map[string]any, propertyKey string) string { + return shortAttributeName(extractCustomWidgetPropertyAssociationQN(ctx, w, propertyKey)) +} + +// associationRefForContext renders a stored association reference for MDL: +// bare when it is declared in the context entity's module, qualified otherwise. +// +// exec qualifies a bare name by looking the association up from the context, +// but the qualified form means the same thing without a lookup — so it is what +// describe emits whenever the modules differ, or the context is unknown. A +// bare `UserRoles` on a page over Administration.Account (extends System.User) +// was written back as `Administration.UserRoles` → CE1613 (ako/mxcli#662). +func associationRefForContext(assocQN, entityContext string) string { + if assocQN == "" { + return "" + } + dot := strings.Index(assocQN, ".") + if dot < 0 { + return assocQN + } + if ctxDot := strings.Index(entityContext, "."); ctxDot > 0 && entityContext[:ctxDot] == assocQN[:dot] { + return assocQN[dot+1:] + } + return assocQN +} + +// extractCustomWidgetPropertyAssociationQN returns the association a +// CustomWidget property binds, as stored: Module.Association. +func extractCustomWidgetPropertyAssociationQN(ctx *ExecContext, w map[string]any, propertyKey string) string { obj, ok := w["Object"].(map[string]any) if !ok { return "" @@ -942,7 +970,7 @@ func extractCustomWidgetPropertyAssociation(ctx *ExecContext, w map[string]any, continue } if assoc := extractString(stepMap["Association"]); assoc != "" { - return shortAttributeName(assoc) + return assoc } } } diff --git a/mdl/executor/cmd_pages_describe_pluggable_test.go b/mdl/executor/cmd_pages_describe_pluggable_test.go index 66574fd854..1c3469aa83 100644 --- a/mdl/executor/cmd_pages_describe_pluggable_test.go +++ b/mdl/executor/cmd_pages_describe_pluggable_test.go @@ -213,3 +213,33 @@ func TestCustomWidgetPropertyActionMap(t *testing.T) { t.Error("NoAction should read as unset (nil)") } } + +// ako/mxcli#662: DESCRIBE emitted an association bound through a +// generalization as a bare name — `Attribute: UserRoles` on a page over +// Administration.Account (extends System.User) — and exec qualified it with the +// page entity's module, writing `Administration.UserRoles` → CE1613. An +// association declared outside the context entity's module keeps its module. +func TestParseRawWidget_ComboBoxAssociation_QualifiedOutsideContextModule(t *testing.T) { + ctx := (&Executor{}).newExecContext(context.Background()) + tests := []struct { + name, assoc, entityCtx, want string + }{ + {"inherited from another module (#662)", "System.UserRoles", "Administration.Account", "System.UserRoles"}, + {"declared in the context's module stays bare", "MyFirstModule.Task_Category", "MyFirstModule.Task", "Task_Category"}, + {"unknown context keeps the qualifier", "MyFirstModule.Task_Category", "", "MyFirstModule.Task_Category"}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + w := buildComboBoxAssocWidget(tc.assoc, "System.UserRole.Name") + w["$Type"] = "CustomWidgets$CustomWidget" + w["Name"] = "comboBox1" + got := parseRawWidget(ctx, w, tc.entityCtx) + if len(got) != 1 { + t.Fatalf("parseRawWidget returned %d widgets, want 1", len(got)) + } + if got[0].Content != tc.want { + t.Errorf("Content = %q, want %q", got[0].Content, tc.want) + } + }) + } +} From ad3cc3c6008fa7b3f63ca8b9b86c45fd8390810e Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 00:50:53 +0000 Subject: [PATCH 15/47] fix(lsp): widget registry sees installed widgets without widget init MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ensureWidgetRegistry loaded only .mxcli/widgets/*.def.json, which a fresh clone lacks (gitignored), so completions offered the nine embedded widgets and the LSP's widget diagnostics read the same thin registry. With a project open it now uses executor.LoadWidgetRegistry — the registry check validates against — which generates the definitions from the installed .mpk first. The no-project path keeps loading global definitions as before. Follow-up to ako/mxcli#663. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + cmd/mxcli/lsp_completion.go | 24 +++++++-- cmd/mxcli/lsp_widget_registry_test.go | 53 +++++++++++++++++++ 3 files changed, 73 insertions(+), 5 deletions(-) create mode 100644 cmd/mxcli/lsp_widget_registry_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 194f28d0d6..d57acc719f 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -692,3 +692,4 @@ {"area":"mdl/executor","date":"2026-09-24","symptom":"`alter page M.P { insert into dv { use fragment SaveCancelFooter } }` passed `check` and failed in exec: \"failed to insert: failed to build widgets: failed to build widget SaveCancelFooter: unsupported widget type: USE_FRAGMENT\". Same for REPLACE, and for a fragment nested inside an inserted container.","cause":"Fragment/building-block expansion (pageBuilder.expandFragments) was called only on the CREATE PAGE / snippet / layout paths. ALTER PAGE's applyInsertWidgetMutator / applyReplaceWidgetMutator passed op.Widgets straight to buildWidgetsFromAST, whose pageBuilder even carried ctx.Fragments — the registry was wired, the expansion call was not. Second defect found on the way: cloneWidget copied only Type/Name/Properties/Children, dropping Specialization and TypeIsGeneric, so a cloned `template for` lost its routing.","file":"`mdl/executor/cmd_alter_page.go` (expandAlterFragments, called first in the INSERT and REPLACE mutators), `mdl/executor/cmd_pages_builder_v3.go` (cloneWidget copies the whole struct)","insight":"Expand before ANYTHING inspects the widget list, not just before the build: the duplicate-name check, allColumns and allListViewTemplates all read op.Widgets, and seeing the sentinel they check the fragment's name instead of its widgets' names. When a registry is threaded into a builder, grep for the call that consumes it (`expandFragments`), not for the field — the field being set on every pageBuilder is what made the ALTER path look covered. A field-by-field clone is a silent-drop hazard; `c := *w` then deep-copy the reference fields.","refs":["#572"]} {"area":"mdl/executor","date":"2026-09-24","symptom":"`describe layout Atlas_Core.Phone_BottomBar` emitted `-- Forms$SimpleMenuBar (simpleMenuBar1) -- NOT re-executable: mxcli cannot author this widget, so re-running this script would drop it`, so a phone layout written in MDL had no bottom bar and a describe -> exec copy of Atlas's lost it","cause":"No keyword, builder, writer or describer for Forms$SimpleMenuBar, although modelsdk/gen already had SimpleMenuBar and MenuDocumentSource. And the widget's whole point is WHICH menu document it renders (a Forms$MenuDocumentSource in MenuSource), a source kind no menu widget could express — menubar/navigationtree always wrote a Forms$NavigationSource, so one pointed at a menu document described as `menubar m` and replayed onto the Responsive profile. Layouts were also never reference-checked, so a new `Menu:` typo would have passed --references and hit CE1613 at build","file":"mdl/grammar/MDLLexer.g4 + domains/MDLPage.g4 + domains/MDLSettings.g4 (SIMPLEMENUBAR), mdl/executor/cmd_pages_builder_v3.go (buildSimpleMenuBarV3, menuSourceV3), mdl/backend/modelsdk/widget_write.go (menuSourceToGen), mdl/executor/cmd_pages_describe_parse.go + cmd_pages_describe_output.go, mdl/executor/helpers.go + validate.go (menu refs, CreateLayoutStmt), sdk/pages/pages_widgets_advanced.go","insight":"**Measure the stored shape before designing the syntax** — dumping the widget from a blank project (`mxcli new --version 11.14.0`, `bson dump --type layout`) showed the source was a menu DOCUMENT, not a profile, which is what made this a source-kind feature rather than a one-keyword copy of `menubar`. Treat MenuSource as one slot with two subtypes on all three menu widgets (one helper each side), or the siblings keep silently rewriting a menu-document source to Responsive. The control that justified the reference check: `menu: Bug573.No_Such_Menu` -> `mxcli check --references` passed, `mx check` CE1613 'The selected menu ... no longer exists.' at Simple menu bar. MDL has no `show menus`, so the not-found message lists the menus that exist. Adding reference validation to CreateLayoutStmt is new coverage for EVERY widget reference in a layout, not just menus","refs":["ako/mxcli#573"],"ce":["CE1613"]} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "On a fresh clone, `DESCRIBE WIDGET fieldset` reports `unknown widget \"fieldset\" — use an MDL keyword (barcodescanner, …) or a full widget id (com.mendix.widget…)` although Fieldset is installed in widgets/, page authoring accepts `fieldset` and DESCRIBE PAGE emits it. `mxcli widget list -p` shows 9 definitions. Both suggested remedies fail: `fieldset` IS the keyword, and the unquoted id is a parse error.", "cause": "DescribeWidget and `widget list` called only `LoadUserDefinitions`, which reads `.mxcli/widgets/*.def.json` — gitignored, so absent on every clone. The page builder and (since mendixlabs/mxcli#1135) LoadWidgetRegistry call RefreshStaleWidgetDefinitions first; these two readers were missed by that fix.", "file": "mdl/executor/validate_widgets.go (LoadProjectWidgetDefinitions, shared), mdl/executor/widget_describe.go (DescribeWidget, widgetNotFoundError), cmd/mxcli/cmd_widget.go (runWidgetList)", "insight": "**Second instance of the same class as mendixlabs/mxcli#1135 — the first fix patched one reader, not the call pattern.** `grep -rn LoadUserDefinitions` lists every reader of the project widget registry; each one that is not preceded by RefreshStaleWidgetDefinitions is this bug. The fix is a single LoadProjectWidgetDefinitions that does both, so the next reader cannot pick half. Still open: `cmd/mxcli/lsp_completion.go` (LSP completions) calls LoadUserDefinitions directly. **Reproduce cold or it is invisible**: `rm -rf .mxcli/widgets` first — any earlier exec/check in the same project self-heals it (the reporter's earlier sweeps read 43 keywords for exactly this reason). `testdata/expr-checker/widgets/com.mendix.widget.web.Fieldset.mpk` copied into a temp dir is the whole fixture; never use the checked-in project, because refresh writes .def.json into it. **Every remedy an error names must be run**: the old message's 'full widget id' is only valid QUOTED in DESCRIBE WIDGET (`'com.mendix.widget.web.fieldset.Fieldset'`), and the keyword list must be deduped (registry `datagrid` + builtin alias `DATAGRID` printed it twice).", "refs": ["ako/mxcli#663", "mendixlabs/mxcli#1135"]} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "In the editor, on a project that never ran `mxcli widget init`, widget completions offer only the nine embedded widgets (textfilter, combobox, … barcodescanner) although e.g. Fieldset is installed in widgets/; LSP widget diagnostics read the same thin registry.", "cause": "cmd/mxcli/lsp_completion.go ensureWidgetRegistry built its registry with NewWidgetRegistry + LoadUserDefinitions, which reads only the gitignored .mxcli/widgets/*.def.json. The last reader left over after mendixlabs/mxcli#1135 and ako/mxcli#663 fixed check, DESCRIBE WIDGET and `widget list`.", "file": "cmd/mxcli/lsp_completion.go (ensureWidgetRegistry)", "insight": "**The LSP registry also feeds diagnostics** (lsp_diagnostics.go calls ensureWidgetRegistry), so it is not just thin completions: route it through executor.LoadWidgetRegistry, the SAME registry check validates against (refresh + .mpk property enrichment), rather than LoadProjectWidgetDefinitions — otherwise editor diagnostics and `check` can still disagree on a widget's properties. LoadWidgetRegistry(\"\") skips the global ~/.mxcli/widgets, so keep the old LoadUserDefinitions(\"\") path when no project is open. The registry is cached by sync.Once per server, so a test must use a fresh mdlServer{mprPath: …} over a temp dir holding one .mpk. After this, `grep -rn LoadUserDefinitions` outside widget_registry.go should show only LoadProjectWidgetDefinitions, the page builder and this no-project branch — anything new there is this bug again.", "refs": ["ako/mxcli#663", "mendixlabs/mxcli#1135"]} diff --git a/cmd/mxcli/lsp_completion.go b/cmd/mxcli/lsp_completion.go index 4e60c22fe8..0c6ea087cd 100644 --- a/cmd/mxcli/lsp_completion.go +++ b/cmd/mxcli/lsp_completion.go @@ -116,12 +116,26 @@ func (s *mdlServer) widgetRegistryCompletions() []protocol.CompletionItem { // appear until the server is restarted. func (s *mdlServer) ensureWidgetRegistry() { s.widgetCompletionsOnce.Do(func() { - registry, err := executor.NewWidgetRegistry() - if err != nil { - return + // With a project, load exactly the registry `check` validates against: + // definitions generated from the installed .mpk first, since .mxcli/ is + // gitignored and a fresh clone has none. Loading .def.json alone offered + // only the nine embedded widgets, and the diagnostics built on this + // registry disagreed with check (ako/mxcli#663). + var registry *executor.WidgetRegistry + if s.mprPath != "" { + registry = executor.LoadWidgetRegistry(s.mprPath) + } else { + r, err := executor.NewWidgetRegistry() + if err != nil { + return + } + if err := r.LoadUserDefinitions(""); err != nil { + log.Printf("warning: loading user widget definitions for LSP: %v", err) + } + registry = r } - if err := registry.LoadUserDefinitions(s.mprPath); err != nil { - log.Printf("warning: loading user widget definitions for LSP: %v", err) + if registry == nil { + return } s.widgetRegistry = registry for _, def := range registry.All() { diff --git a/cmd/mxcli/lsp_widget_registry_test.go b/cmd/mxcli/lsp_widget_registry_test.go new file mode 100644 index 0000000000..dda431040f --- /dev/null +++ b/cmd/mxcli/lsp_widget_registry_test.go @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "os" + "path/filepath" + "testing" +) + +// A fresh clone: Fieldset's .mpk is installed in widgets/ and .mxcli/ — which +// is gitignored — does not exist. +// +// The LSP loaded its widget registry with LoadUserDefinitions alone, which reads +// only .mxcli/widgets/*.def.json, so completions offered the nine embedded +// widgets and the diagnostics built on the same registry knew nothing else. +// DESCRIBE WIDGET, `widget list` and check had the same defect +// (ako/mxcli#663, mendixlabs/mxcli#1135). +func TestWidgetRegistryCompletions_InstalledWidgetNeedsNoWidgetInit(t *testing.T) { + dir := t.TempDir() + if err := os.MkdirAll(filepath.Join(dir, "widgets"), 0o755); err != nil { + t.Fatal(err) + } + const mpk = "com.mendix.widget.web.Fieldset.mpk" + data, err := os.ReadFile(filepath.Join("..", "..", "testdata", "expr-checker", "widgets", mpk)) + if err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "widgets", mpk), data, 0o644); err != nil { + t.Fatal(err) + } + + s := &mdlServer{mprPath: filepath.Join(dir, "App.mpr")} + var labels []string + for _, item := range s.widgetRegistryCompletions() { + if item.Label == "fieldset" { + return + } + labels = append(labels, item.Label) + } + t.Fatalf("installed widget `fieldset` missing from completions; got %v", labels) +} + +// Control: with no project the embedded widgets are still offered. +func TestWidgetRegistryCompletions_NoProjectStillOffersEmbeddedWidgets(t *testing.T) { + s := &mdlServer{} + for _, item := range s.widgetRegistryCompletions() { + if item.Label == "combobox" { + return + } + } + t.Fatal("embedded widget `combobox` missing from completions with no project") +} From f38696f4c46b45061f237823d40a3fa6cb515edf Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 01:00:55 +0000 Subject: [PATCH 16/47] fix(describe): keep comment entries in a widget property list parseable A `--` comment entry runs to the end of its line. formatWidgetProps treated it as an ordinary property, so it swallowed the rest of the list and the `)` on the single-line form, and as the last entry it left the previous line's `,` dangling before `)`. Either way `exec` of the describe output failed with "extraneous input '}' expecting the start of a statement". Comment entries now force the multi-line form, sit on their own line, and never take or receive a separator. Co-Authored-By: Claude Opus 5.5 --- .../cmd_pages_describe_comment_props_test.go | 61 +++++++++++++++++++ mdl/executor/cmd_pages_describe_output.go | 39 ++++++++++-- 2 files changed, 96 insertions(+), 4 deletions(-) create mode 100644 mdl/executor/cmd_pages_describe_comment_props_test.go diff --git a/mdl/executor/cmd_pages_describe_comment_props_test.go b/mdl/executor/cmd_pages_describe_comment_props_test.go new file mode 100644 index 0000000000..1b3a582ae5 --- /dev/null +++ b/mdl/executor/cmd_pages_describe_comment_props_test.go @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// A `--` comment in a widget's property list runs to the end of its line, so +// it swallows whatever the formatter puts after it there: the next property on +// the single-line form, or the `,` separator — and, on the last property, the +// separator's absence leaves the previous line's `,` dangling before `)`. +// +// Measured on FeedbackModule.PopupSuccess (Feedback v4.0.2, Mendix 11.13.0): +// DESCRIBE emitted +// +// Action: -- open_link with a dynamic address (…) — MDL cannot author this; the button is left as-is, +// +// and exec of the output failed `extraneous input '}' expecting the start of a +// statement`. Five of 17 pages of that project failed to round-trip this way. +func TestFormatWidgetProps_CommentPropsStayParseable(t *testing.T) { + const comment = "-- DataSource (Forms$Something) has no MDL spelling and is not reproduced here" + tests := []struct { + name string + props []string + }{ + {"comment between properties", []string{"Caption: 'x'", comment, "ButtonStyle: Primary"}}, + {"comment last", []string{"Caption: 'x'", comment}}, + {"comment first", []string{comment, "Caption: 'x'"}}, + {"comment only", []string{comment}}, + // Short enough that the single-line form would otherwise be chosen. + {"short comment", []string{"Caption: 'x'", "-- note"}}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + var buf bytes.Buffer + formatWidgetProps(&buf, " ", "actionbutton b", tc.props, "\n") + src := "create page M.P (Title: 'x', Layout: A.L) {\n" + buf.String() + "}\n" + if _, errs := visitor.Build(src); len(errs) > 0 { + t.Fatalf("formatted widget does not parse: %v\n%s", errs, src) + } + // The comment is still there for a reader. + if !strings.Contains(buf.String(), strings.TrimPrefix(tc.props[indexOfComment(tc.props)], "-- ")) { + t.Errorf("comment text lost:\n%s", buf.String()) + } + }) + } +} + +func indexOfComment(props []string) int { + for i, p := range props { + if strings.HasPrefix(p, "--") { + return i + } + } + return 0 +} diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index c0c5c0b5e4..724f3241ea 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -225,16 +225,36 @@ func formatWidgetProps(w io.Writer, prefix string, header string, props []string fmt.Fprintf(w, "%s%s%s", prefix, header, suffix) return } - singleLine := fmt.Sprintf("%s%s (%s)%s", prefix, header, strings.Join(props, ", "), suffix) - if len(singleLine) <= 120 { - fmt.Fprint(w, singleLine) + // A `--` comment entry runs to the end of its line, so it must sit on a line + // of its own and never carry a separator: on the single-line form it + // swallowed the rest of the list and the `)`, and as the last entry it left + // the previous line's `,` dangling. Either way the output did not parse. + lastProp := -1 + for i, p := range props { + if !isCommentProp(p) { + lastProp = i + } + } + if lastProp == len(props)-1 { + singleLine := fmt.Sprintf("%s%s (%s)%s", prefix, header, strings.Join(props, ", "), suffix) + if len(singleLine) <= 120 && !containsCommentProp(props) { + fmt.Fprint(w, singleLine) + return + } + } + if lastProp < 0 { + // Only comments: there is no property list to write. + for _, p := range props { + fmt.Fprintf(w, "%s%s\n", prefix, p) + } + fmt.Fprintf(w, "%s%s%s", prefix, header, suffix) return } // Multi-line indent := prefix + " " fmt.Fprintf(w, "%s%s (\n", prefix, header) for i, p := range props { - if i < len(props)-1 { + if i < lastProp && !isCommentProp(p) { fmt.Fprintf(w, "%s%s,\n", indent, p) } else { fmt.Fprintf(w, "%s%s\n", indent, p) @@ -243,6 +263,17 @@ func formatWidgetProps(w io.Writer, prefix string, header string, props []string fmt.Fprintf(w, "%s)%s", prefix, suffix) } +func isCommentProp(p string) bool { return strings.HasPrefix(p, "--") } + +func containsCommentProp(props []string) bool { + for _, p := range props { + if isCommentProp(p) { + return true + } + } + return false +} + // outputDataContainerContext writes a comment showing available variables inside a data container. // isList indicates list containers (DataGrid2, ListView, Gallery) where a selection variable is available. func outputDataContainerContext(w io.Writer, prefix string, widgetName string, entityRef string, isList bool) { From 95eb07ea8816e14da82940808642cff0b3e5f6bf Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 01:10:18 +0000 Subject: [PATCH 17/47] feat(pages): author open_link with a dynamic address ($currentObject/Attr) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DESCRIBE PAGE on FeedbackModule.PopupSuccess (Feedback v4.0.2) emitted Action: -- open_link with a dynamic address (FeedbackModule.ResponseHelper.URL) — MDL cannot author this; the button is left as-is, which left `Action:` without a value, so exec of the output failed with "extraneous input '}'". The note was also wrong: CREATE OR REPLACE PAGE rebuilds the page, so an omitted Action writes a button with no action. I measured that: 0 OpenLinkClientAction left, and `mx check` clean. MDL now spells Studio Pro's "Address: attribute" as Action: open_link $currentObject/URL and writes the stored shape, taken from the Studio Pro-authored page: Forms$StaticOrDynamicString { IsDynamic: true, Value: "", AttributeRef { Attribute: Module.Entity.Attr, EntityRef: null } }. The address resolves against the enclosing data container, inherited attributes included. It is refused outside a data container, for any variable other than $currentObject, and over an association path. An association-path address still describes as a standalone NOT re-executable note, never an inline `Action: -- …`. Round trip of PopupSuccess and PopupSuccess_Logo: exec succeeds, the link action BSON is identical before and after, and `mx check` reports 0 errors. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../mendix/create-page/reference/widgets.md | 2 + cmd/mxcli/syntax/features_page.go | 2 +- .../bug-tests/open-link-dynamic-address.mdl | 46 ++++++++++ mdl/ast/ast_page_v3.go | 18 ++-- mdl/backend/modelsdk/widget_write.go | 18 +++- .../modelsdk/widget_write_signout_test.go | 40 +++++++++ mdl/executor/cmd_pages_builder_v3.go | 26 +++++- mdl/executor/cmd_pages_describe_output.go | 65 +++++++++----- .../cmd_pages_open_link_dynamic_test.go | 90 +++++++++++++++++++ mdl/executor/validate_widget_action_slot.go | 2 +- mdl/grammar/domains/MDLPage.g4 | 1 + .../visitor_page_open_link_dynamic_test.go | 42 +++++++++ mdl/visitor/visitor_page_v3.go | 10 ++- sdk/pages/pages_widgets_action.go | 3 + 15 files changed, 331 insertions(+), 35 deletions(-) create mode 100644 mdl-examples/bug-tests/open-link-dynamic-address.mdl create mode 100644 mdl/executor/cmd_pages_open_link_dynamic_test.go create mode 100644 mdl/visitor/visitor_page_open_link_dynamic_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index b095f0c388..e51d4d4e8e 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -691,3 +691,4 @@ {"area":"mdl/executor","date":"2026-09-24","symptom":"`mxcli check -p … --references` passes a view entity whose association-path join is written `join s/Mod.A_B/System.UserRole AS r` (uppercase AS) even when a pass-through column from `r` declares the wrong string length; mxbuild then fails with CE6770 \"View Entity is out of sync with the OQL Query.\" Lowercase `as` reports MDL031 correctly.","cause":"extractAliasMap matched the path join case-insensitively ((?i)…(?:as\\s+)?) but then recovered the path from match[0] with strings.TrimSuffix(path, \"as\") — case-sensitive — so with `AS` the path kept a trailing ` AS`, the end-anchored lastEntity regex failed, and the alias was never mapped. Every column from that alias silently went without type inference.","file":"mdl/executor/oql_type_inference.go","insight":"A (?i) regex followed by string surgery on the whole match reintroduces case sensitivity by the back door: capture every piece you need as its own group instead of trimming it back out. The tell is a single control table varying only the case of one keyword — every other keyword in upper case was harmless, which points straight at code that handles that one token outside the regex. An unresolved alias is silent (the checker skips unknown types rather than reporting), so the symptom is a check that PASSES; test at extractAliasMap directly, and control with the unfixed binary on a real project (it printed `Check passed!` for AS, the error for as). DESCRIBE prints AS in upper case, so round-tripped OQL hits this by default.","refs":["#652"],"ce":["CE6770"],"rules":["MDL031"]} {"area":"mdl/executor","date":"2026-09-24","symptom":"`alter page M.P { insert into dv { use fragment SaveCancelFooter } }` passed `check` and failed in exec: \"failed to insert: failed to build widgets: failed to build widget SaveCancelFooter: unsupported widget type: USE_FRAGMENT\". Same for REPLACE, and for a fragment nested inside an inserted container.","cause":"Fragment/building-block expansion (pageBuilder.expandFragments) was called only on the CREATE PAGE / snippet / layout paths. ALTER PAGE's applyInsertWidgetMutator / applyReplaceWidgetMutator passed op.Widgets straight to buildWidgetsFromAST, whose pageBuilder even carried ctx.Fragments — the registry was wired, the expansion call was not. Second defect found on the way: cloneWidget copied only Type/Name/Properties/Children, dropping Specialization and TypeIsGeneric, so a cloned `template for` lost its routing.","file":"`mdl/executor/cmd_alter_page.go` (expandAlterFragments, called first in the INSERT and REPLACE mutators), `mdl/executor/cmd_pages_builder_v3.go` (cloneWidget copies the whole struct)","insight":"Expand before ANYTHING inspects the widget list, not just before the build: the duplicate-name check, allColumns and allListViewTemplates all read op.Widgets, and seeing the sentinel they check the fragment's name instead of its widgets' names. When a registry is threaded into a builder, grep for the call that consumes it (`expandFragments`), not for the field — the field being set on every pageBuilder is what made the ALTER path look covered. A field-by-field clone is a silent-drop hazard; `c := *w` then deep-copy the reference fields.","refs":["#572"]} {"area":"mdl/executor","date":"2026-09-24","symptom":"`describe layout Atlas_Core.Phone_BottomBar` emitted `-- Forms$SimpleMenuBar (simpleMenuBar1) -- NOT re-executable: mxcli cannot author this widget, so re-running this script would drop it`, so a phone layout written in MDL had no bottom bar and a describe -> exec copy of Atlas's lost it","cause":"No keyword, builder, writer or describer for Forms$SimpleMenuBar, although modelsdk/gen already had SimpleMenuBar and MenuDocumentSource. And the widget's whole point is WHICH menu document it renders (a Forms$MenuDocumentSource in MenuSource), a source kind no menu widget could express — menubar/navigationtree always wrote a Forms$NavigationSource, so one pointed at a menu document described as `menubar m` and replayed onto the Responsive profile. Layouts were also never reference-checked, so a new `Menu:` typo would have passed --references and hit CE1613 at build","file":"mdl/grammar/MDLLexer.g4 + domains/MDLPage.g4 + domains/MDLSettings.g4 (SIMPLEMENUBAR), mdl/executor/cmd_pages_builder_v3.go (buildSimpleMenuBarV3, menuSourceV3), mdl/backend/modelsdk/widget_write.go (menuSourceToGen), mdl/executor/cmd_pages_describe_parse.go + cmd_pages_describe_output.go, mdl/executor/helpers.go + validate.go (menu refs, CreateLayoutStmt), sdk/pages/pages_widgets_advanced.go","insight":"**Measure the stored shape before designing the syntax** — dumping the widget from a blank project (`mxcli new --version 11.14.0`, `bson dump --type layout`) showed the source was a menu DOCUMENT, not a profile, which is what made this a source-kind feature rather than a one-keyword copy of `menubar`. Treat MenuSource as one slot with two subtypes on all three menu widgets (one helper each side), or the siblings keep silently rewriting a menu-document source to Responsive. The control that justified the reference check: `menu: Bug573.No_Such_Menu` -> `mxcli check --references` passed, `mx check` CE1613 'The selected menu ... no longer exists.' at Simple menu bar. MDL has no `show menus`, so the not-found message lists the menus that exist. Adding reference validation to CreateLayoutStmt is new coverage for EVERY widget reference in a layout, not just menus","refs":["ako/mxcli#573"],"ce":["CE1613"]} +{"area": "mdl/executor", "symptom": "`DESCRIBE PAGE FeedbackModule.PopupSuccess` → `exec` fails `Parse error: line 35:0 extraneous input '}' expecting the start of a statement`; describe emitted `Action: -- open_link with a dynamic address (FeedbackModule.ResponseHelper.URL) — MDL cannot author this; the button is left as-is,`", "cause": "Two layers. (1) formatWidgetProps treated a `--` note as an ordinary property: inline it swallowed the value slot, single-line it swallowed `)`, last-in-list it left a dangling `,`. (2) MDL had no spelling for a dynamic link address (Forms$StaticOrDynamicString IsDynamic + AttributeRef), so describe could only write a note — and the note's 'left as-is' was false: CREATE OR REPLACE PAGE rebuilds the page, so an omitted Action writes a button with no action, mx check clean", "file": "`mdl/executor/cmd_pages_describe_output.go` (`formatWidgetProps`, `actionProp`, `renderClientActionMDL`), `mdl/grammar/domains/MDLPage.g4` (actionExprV3 `OPEN_LINK VARIABLE SLASH attributePathV3`), `mdl/visitor/visitor_page_v3.go`, `mdl/executor/cmd_pages_builder_v3.go`, `mdl/backend/modelsdk/widget_write.go` (`dynamicAddressToGen`)", "insight": "**A describe-side note that promises preservation needs the write path to deliver it, and a full-replacement write cannot**: omitting the slot is indistinguishable from the author deleting it, so the only honest options are authoring the value or saying re-running drops it. Measured the omit design before rejecting it: mx check 0 errors, 0 OpenLinkClientAction left — silent loss. Authoring was cheap because the stored shape was fully visible in one `bson dump --format ndsl` of a Studio Pro page (IsDynamic true, Value \"\", AttributeRef with null EntityRef) and `attributeRefToGen` already existed. The action grammar's `$handler` VARIABLE alternative is checked first in buildActionV3 — any new action form carrying a VARIABLE must exclude itself there. **The reported page list was wrong for 3 of 5**: same parse error text, different emitter (a nameless `statictext`) — check the first error line of each failing page, not the last, before assuming one cause", "refs": ["FeedbackModule.PopupSuccess", "Administration.Account_Edit"], "date": "2026-09-25"} diff --git a/.claude/skills/mendix/create-page/reference/widgets.md b/.claude/skills/mendix/create-page/reference/widgets.md index c0d3911e38..1f364209bd 100644 --- a/.claude/skills/mendix/create-page/reference/widgets.md +++ b/.claude/skills/mendix/create-page/reference/widgets.md @@ -114,6 +114,8 @@ describe icon collection Atlas_Core.Atlas_Filled -- every icon + its reference - `action: nanoflow Module.NanoflowName` - Call nanoflow (client-side) - `action: nanoflow Module.NanoflowName(Param: $value)` - Call nanoflow with parameters - `action: nanoflow Module.NanoflowName($Param = $value)` - Also accepted (microflow-style) +- `action: open_link 'https://example.com'` - Open a fixed web address +- `action: open_link $currentObject/URL` - Open the address held in an attribute of the enclosing data container's object (inside a data container only; not over an association) - **Every parameter needs an argument, or an enclosing data container of its type.** A flow called with a parameter nothing fills is **CE1571**; `mxcli check -p` reports it. This is the same on every widget that takes an action, diff --git a/cmd/mxcli/syntax/features_page.go b/cmd/mxcli/syntax/features_page.go index 6094acbdaa..ecd42e9915 100644 --- a/cmd/mxcli/syntax/features_page.go +++ b/cmd/mxcli/syntax/features_page.go @@ -259,7 +259,7 @@ CREATE PAGE Sales.Detail (Title: 'Detail', Layout: Atlas_Core.Atlas_Default) { "icon", "linkbutton", "link button", "nothing", "no action", "inert", "dead button", }, - Syntax: "Action: NOTHING -- deliberately no action (Forms$NoAction)\nAction: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: NANOFLOW Module.NF(Param: $val)\nAction: OPEN_LINK 'https://example.com'\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $currentObject)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nThe list above is exhaustive. Anything else in an action slot is an\nERROR (MDL-WIDGET28), including a real keyword short its argument --\n`Action: OPEN_LINK` without a URL, `Action: SHOW_PAGE` without a page.\nSuch a widget used to be written with NO action at all and rendered as a\ndead control, with check, exec and mxbuild all clean, because a\nno-action widget is legal Mendix (mendixlabs/mxcli#1062). Write NOTHING\nwhen a control really is meant to be inert.\n\nThe same forms serve `OnClick:` (an alias of `Action:`) and `OnChange:`.\n\nA microflow or nanoflow action is a CALL: it needs an argument for every\nparameter the flow declares, or Mendix rejects the page with CE1571. The\nargument list is the same on every widget that takes an action -- a\nCONTAINER (which is clickable) as much as an ACTIONBUTTON. An enclosing\ndata container of the right type supplies it without an argument; a data\ngrid's CONTROL BAR does not, because it is not row-scoped -- pass the\ngrid's selection there (`$dgOrders`).\n\nA SHOW_PAGE argument must be the enclosing widget's context object --\neither $currentObject or the name of the variable the enclosing data\nwidget is bound to. Mendix infers it from that widget, so naming any\nother variable is refused (MDL-PAGEARG01); call a microflow instead.\nOutside any data widget there is no context object to infer, so such a\nbutton takes NO argument at all -- not a page parameter, not\n$currentObject, not a literal. mxcli used to drop it in silence and\nmxbuild then reported CE1571 per parameter of the target page\n(mendixlabs/mxcli#1029). Route that navigation through a microflow.\n\nOPEN_LINK takes a static web address and stores it as a\nForms$StaticOrDynamicString. Mendix also supports a DYNAMIC address, read\nfrom an attribute at runtime; MDL cannot author that one, and DESCRIBE\nflags such a button rather than printing its address as a literal.\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\n\nMendix has THREE icon elements and the keyword picks which one:\n\nIcon: 'Atlas_Core.Atlas_Filled.pencil' -- an icon collection\nIcon: image MyModule.Images.logo -- an IMAGE collection\nIcon: glyph 57377 -- a font code point\n\nThe bare form is the icon-collection icon and any collection in the\nproject works, third-party ones included. The image form points into a\ndifferent document, and is spelled the same way apart from the keyword\n-- write it without `image` and mxcli stores a custom-icon reference,\nwhich fails the build with CE1613 (mendixlabs/mxcli#1059).\n`mxcli check -p … --references` resolves each kind against its own\ncollection and names the remedy when the kind is wrong.\n\nA glyph carries a code and no name. Codes are sparse, and an undefined\none fails only at `mxbuild --target=deploy`, naming the PAGE rather\nthan the icon -- so MDL078 checks it against the font's own table.\nList them with `show glyphs`.\n\nA name may be quoted or bare; a hyphenated segment is double-quoted on\nits own: Atlas_Core.Atlas.\"align-center\".\n\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", + Syntax: "Action: NOTHING -- deliberately no action (Forms$NoAction)\nAction: SAVE_CHANGES\nAction: SAVE_CHANGES CLOSE_PAGE -- save, then close the pop-up\nAction: CANCEL_CHANGES\nAction: CANCEL_CHANGES CLOSE_PAGE\nAction: CLOSE_PAGE\nAction: DELETE\nAction: DELETE CLOSE_PAGE\nAction: DELETE_OBJECT\nAction: NANOFLOW Module.NF\nAction: NANOFLOW Module.NF(Param: $val)\nAction: OPEN_LINK 'https://example.com'\nAction: OPEN_LINK $currentObject/URL -- address read from an attribute\nAction: SIGN_OUT\nAction: COMPLETE_TASK 'OutcomeName'\nAction: SHOW_PAGE Module.Page\nAction: SHOW_PAGE Module.Page(Param: $currentObject)\nAction: MICROFLOW Module.MF\nAction: MICROFLOW Module.MF(Param: $val)\nAction: CREATE_OBJECT Module.Entity THEN SHOW_PAGE Module.Page\n\nThe list above is exhaustive. Anything else in an action slot is an\nERROR (MDL-WIDGET28), including a real keyword short its argument --\n`Action: OPEN_LINK` without a URL, `Action: SHOW_PAGE` without a page.\nSuch a widget used to be written with NO action at all and rendered as a\ndead control, with check, exec and mxbuild all clean, because a\nno-action widget is legal Mendix (mendixlabs/mxcli#1062). Write NOTHING\nwhen a control really is meant to be inert.\n\nThe same forms serve `OnClick:` (an alias of `Action:`) and `OnChange:`.\n\nA microflow or nanoflow action is a CALL: it needs an argument for every\nparameter the flow declares, or Mendix rejects the page with CE1571. The\nargument list is the same on every widget that takes an action -- a\nCONTAINER (which is clickable) as much as an ACTIONBUTTON. An enclosing\ndata container of the right type supplies it without an argument; a data\ngrid's CONTROL BAR does not, because it is not row-scoped -- pass the\ngrid's selection there (`$dgOrders`).\n\nA SHOW_PAGE argument must be the enclosing widget's context object --\neither $currentObject or the name of the variable the enclosing data\nwidget is bound to. Mendix infers it from that widget, so naming any\nother variable is refused (MDL-PAGEARG01); call a microflow instead.\nOutside any data widget there is no context object to infer, so such a\nbutton takes NO argument at all -- not a page parameter, not\n$currentObject, not a literal. mxcli used to drop it in silence and\nmxbuild then reported CE1571 per parameter of the target page\n(mendixlabs/mxcli#1029). Route that navigation through a microflow.\n\nOPEN_LINK takes a web address, stored as a Forms$StaticOrDynamicString:\neither a literal, or $currentObject/Attr to read it from an attribute of\nthe enclosing data container's object at runtime (Studio Pro's \"Address:\nattribute\"). An address over an association path is not supported yet.\n\nButton styles: Default, Primary, Success, Info, Warning, Danger\n\nMendix has THREE icon elements and the keyword picks which one:\n\nIcon: 'Atlas_Core.Atlas_Filled.pencil' -- an icon collection\nIcon: image MyModule.Images.logo -- an IMAGE collection\nIcon: glyph 57377 -- a font code point\n\nThe bare form is the icon-collection icon and any collection in the\nproject works, third-party ones included. The image form points into a\ndifferent document, and is spelled the same way apart from the keyword\n-- write it without `image` and mxcli stores a custom-icon reference,\nwhich fails the build with CE1613 (mendixlabs/mxcli#1059).\n`mxcli check -p … --references` resolves each kind against its own\ncollection and names the remedy when the kind is wrong.\n\nA glyph carries a code and no name. Codes are sparse, and an undefined\none fails only at `mxbuild --target=deploy`, naming the PAGE rather\nthan the icon -- so MDL078 checks it against the font's own table.\nList them with `show glyphs`.\n\nA name may be quoted or bare; a hyphenated segment is double-quoted on\nits own: Atlas_Core.Atlas.\"align-center\".\n\nUse `linkbutton` instead of `actionbutton` for link render mode (same properties).", Example: "ACTIONBUTTON btnSave (Caption: 'Save', Action: SAVE_CHANGES, ButtonStyle: Primary)\nACTIONBUTTON btnEdit (Caption: 'Edit',\n Action: SHOW_PAGE Module.EditPage(Item: $currentObject))\nLINKBUTTON btnDelete (Caption: 'Delete', Action: DELETE,\n Icon: 'Atlas_Core.Atlas_Filled.pencil')\n\n-- A clickable CONTAINER in a data grid's control bar, calling a nanoflow\n-- with the grid's selection as its argument.\nDATAGRID dgOrders (DataSource: DATABASE FROM Sales.Order, Selection: Single) {\n COLUMN colNr (Attribute: Number, Caption: 'Order #')\n CONTROLBAR cb {\n CONTAINER cShip (Class: 'command',\n Action: NANOFLOW Sales.ACT_Ship($Order = $dgOrders)) {\n ACTIONBUTTON btnShip (Caption: 'Ship')\n }\n }\n}", SeeAlso: []string{"page.widgets"}, }) diff --git a/mdl-examples/bug-tests/open-link-dynamic-address.mdl b/mdl-examples/bug-tests/open-link-dynamic-address.mdl new file mode 100644 index 0000000000..e55a97b392 --- /dev/null +++ b/mdl-examples/bug-tests/open-link-dynamic-address.mdl @@ -0,0 +1,46 @@ +-- ============================================================================ +-- open_link with a DYNAMIC address (read from an attribute at runtime) +-- ============================================================================ +-- +-- Symptom: DESCRIBE PAGE on FeedbackModule.PopupSuccess (Feedback v4.0.2, +-- Mendix 11.13.0) emitted +-- +-- Action: -- open_link with a dynamic address (FeedbackModule.ResponseHelper.URL) — MDL cannot author this; the button is left as-is, +-- +-- The comment left `Action:` without a value and swallowed the separator, so +-- exec of the describe output failed: +-- Parse error: line 35:0 extraneous input '}' expecting the start of a statement +-- And "left as-is" was never true: CREATE OR REPLACE PAGE rebuilds the page, +-- so dropping the Action line wrote a button with no action at all — measured, +-- with `mx check` clean. +-- +-- Fixes: +-- 1) `open_link $currentObject/Attr` authors the dynamic address (Studio Pro's +-- "Address: attribute"), and DESCRIBE emits it, so the round trip keeps +-- the action byte-for-byte. +-- 2) A `--` note in a widget's property list is placed on its own line, so +-- what describe still cannot spell (e.g. an address over an association) +-- no longer breaks parsing. +-- +-- Verify: exec, then `mxcli docker check` — 0 errors; `describe page` shows +-- `Action: open_link $currentObject/URL`. +-- ============================================================================ + +create entity MyFirstModule.LinkTarget ( + Title: String(200), + URL: String(2000) +); +/ + +create or replace page MyFirstModule.LinkTarget_View +( Title: 'Link', Layout: Atlas_Core.Atlas_Default, Params: { $LinkTarget: MyFirstModule.LinkTarget } ) +{ + dataview dv (datasource: $LinkTarget) { + actionbutton btnStatic (Caption: 'Docs', Action: open_link 'https://docs.mendix.com') + actionbutton btnDynamic (Caption: 'Open', Action: open_link $currentObject/URL, ButtonStyle: Primary) + linkbutton lnkDynamic (Caption: 'Open as link', Action: open_link $currentObject/URL) + } +} +/ + +describe page MyFirstModule.LinkTarget_View; diff --git a/mdl/ast/ast_page_v3.go b/mdl/ast/ast_page_v3.go index b8abe6c07d..23c176a5e1 100644 --- a/mdl/ast/ast_page_v3.go +++ b/mdl/ast/ast_page_v3.go @@ -218,13 +218,17 @@ type OrderByItemV3 struct { // ActionV3 represents a V3 action expression. type ActionV3 struct { - Type string // "save", "cancel", "close", "delete", "create", "showPage", "microflow", "nanoflow", "openLink", "signOut", "completeTask" - Target string // Entity, page, or flow qualified name (for create/showPage/microflow/nanoflow) - Args []FlowArgV3 // Arguments for showPage/microflow calls - ThenAction *ActionV3 // For CREATE_OBJECT ... THEN ... - ClosePage bool // For SAVE_CHANGES CLOSE_PAGE - LinkURL string // For OPEN_LINK - OutcomeValue string // For COMPLETE_TASK + Type string // "save", "cancel", "close", "delete", "create", "showPage", "microflow", "nanoflow", "openLink", "signOut", "completeTask" + Target string // Entity, page, or flow qualified name (for create/showPage/microflow/nanoflow) + Args []FlowArgV3 // Arguments for showPage/microflow calls + ThenAction *ActionV3 // For CREATE_OBJECT ... THEN ... + ClosePage bool // For SAVE_CHANGES CLOSE_PAGE + LinkURL string // For OPEN_LINK 'https://…' (static address) + // For OPEN_LINK $currentObject/Attr: the address is read from an attribute + // at runtime (a dynamic Forms$StaticOrDynamicString). + LinkVariable string + LinkAttribute string + OutcomeValue string // For COMPLETE_TASK } // ColumnV3 represents a V3 datagrid column. diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index bd297391e3..bf122ad213 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -1689,6 +1689,18 @@ func formSettingsToGen(pageName string) element.Element { // `Attribute`, that not one of the 31 documents carries; writing a key Mendix // does not store is what makes a document mxbuild accepts and Studio Pro cannot // open (CLAUDE.md, "Overlay Writes: Never Invent a Key"). +// dynamicAddressToGen builds a link address read from an attribute at runtime. +// Pinned against FeedbackModule.PopupSuccess (Feedback v4.0.2): IsDynamic true, +// Value "", and an AttributeRef with a null EntityRef naming the attribute. +func dynamicAddressToGen(attrQN string) element.Element { + s := genPg.NewStaticOrDynamicString() + assignID(s) + s.SetIsDynamic(true) + s.SetValue("") + s.SetAttributeRef(attributeRefToGen(attrQN)) + return s +} + func staticAddressToGen(address string) element.Element { s := genPg.NewStaticOrDynamicString() assignID(s) @@ -1774,7 +1786,11 @@ func clientActionToGen(a pages.ClientAction) (element.Element, error) { linkType = "Web" } g.SetLinkType(linkType) - g.SetAddress(staticAddressToGen(x.Address)) + if x.AddressAttribute != "" { + g.SetAddress(dynamicAddressToGen(x.AddressAttribute)) + } else { + g.SetAddress(staticAddressToGen(x.Address)) + } return g, nil case *pages.SignOutClientAction: // sign_out → Forms$SignOutClientAction. One property, and the reference diff --git a/mdl/backend/modelsdk/widget_write_signout_test.go b/mdl/backend/modelsdk/widget_write_signout_test.go index 46052fa966..7571b480b7 100644 --- a/mdl/backend/modelsdk/widget_write_signout_test.go +++ b/mdl/backend/modelsdk/widget_write_signout_test.go @@ -7,6 +7,7 @@ import ( "testing" "github.com/mendixlabs/mxcli/model" + genDm "github.com/mendixlabs/mxcli/modelsdk/gen/domainmodels" genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" "github.com/mendixlabs/mxcli/sdk/pages" ) @@ -143,3 +144,42 @@ func TestClientActionToGen_ExistingActionsUnchanged(t *testing.T) { } } } + +// A dynamic address — `open_link $currentObject/URL` — is the same five-key +// action with the StaticOrDynamicString flipped: IsDynamic true, an empty +// Value, and an AttributeRef naming the attribute. Pinned against +// FeedbackModule.PopupSuccess (Feedback v4.0.2, Studio Pro-authored): +// +// Address: Forms$StaticOrDynamicString +// AttributeRef: DomainModels$AttributeRef +// Attribute: "FeedbackModule.ResponseHelper.URL" +// EntityRef: null +// IsDynamic: true +// Value: "" +func TestClientActionToGen_OpenLinkDynamicAddress(t *testing.T) { + el, err := clientActionToGen(&pages.LinkClientAction{ + BaseElement: model.BaseElement{ID: "link-id"}, + LinkType: pages.LinkTypeWeb, + AddressAttribute: "FeedbackModule.ResponseHelper.URL", + }) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + addr, ok := el.(*genPg.OpenLinkClientAction).Address().(*genPg.StaticOrDynamicString) + if !ok { + t.Fatalf("Address is not a StaticOrDynamicString") + } + if !addr.IsDynamic() { + t.Error("IsDynamic is false — the runtime would open the empty static Value") + } + if addr.Value() != "" { + t.Errorf("Value = %q, want empty", addr.Value()) + } + ref, ok := addr.AttributeRef().(*genDm.AttributeRef) + if !ok { + t.Fatalf("AttributeRef is %T, want *domainmodels.AttributeRef", addr.AttributeRef()) + } + if ref.AttributeQualifiedName() != "FeedbackModule.ResponseHelper.URL" { + t.Errorf("AttributeRef.Attribute = %q", ref.AttributeQualifiedName()) + } +} diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index b93b31a958..19c965805d 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -1647,6 +1647,27 @@ func (pb *pageBuilder) buildClientActionV3(action *ast.ActionV3) (pages.ClientAc return nfAction, nil case "openLink": + addressAttr := "" + if action.LinkAttribute != "" { + // A dynamic address, read from the context object at runtime — + // the only variable Studio Pro's "Address: attribute" choice binds. + if !strings.EqualFold(action.LinkVariable, "$currentObject") { + return nil, mdlerrors.NewValidationf( + "open_link %s/%s: a dynamic link address is read from $currentObject — write `open_link $currentObject/%s` inside the data container that holds it", + action.LinkVariable, action.LinkAttribute, action.LinkAttribute) + } + if strings.Contains(action.LinkAttribute, "/") { + return nil, mdlerrors.NewValidationf( + "open_link $currentObject/%s: an address over an association path is not supported yet — bind an attribute of the data container's own entity", + action.LinkAttribute) + } + if pb.entityContext == "" { + return nil, mdlerrors.NewValidationf( + "open_link $currentObject/%s: a dynamic link address needs an object to read it from — place the button inside a data container", + action.LinkAttribute) + } + addressAttr = pb.resolveAttributePath(action.LinkAttribute) + } return &pages.LinkClientAction{ BaseElement: model.BaseElement{ ID: model.ID(types.GenerateID()), @@ -1656,8 +1677,9 @@ func (pb *pageBuilder) buildClientActionV3(action *ast.ActionV3) (pages.ClientAc // only because neither engine could write the action at all. TypeName: "Forms$OpenLinkClientAction", }, - LinkType: pages.LinkTypeWeb, - Address: action.LinkURL, + LinkType: pages.LinkTypeWeb, + Address: action.LinkURL, + AddressAttribute: addressAttr, }, nil case "signOut": diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index 724f3241ea..414670d6a3 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -392,7 +392,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { header := fmt.Sprintf("container %s", mdlIdent(w.Name)) props := appendAppearanceProps(nil, w) if w.Action != "" { - props = append(props, fmt.Sprintf("Action: %s", w.Action)) + props = append(props, actionProp("Action", w.Action)) } if len(w.Children) > 0 { formatWidgetProps(ctx.Output, prefix, header, props, " {\n") @@ -496,7 +496,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = append(props, fmt.Sprintf("CaptionParams: [%s]", strings.Join(formatParametersV3(w.Parameters), ", "))) } if w.Action != "" { - props = append(props, fmt.Sprintf("Action: %s", w.Action)) + props = append(props, actionProp("Action", w.Action)) } if w.ButtonStyle != "" && w.ButtonStyle != "Default" { props = append(props, fmt.Sprintf("ButtonStyle: %s", w.ButtonStyle)) @@ -570,7 +570,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = append(props, fmt.Sprintf("Placeholder: %s", mdlQuote(w.Placeholder))) } if w.OnChange != "" { - props = append(props, fmt.Sprintf("OnChange: %s", w.OnChange)) + props = append(props, actionProp("OnChange", w.OnChange)) } props = appendInputValidationProps(props, w) props = appendAppearanceProps(props, w) @@ -586,7 +586,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = append(props, fmt.Sprintf("Attribute: %s", w.Content)) } if w.OnChange != "" { - props = append(props, fmt.Sprintf("OnChange: %s", w.OnChange)) + props = append(props, actionProp("OnChange", w.OnChange)) } props = appendAppearanceProps(props, w) formatWidgetProps(ctx.Output, prefix, header, props, "\n") @@ -601,7 +601,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = append(props, fmt.Sprintf("Attribute: %s", w.Content)) } if w.OnChange != "" { - props = append(props, fmt.Sprintf("OnChange: %s", w.OnChange)) + props = append(props, actionProp("OnChange", w.OnChange)) } props = appendAppearanceProps(props, w) formatWidgetProps(ctx.Output, prefix, header, props, "\n") @@ -616,7 +616,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = append(props, fmt.Sprintf("Attribute: %s", w.Content)) } if w.OnChange != "" { - props = append(props, fmt.Sprintf("OnChange: %s", w.OnChange)) + props = append(props, actionProp("OnChange", w.OnChange)) } props = appendAppearanceProps(props, w) formatWidgetProps(ctx.Output, prefix, header, props, "\n") @@ -639,7 +639,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = append(props, "ShowLabel: No") } if w.OnChange != "" { - props = append(props, fmt.Sprintf("OnChange: %s", w.OnChange)) + props = append(props, actionProp("OnChange", w.OnChange)) } props = appendAppearanceProps(props, w) formatWidgetProps(ctx.Output, prefix, header, props, "\n") @@ -660,7 +660,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { } // onClick action (ledger #67) if w.OnClick != "" { - props = append(props, fmt.Sprintf("onClick: %s", w.OnClick)) + props = append(props, actionProp("onClick", w.OnClick)) } props = appendNamedActionProps(props, w) // Add paging properties if non-default @@ -778,12 +778,12 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { } // onClick action (ledger #67 — reported on CustomChart) if w.OnClick != "" { - props = append(props, fmt.Sprintf("onClick: %s", w.OnClick)) + props = append(props, actionProp("onClick", w.OnClick)) } // OnChange too — a Slider/RangeSlider/StarRating reaches describe // through this branch, and its action slot is the only one it has. if w.OnChange != "" { - props = append(props, fmt.Sprintf("OnChange: %s", w.OnChange)) + props = append(props, actionProp("OnChange", w.OnChange)) } props = appendNamedActionProps(props, w) props = appendAppearanceProps(props, w) @@ -861,7 +861,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { // Emitted for the same reason as the built-in inputs above: without // it a describe→edit→exec cycle silently drops the action. if w.OnChange != "" { - props = append(props, fmt.Sprintf("OnChange: %s", w.OnChange)) + props = append(props, actionProp("OnChange", w.OnChange)) } // Show filter attributes for filter widgets if len(w.FilterAttributes) > 0 { @@ -881,7 +881,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { itemHeader := fmt.Sprintf("item %s", mdlIdent(child.Name)) props := []string{} if child.Action != "" { - props = append(props, fmt.Sprintf("Action: %s", child.Action)) + props = append(props, actionProp("Action", child.Action)) } if child.ButtonStyle != "" && child.ButtonStyle != "Default" { props = append(props, fmt.Sprintf("ButtonStyle: %s", child.ButtonStyle)) @@ -944,7 +944,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = append(props, "Responsive: false") } if w.Action != "" { - props = append(props, fmt.Sprintf("Action: %s", w.Action)) + props = append(props, actionProp("Action", w.Action)) } props = appendAppearanceProps(props, w) formatWidgetProps(ctx.Output, prefix, header, props, "\n") @@ -985,7 +985,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = append(props, "OnClickType: enlarge") } if w.Action != "" { - props = append(props, fmt.Sprintf("Action: %s", w.Action)) + props = append(props, actionProp("Action", w.Action)) } props = appendAppearanceProps(props, w) formatWidgetProps(ctx.Output, prefix, header, props, "\n") @@ -1029,7 +1029,7 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { // the action is dropped on the next describe -> exec, which is the // half-shell trap: valid BSON, clean build, construct silently gone. if w.Action != "" { - props = append(props, fmt.Sprintf("Action: %s", w.Action)) + props = append(props, actionProp("Action", w.Action)) } props = appendAppearanceProps(props, w) if len(w.Children) > 0 { @@ -1441,21 +1441,30 @@ func renderClientActionMDL(ctx *ExecContext, action map[string]any) string { case "Forms$SignOutClientAction", "Pages$SignOutClientAction": return "sign_out" case "Forms$OpenLinkClientAction", "Pages$OpenLinkClientAction": - // The address is a nested Forms$StaticOrDynamicString. MDL can spell the - // static form only; a DYNAMIC address (6 of the 31 Studio Pro references - // use one) reads its value from an attribute at runtime, so rendering it - // as a literal would round-trip into a different link. Say so instead. + // The address is a nested Forms$StaticOrDynamicString: a literal, or — + // DYNAMIC, 6 of the 31 Studio Pro references — an attribute read at + // runtime, spelled `open_link $currentObject/Attr`. It used to render + // as an inline `--` note, which left `Action:` without a value and made + // the describe output unparseable. addr := actionMapForKey(action, "Address") if addr == nil { return "open_link ''" } if isDynamic, _ := addr["IsDynamic"].(bool); isDynamic { attr := "" + overAssociation := false if ref := actionMapForKey(addr, "AttributeRef"); ref != nil { attr, _ = ref["Attribute"].(string) + overAssociation = actionMapForKey(ref, "EntityRef") != nil } - return "-- open_link with a dynamic address (" + attr + ") — MDL cannot author this; " + - "the button is left as-is" + if attr != "" && !overAssociation { + return "open_link $currentObject/" + shortAttributeName(attr) + } + // No MDL spelling: a note, which actionProp puts on its own line. + // CREATE OR REPLACE PAGE rebuilds the page, so say plainly that + // re-running drops the action rather than implying it survives. + return "-- NOT re-executable: open_link with a dynamic address over an association (" + + attr + ") — re-running this script would drop the button's action" } value, _ := addr["Value"].(string) return "open_link '" + strings.ReplaceAll(value, "'", "''") + "'" @@ -2005,7 +2014,19 @@ func describeImageWidgetProps(w rawWidget) []string { props = append(props, "OnClickType: enlarge") } if w.Action != "" { - props = append(props, fmt.Sprintf("OnClick: %s", w.Action)) + props = append(props, actionProp("OnClick", w.Action)) } return props } + +// actionProp renders an action slot as `Key: `, or — when the action +// renderer returned a `--` note because MDL cannot spell the action — as the +// bare note, which formatWidgetProps places on its own line. Written inline, +// `Action: -- …` left the slot without a value and swallowed the separator +// after it, so the describe output did not parse. +func actionProp(key, rendered string) string { + if isCommentProp(rendered) { + return rendered + } + return key + ": " + rendered +} diff --git a/mdl/executor/cmd_pages_open_link_dynamic_test.go b/mdl/executor/cmd_pages_open_link_dynamic_test.go new file mode 100644 index 0000000000..95a887b30c --- /dev/null +++ b/mdl/executor/cmd_pages_open_link_dynamic_test.go @@ -0,0 +1,90 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "context" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// `open_link $currentObject/Attr` — an address read from an attribute — builds +// a LinkClientAction bound to that attribute, qualified against the enclosing +// data container's entity. +func TestBuildClientActionV3_OpenLinkDynamicAddress(t *testing.T) { + pb := &pageBuilder{entityContext: "FeedbackModule.ResponseHelper"} + got, err := pb.buildClientActionV3(&ast.ActionV3{Type: "openLink", LinkVariable: "$currentObject", LinkAttribute: "URL"}) + if err != nil { + t.Fatalf("unexpected error: %v", err) + } + link, ok := got.(*pages.LinkClientAction) + if !ok { + t.Fatalf("got %T, want *pages.LinkClientAction", got) + } + if link.AddressAttribute != "FeedbackModule.ResponseHelper.URL" { + t.Errorf("AddressAttribute = %q, want FeedbackModule.ResponseHelper.URL", link.AddressAttribute) + } + if link.Address != "" { + t.Errorf("Address = %q, want empty", link.Address) + } +} + +// Outside a data container there is no object to read the address from, and +// only $currentObject is supported — refuse rather than write a dangling ref. +func TestBuildClientActionV3_OpenLinkDynamicAddressRefusals(t *testing.T) { + for _, tc := range []struct { + name string + pb *pageBuilder + act *ast.ActionV3 + want string + }{ + {"no entity context", &pageBuilder{}, &ast.ActionV3{Type: "openLink", LinkVariable: "$currentObject", LinkAttribute: "URL"}, "data container"}, + {"other variable", &pageBuilder{entityContext: "M.E"}, &ast.ActionV3{Type: "openLink", LinkVariable: "$Param", LinkAttribute: "URL"}, "$currentObject"}, + {"association path", &pageBuilder{entityContext: "M.E"}, &ast.ActionV3{Type: "openLink", LinkVariable: "$currentObject", LinkAttribute: "E_Other/URL"}, "association"}, + } { + t.Run(tc.name, func(t *testing.T) { + _, err := tc.pb.buildClientActionV3(tc.act) + if err == nil || !strings.Contains(err.Error(), tc.want) { + t.Errorf("err = %v, want it to mention %q", err, tc.want) + } + }) + } +} + +// DESCRIBE renders the stored dynamic address as the syntax above instead of +// an inline `--` note, which left `Action:` without a value and made the +// output unparseable (5 of 17 pages of a stock Feedback-module project). +func TestRenderClientActionMDL_OpenLinkDynamicAddress(t *testing.T) { + ctx := (&Executor{}).newExecContext(context.Background()) + action := map[string]any{ + "$Type": "Forms$OpenLinkClientAction", + "Address": map[string]any{ + "$Type": "Forms$StaticOrDynamicString", + "IsDynamic": true, + "Value": "", + "AttributeRef": map[string]any{ + "$Type": "DomainModels$AttributeRef", + "Attribute": "FeedbackModule.ResponseHelper.URL", + "EntityRef": nil, + }, + }, + "LinkType": "Web", + } + if got, want := renderClientActionMDL(ctx, action), "open_link $currentObject/URL"; got != want { + t.Errorf("renderClientActionMDL = %q, want %q", got, want) + } + + // Over an association MDL still has no spelling: a standalone note, + // never an inline `Action: -- …`. + action["Address"].(map[string]any)["AttributeRef"].(map[string]any)["EntityRef"] = map[string]any{ + "$Type": "DomainModels$IndirectEntityRef", + "Steps": []any{int32(2), map[string]any{"Association": "M.E_Other", "DestinationEntity": "M.Other"}}, + } + got := renderClientActionMDL(ctx, action) + if !strings.HasPrefix(got, "-- ") || actionProp("Action", got) != got { + t.Errorf("association-path address rendered %q, want a standalone -- note", got) + } +} diff --git a/mdl/executor/validate_widget_action_slot.go b/mdl/executor/validate_widget_action_slot.go index f4c2a581a9..f20166172b 100644 --- a/mdl/executor/validate_widget_action_slot.go +++ b/mdl/executor/validate_widget_action_slot.go @@ -55,7 +55,7 @@ var actionSlotKeys = []string{"Action", "OnClick", "OnChange"} // can name the one token the author left out rather than printing the whole // grammar. Keyed lowercase; looked up case-insensitively. var underSpecified = map[string]string{ - "open_link": "a URL — `Action: OPEN_LINK 'https://example.com'`", + "open_link": "a URL — `Action: OPEN_LINK 'https://example.com'`, or an attribute holding one — `Action: OPEN_LINK $currentObject/URL`", "complete_task": "an outcome name — `Action: COMPLETE_TASK 'Approved'`", "show_page": "a page — `Action: SHOW_PAGE Module.Page`", "create_object": "an entity — `Action: CREATE_OBJECT Module.Entity`", diff --git a/mdl/grammar/domains/MDLPage.g4 b/mdl/grammar/domains/MDLPage.g4 index d2d6190e00..d34e3d3e88 100644 --- a/mdl/grammar/domains/MDLPage.g4 +++ b/mdl/grammar/domains/MDLPage.g4 @@ -642,6 +642,7 @@ actionExprV3 | MICROFLOW qualifiedName microflowArgsV3? // MICROFLOW Module.Flow | NANOFLOW qualifiedName microflowArgsV3? // NANOFLOW Module.Flow | OPEN_LINK STRING_LITERAL // OPEN_LINK 'https://...' + | OPEN_LINK VARIABLE SLASH attributePathV3 // OPEN_LINK $currentObject/URL (address read from an attribute) | SIGN_OUT // SIGN_OUT | COMPLETE_TASK STRING_LITERAL // COMPLETE_TASK 'OutcomeName' ; diff --git a/mdl/visitor/visitor_page_open_link_dynamic_test.go b/mdl/visitor/visitor_page_open_link_dynamic_test.go new file mode 100644 index 0000000000..51b6241464 --- /dev/null +++ b/mdl/visitor/visitor_page_open_link_dynamic_test.go @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// A link button can read its address from an attribute at runtime — Studio +// Pro's "Address: attribute" choice, stored as a Forms$StaticOrDynamicString +// with IsDynamic true. MDL could only spell the static form, so DESCRIBE had +// nothing to emit and a describe → exec cycle could not keep the action. +// Measured: FeedbackModule.PopupSuccess (Feedback v4.0.2) binds +// FeedbackModule.ResponseHelper.URL this way. +func TestAction_OpenLinkDynamicAddress(t *testing.T) { + src := "create page M.P (Title: 'x', Layout: A.L) { actionbutton b (Action: open_link $currentObject/URL) };" + raw := actionSlotValue(t, src, "Action") + action, ok := raw.(*ast.ActionV3) + if !ok { + t.Fatalf("Action = %T (%v), want *ast.ActionV3", raw, raw) + } + if action.Type != "openLink" { + t.Errorf("Type = %q, want openLink", action.Type) + } + if action.LinkAttribute != "URL" || action.LinkVariable != "$currentObject" { + t.Errorf("LinkVariable/LinkAttribute = %q/%q, want $currentObject/URL", action.LinkVariable, action.LinkAttribute) + } + if action.LinkURL != "" { + t.Errorf("LinkURL = %q, want empty for a dynamic address", action.LinkURL) + } +} + +// The static form is unchanged. +func TestAction_OpenLinkStaticAddressUnchanged(t *testing.T) { + src := "create page M.P (Title: 'x', Layout: A.L) { actionbutton b (Action: open_link 'https://x.io') };" + action, ok := actionSlotValue(t, src, "Action").(*ast.ActionV3) + if !ok || action.Type != "openLink" || action.LinkURL != "https://x.io" || action.LinkAttribute != "" { + t.Errorf("static open_link = %+v", action) + } +} diff --git a/mdl/visitor/visitor_page_v3.go b/mdl/visitor/visitor_page_v3.go index faac425e84..e2cbfe1115 100644 --- a/mdl/visitor/visitor_page_v3.go +++ b/mdl/visitor/visitor_page_v3.go @@ -1050,8 +1050,9 @@ func buildActionV3(ctx parser.IActionExprV3Context) *ast.ActionV3 { actCtx := ctx.(*parser.ActionExprV3Context) action := &ast.ActionV3{} - if v := actCtx.VARIABLE(); v != nil { + if v := actCtx.VARIABLE(); v != nil && actCtx.OPEN_LINK() == nil { // $handler — a fragment action parameter; resolved at expansion. + // (OPEN_LINK $currentObject/Attr also carries a VARIABLE.) action.Type = "param" action.Target = strings.TrimPrefix(v.GetText(), "$") } else if actCtx.NOTHING() != nil { @@ -1112,6 +1113,13 @@ func buildActionV3(ctx parser.IActionExprV3Context) *ast.ActionV3 { if str := actCtx.STRING_LITERAL(); str != nil { action.LinkURL = unquoteString(str.GetText()) } + // A dynamic address: `open_link $currentObject/URL`. + if v := actCtx.VARIABLE(); v != nil { + action.LinkVariable = v.GetText() + if pathCtx := actCtx.AttributePathV3(); pathCtx != nil { + action.LinkAttribute = buildAttributePathV3(pathCtx) + } + } } else if actCtx.SIGN_OUT() != nil { action.Type = "signOut" } else if actCtx.COMPLETE_TASK() != nil { diff --git a/sdk/pages/pages_widgets_action.go b/sdk/pages/pages_widgets_action.go index 8a17145316..dc4fcca295 100644 --- a/sdk/pages/pages_widgets_action.go +++ b/sdk/pages/pages_widgets_action.go @@ -308,6 +308,9 @@ type LinkClientAction struct { model.BaseElement LinkType LinkType `json:"linkType"` Address string `json:"address,omitempty"` + // AddressAttribute, when set, makes the address dynamic: the runtime reads + // it from this attribute (Module.Entity.Attr) and Address is ignored. + AddressAttribute string `json:"addressAttribute,omitempty"` } func (LinkClientAction) isClientAction() {} From e91b8baeb76ea8f6861f7be396113c01fd353f69 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 01:22:35 +0000 Subject: [PATCH 18/47] fix(pages): refuse an expression property written as a bracketed list Slice 0 of PROPOSAL_first_class_expressions.md. The spelling mendixlabs/mxcli#750 proposes, `dynamicclasses: [ if ... then 'a' else 'b' ]`, already parses - as propertyValueV3's array alternative, into a []string - and no writer reads a list: - create page: GetStringProp / the columnClass builder take only a string, so the widget was stored with no dynamic class while check was clean and exec said "Created page". - alter page set DynamicClasses = [...]: the mutator returned nil when the type check failed - "Altered page", nothing written. - alter page set DynamicCellClass = [...] on a column: %v wrote the fused tokens `[if$x/Ythen'a'else'b']` into the Expression field. Measured with pre-fix and fixed binaries on copies of ako/TestApp (11.14.0). Pre-fix, describe showed `container c1` with no DynamicClasses while the quoted control kept its expression, and the alter left the stored value unchanged. MDL-WIDGET32 reports DynamicClasses / DynamicCellClass holding a list, with no project needed (keyed on the property, not the brackets: `visible: [cond]` stays valid). The mutator now returns an error for a list, which check -p reports through its dry run and exec stops on. Tests failed first with "got 0 violation(s), want 1" and "a bracketed list was accepted ... and reported as success"; quoted-expression and `visible: [...]` controls pass throughout. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .claude/skills/mendix/create-page/SKILL.md | 4 + CHANGELOG.md | 4 + .../750-dynamicclasses-bracket-list.fail.mdl | 23 +++++ .../pagemutator/expression_list_value_test.go | 67 +++++++++++++ mdl/backend/pagemutator/mutator.go | 32 ++++++- .../validate_widget_expression_list.go | 75 +++++++++++++++ .../validate_widget_expression_list_test.go | 94 +++++++++++++++++++ mdl/executor/validate_widgets.go | 3 + 9 files changed, 300 insertions(+), 3 deletions(-) create mode 100644 mdl-examples/bug-tests/750-dynamicclasses-bracket-list.fail.mdl create mode 100644 mdl/backend/pagemutator/expression_list_value_test.go create mode 100644 mdl/executor/validate_widget_expression_list.go create mode 100644 mdl/executor/validate_widget_expression_list_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 01d5e84aee..992e80ccd2 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -695,3 +695,4 @@ {"area":"mdl/executor","date":"2026-09-24","symptom":"`alter page M.P { insert into dv { use fragment SaveCancelFooter } }` passed `check` and failed in exec: \"failed to insert: failed to build widgets: failed to build widget SaveCancelFooter: unsupported widget type: USE_FRAGMENT\". Same for REPLACE, and for a fragment nested inside an inserted container.","cause":"Fragment/building-block expansion (pageBuilder.expandFragments) was called only on the CREATE PAGE / snippet / layout paths. ALTER PAGE's applyInsertWidgetMutator / applyReplaceWidgetMutator passed op.Widgets straight to buildWidgetsFromAST, whose pageBuilder even carried ctx.Fragments — the registry was wired, the expansion call was not. Second defect found on the way: cloneWidget copied only Type/Name/Properties/Children, dropping Specialization and TypeIsGeneric, so a cloned `template for` lost its routing.","file":"`mdl/executor/cmd_alter_page.go` (expandAlterFragments, called first in the INSERT and REPLACE mutators), `mdl/executor/cmd_pages_builder_v3.go` (cloneWidget copies the whole struct)","insight":"Expand before ANYTHING inspects the widget list, not just before the build: the duplicate-name check, allColumns and allListViewTemplates all read op.Widgets, and seeing the sentinel they check the fragment's name instead of its widgets' names. When a registry is threaded into a builder, grep for the call that consumes it (`expandFragments`), not for the field — the field being set on every pageBuilder is what made the ALTER path look covered. A field-by-field clone is a silent-drop hazard; `c := *w` then deep-copy the reference fields.","refs":["#572"]} {"area":"mdl/executor","date":"2026-09-24","symptom":"`describe layout Atlas_Core.Phone_BottomBar` emitted `-- Forms$SimpleMenuBar (simpleMenuBar1) -- NOT re-executable: mxcli cannot author this widget, so re-running this script would drop it`, so a phone layout written in MDL had no bottom bar and a describe -> exec copy of Atlas's lost it","cause":"No keyword, builder, writer or describer for Forms$SimpleMenuBar, although modelsdk/gen already had SimpleMenuBar and MenuDocumentSource. And the widget's whole point is WHICH menu document it renders (a Forms$MenuDocumentSource in MenuSource), a source kind no menu widget could express — menubar/navigationtree always wrote a Forms$NavigationSource, so one pointed at a menu document described as `menubar m` and replayed onto the Responsive profile. Layouts were also never reference-checked, so a new `Menu:` typo would have passed --references and hit CE1613 at build","file":"mdl/grammar/MDLLexer.g4 + domains/MDLPage.g4 + domains/MDLSettings.g4 (SIMPLEMENUBAR), mdl/executor/cmd_pages_builder_v3.go (buildSimpleMenuBarV3, menuSourceV3), mdl/backend/modelsdk/widget_write.go (menuSourceToGen), mdl/executor/cmd_pages_describe_parse.go + cmd_pages_describe_output.go, mdl/executor/helpers.go + validate.go (menu refs, CreateLayoutStmt), sdk/pages/pages_widgets_advanced.go","insight":"**Measure the stored shape before designing the syntax** — dumping the widget from a blank project (`mxcli new --version 11.14.0`, `bson dump --type layout`) showed the source was a menu DOCUMENT, not a profile, which is what made this a source-kind feature rather than a one-keyword copy of `menubar`. Treat MenuSource as one slot with two subtypes on all three menu widgets (one helper each side), or the siblings keep silently rewriting a menu-document source to Responsive. The control that justified the reference check: `menu: Bug573.No_Such_Menu` -> `mxcli check --references` passed, `mx check` CE1613 'The selected menu ... no longer exists.' at Simple menu bar. MDL has no `show menus`, so the not-found message lists the menus that exist. Adding reference validation to CreateLayoutStmt is new coverage for EVERY widget reference in a layout, not just menus","refs":["ako/mxcli#573"],"ce":["CE1613"]} {"area": "mdl/executor", "symptom": "`DESCRIBE PAGE Administration.Account_New` → `exec` → mx check: **CE1613** \"The selected association 'Administration.UserRoles' no longer exists\" (also User_Language, User_TimeZone, and a DataGrid2 column `Administration.Account.UserRoles/Name`). `check --references` and `exec` both report success; describe → exec → describe is byte-identical", "cause": "A bare association name was qualified with the MODULE of an entity instead of looked up. Administration.Account extends System.User, which declares UserRoles, so the page entity's module named a nonexistent association. The describer always emitted the bare name (shortAttributeName, since 41d01f01); the regression was f0d1aea80 (issuetracker #19), which switched the combobox writer from the option list's module (right here by coincidence) to the page entity's. resolveAssociationAttributePath also qualified every hop against the path's START entity", "file": "`mdl/executor/cmd_pages_builder_input.go` (`resolveAssociationPathIn` → `declaredAssociationQN`), `cmd_pages_builder_v3.go` (`resolveAssociationAttributePath` per-hop context), `cmd_pages_describe_pluggable.go` (`associationRefForContext`)", "insight": "**Two wrong heuristics each fixed the other's case**: 'module of the option list' broke issuetracker #19, 'module of the context entity' broke #662 — both guess a module from an entity name where the model can be asked. Resolve by lookup: the association with that name having an end on the context entity or a generalization, nearest first, qualified with its DECLARING module; ambiguous or unknown keeps the old guess so the validator reports the author's spelling. To find which change regressed it, build the suspect commit and its parent side by side and diff `mx check` on a copy of the project — describe output was identical on all three builds, so the writer changed, not the describer. The bisect subject must be an INHERITED association from a DIFFERENT module whose option list lives in the declaring module; a same-module fixture passes both old rules. A full-project round trip (every page, describe → exec → mx check) found the DataGrid2 `UserRoles/Name` column the issue did not name — the resolver has four call sites, fix it once there", "refs": ["ako/mxcli#662", "issuetracker #19", "ako/mxcli#664"], "ce": ["CE1613"], "date": "2026-09-25"} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "`container c1 (dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ])` - the first-class spelling mendixlabs/mxcli#750 proposes - checked clean and `exec` reported `Created page`, but the widget was stored with no DynamicClasses at all. `alter page ... set DynamicClasses = [ ... ] on w` reported `Altered page` and changed nothing; a column's `DynamicCellClass` on ALTER stored `[if$x/Ythen'a'else'b']` as its expression", "cause": "`[ ... ]` parses as propertyValueV3's array alternative, so the value reaches the AST as a []string. The create writers read only a string (GetStringProp for DynamicClasses, `v.(string)` for columnClass); the mutator's dynamicclasses case returned nil when the type check failed; setColumnPropertyMut formatted the list with %v, after the visitor's GetText() had already fused its tokens", "file": "`mdl/executor/validate_widget_expression_list.go` (MDL-WIDGET32), `mdl/backend/pagemutator/mutator.go` (`errExpressionNotAString`)", "insight": "**A proposal's example syntax is worth running through `check` before designing around it** - here the proposed spelling already parsed, and the parse was the bug. Same class as #999 / MDL-WIDGET27 (empty `[]`): a value shape no writer claims. Fix at both ends and they stay in step: a no-project check rule for CREATE, and an error (not a nil return) from the ALTER setter, which `check -p` reports for free because validateAlterSetProperties dry-runs the setter. Key the rule on the property, never on the brackets - `visible: [cond]` is valid MDL. Measured with pre-fix and fixed binaries on copies of ako/TestApp: pre-fix `describe` shows the container without DynamicClasses and the quoted control intact. Tests `validate_widget_expression_list_test.go`, `pagemutator/expression_list_value_test.go`", "refs": ["mendixlabs/mxcli#750", "#999"], "rules": ["MDL-WIDGET32"]} diff --git a/.claude/skills/mendix/create-page/SKILL.md b/.claude/skills/mendix/create-page/SKILL.md index 11542a4aab..d5bca7e7e8 100644 --- a/.claude/skills/mendix/create-page/SKILL.md +++ b/.claude/skills/mendix/create-page/SKILL.md @@ -144,6 +144,10 @@ dynamictext ovChip ( ) ``` +Write it quoted, not in brackets: `dynamicclasses: [ … ]` (and a column's +`DynamicCellClass: [ … ]`) parses as a list, which no writer reads — `check` +reports it as MDL-WIDGET32 rather than letting the value be dropped. + **All can be combined on a single widget:** ```sql container ctnHero ( diff --git a/CHANGELOG.md b/CHANGELOG.md index 3b8ad628c9..711533306e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +### Fixed + +- **An expression property written in brackets was silently dropped** (mendixlabs/mxcli#750) — `dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ]`, the spelling #750 proposes, parsed as a list that no writer reads: `check` was clean, `exec` said `Created page`, and the widget was stored with no dynamic class. `alter page … set DynamicClasses = [ … ]` said `Altered page` and changed nothing, and a column's `DynamicCellClass` stored the list's text — tokens fused, `[if$x/Ythen'a'else'b']` — as its expression. Measured on a copy of a Mendix 11.14.0 project with the pre-fix binary. `mxcli check` now reports **MDL-WIDGET32** for `DynamicClasses` and `DynamicCellClass` written as a list (no project needed), and ALTER refuses it, so `check -p` reports that too. Write the expression quoted. + ## [0.24.0] - 2026-09-24 Headline: **An element's storage GUID is the database's identity, and mxcli now treats it as one.** A production report of 28 attributes emptied across 607 rows by a single edit (mendixlabs/mxcli#1119) traced to five write paths that re-minted GUIDs — one of them moving 282 in a single module. They are fixed, and a new guard at the write choke point refuses any write that moves one: a class of data loss that leaves the model valid, `mx check` clean and `DESCRIBE` byte-identical, and surfaces only when the package meets a database that already holds data. Alongside it, `MOVE ENTITY` and `RENAME` stop leaving a project unbuildable, and four more scripts that passed every gate and failed the build are refused. diff --git a/mdl-examples/bug-tests/750-dynamicclasses-bracket-list.fail.mdl b/mdl-examples/bug-tests/750-dynamicclasses-bracket-list.fail.mdl new file mode 100644 index 0000000000..f509e602b2 --- /dev/null +++ b/mdl-examples/bug-tests/750-dynamicclasses-bracket-list.fail.mdl @@ -0,0 +1,23 @@ +-- Bug mendixlabs/mxcli#750 (slice 0 of PROPOSAL_first_class_expressions.md): +-- an expression property written in brackets was silently dropped. +-- +-- #750 proposes `dynamicclasses: [ … ]` as the first-class spelling. That text +-- already parsed — as propertyValueV3's ARRAY alternative, into a []string — and +-- both writers take only a string, so the value was discarded. Measured on a copy +-- of ako/TestApp (Mendix 11.14.0) with the pre-fix binary: +-- +-- exec -> "Created page S0.P"; describe -> `container c1` with no +-- DynamicClasses at all, while a quoted expression on c2 survived. +-- alter page … set DynamicClasses = [ … ] on c2 -> "Altered page S0.P", +-- value unchanged. +-- +-- A datagrid column's DynamicCellClass on ALTER was worse: the list was +-- formatted with %v and `[if$x/Ythen'a'else'b']` written into its Expression. +-- +-- Now `check` reports MDL-WIDGET32 (create) and the page mutator refuses the +-- list (alter, reported by `check -p` through its dry run). This file must FAIL +-- `mxcli check`. + +create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + container c1 (dynamicclasses: [ if $currentObject/Featured then 'is-featured' else 'plain' ]) { } +} diff --git a/mdl/backend/pagemutator/expression_list_value_test.go b/mdl/backend/pagemutator/expression_list_value_test.go new file mode 100644 index 0000000000..8b1100974d --- /dev/null +++ b/mdl/backend/pagemutator/expression_list_value_test.go @@ -0,0 +1,67 @@ +// SPDX-License-Identifier: Apache-2.0 + +package pagemutator + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/bsonnav" +) + +// `alter page … set DynamicClasses = [ if … then 'a' else 'b' ] on w` — the +// bracketed spelling mendixlabs/mxcli#750 proposes — reaches the mutator as a +// []string, because it parses as propertyValueV3's array alternative. The two +// expression setters handled that differently and both wrongly: +// +// - DynamicClasses wrote only `if s, ok := value.(string)` and returned nil +// otherwise: success reported, nothing written. +// - a column's DynamicCellClass formatted the list with %v and wrote +// `[if$currentObject/Featuredthen'a'else'b']` (the visitor had already fused +// the tokens) into the Expression field: success reported, garbage stored. +// +// Refusing here also makes `check -p` report it, because validateAlterSetProperties +// dry-runs this setter and keeps its error. + +func TestSetWidgetProperty_DynamicClassesRefusesAList(t *testing.T) { + rawData := makeRawPage(makeStyleableWidget("ctn1")) + m := &Mutator{rawData: rawData, widgetFinder: findBsonWidget} + + // Control first: a string is written, so the fixture can hold a value. + const stored = "'kept'" + if err := m.SetWidgetProperty("ctn1", "DynamicClasses", stored); err != nil { + t.Fatalf("control: SetWidgetProperty(DynamicClasses, string) failed: %v", err) + } + + err := m.SetWidgetProperty("ctn1", "DynamicClasses", []string{"if$currentObject/Featuredthen'a'else'b'"}) + if err == nil { + t.Fatal("a bracketed list was accepted for DynamicClasses and reported as success") + } + if !strings.Contains(err.Error(), "quoted") { + t.Errorf("error = %q, want it to name the quoted spelling that works", err) + } + app := bsonnav.DGetDoc(findBsonWidget(rawData, "ctn1").widget, "Appearance") + if got := bsonnav.DGetString(app, "DynamicClasses"); got != stored { + t.Errorf("Appearance.DynamicClasses = %q after a refused set, want %q unchanged", got, stored) + } +} + +func TestSetColumnProperty_ExpressionRefusesAList(t *testing.T) { + col, keys, kinds := columnFixture() + + // Control: a string reaches the Expression field. + if err := setColumnPropertyMut(col, keys, kinds, "DynamicCellClass", "'kept'"); err != nil { + t.Fatalf("control: DynamicCellClass string rejected: %v", err) + } + + err := setColumnPropertyMut(col, keys, kinds, "DynamicCellClass", []string{"if$currentObject/Featuredthen'a'else'b'"}) + if err == nil { + t.Fatal("a bracketed list was accepted for DynamicCellClass and reported as success") + } + if !strings.Contains(err.Error(), "quoted") { + t.Errorf("error = %q, want it to name the quoted spelling that works", err) + } + if got := fieldOf(t, col, idClass, "Expression"); got != "'kept'" { + t.Errorf("Expression = %v after a refused set, want 'kept' unchanged", got) + } +} diff --git a/mdl/backend/pagemutator/mutator.go b/mdl/backend/pagemutator/mutator.go index 669d001158..8e3ecdc57a 100644 --- a/mdl/backend/pagemutator/mutator.go +++ b/mdl/backend/pagemutator/mutator.go @@ -2341,6 +2341,12 @@ func columnValueField(kind string) (string, bool) { } func setColumnPropertyMut(colDoc bson.D, propKeyMap map[string]string, propKindMap map[string]string, propName string, value any) error { + // No column property takes a list. A bracketed value arrives as a []string, + // and the %v below turned it into `[if$x/Ythen'a'else'b']` — the tokens + // already fused by the visitor — written into Expression as success. + if _, isList := value.([]string); isList { + return errExpressionNotAString(propName, value) + } internalKey := resolveColumnPropertyKey(propName, propKeyMap) if internalKey == "" { return fmt.Errorf("column property %q not found — settable column properties on this grid are: %s", @@ -2656,10 +2662,15 @@ func setRawWidgetPropertyMut(widget bson.D, propName string, value any) error { } return nil case "dynamicclasses": + // One expression. A bracketed `[ … ]` — the spelling mendixlabs/mxcli#750 + // proposes — arrives as a []string; it used to fall past the type check + // and return nil, reporting success with nothing written. + s, ok := value.(string) + if !ok { + return errExpressionNotAString("DynamicClasses", value) + } if appearance := bsonnav.DGetDoc(widget, "Appearance"); appearance != nil { - if s, ok := value.(string); ok { - bsonnav.DSet(appearance, "DynamicClasses", s) - } + bsonnav.DSet(appearance, "DynamicClasses", s) } return nil case "editable": @@ -3177,3 +3188,18 @@ func (m *Mutator) lookupParameter(name string) (entity string, isSnippetParam bo } return "", false, false } + +// errExpressionNotAString refuses a bracketed list where a property takes a +// single value. Returned rather than skipped: ALTER's check dry-runs the setter +// (validateAlterSetProperties) and reports this error, and exec stops on it, +// instead of either reporting success for a value that was never written. +// +// The list itself is not echoed: the visitor has already fused its tokens +// (`if1>0then…`), which reads as a second, unrelated problem. +func errExpressionNotAString(propName string, _ any) error { + return fmt.Errorf( + "property %q takes a single value, but was given a bracketed list — "+ + "write an expression as a quoted string, doubling the quotes inside it: "+ + "set %s = 'if $currentObject/Featured then ''a'' else ''b'''", + propName, propName) +} diff --git a/mdl/executor/validate_widget_expression_list.go b/mdl/executor/validate_widget_expression_list.go new file mode 100644 index 0000000000..732172e56f --- /dev/null +++ b/mdl/executor/validate_widget_expression_list.go @@ -0,0 +1,75 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "sort" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// listValuedExpressionProps are the widget properties whose value is ONE Mendix +// expression, read by a writer that accepts only a string: DynamicClasses +// (WidgetV3.GetDynamicClasses) and a datagrid column's DynamicCellClass (the +// `columnClass` builder in widgetobj/datagrid_column.go). +var listValuedExpressionProps = []string{"DynamicClasses", "DynamicCellClass"} + +// validateExpressionPropertyLists (MDL-WIDGET32) rejects an expression property +// written in brackets: +// +// container c1 (dynamicclasses: [ if $currentObject/Featured then 'x' else '' ]) +// +// mendixlabs/mxcli#750 proposes exactly this spelling, and it already parses — +// as propertyValueV3's array alternative, into a []string. Both writers take +// only a string, so the value was discarded: `check` clean, `exec` reporting +// success, the widget stored with no dynamic class. The same silent drop as +// #999 (MDL-WIDGET27, which covers the empty `[]`), reached by the spelling an +// issue proposes as the fix. +// +// An error, not a warning, for MDL-WIDGET27's reason: `exec` refuses only on +// errors, and a warning would leave it free to write the page and drop the +// value. Keyed on the property, never on the brackets — `visible: [cond]` is how +// MDL spells a conditional, and a filter's `attributes: [Name]` is a real list. +// Needs no project: the value's shape is wrong whatever the widget declares. +func validateExpressionPropertyLists(w *ast.WidgetV3, locationPrefix string) []linter.Violation { + if w == nil || len(w.Properties) == 0 { + return nil + } + keys := make([]string, 0, len(w.Properties)) + for k := range w.Properties { + keys = append(keys, k) + } + sort.Strings(keys) // stable output: map order would make two runs disagree + + var out []linter.Violation + for _, key := range keys { + items, ok := w.Properties[key].([]string) + if !ok || len(items) == 0 || !isListValuedExpressionProp(key) { + continue + } + out = append(out, linter.Violation{ + RuleID: "MDL-WIDGET32", + Severity: linter.SeverityError, + Message: fmt.Sprintf( + "%s: widget `%s` property `%s` holds one Mendix expression, but is written as a bracketed list — "+ + "the value is discarded on write", + locationPrefix, w.Name, key), + Suggestion: fmt.Sprintf( + "write the expression as a quoted string, doubling the quotes inside it: "+ + "%s: 'if $currentObject/Featured then ''is-featured'' else '''''", key), + }) + } + return out +} + +func isListValuedExpressionProp(key string) bool { + for _, p := range listValuedExpressionProps { + if strings.EqualFold(p, key) { + return true + } + } + return false +} diff --git a/mdl/executor/validate_widget_expression_list_test.go b/mdl/executor/validate_widget_expression_list_test.go new file mode 100644 index 0000000000..09d4d81733 --- /dev/null +++ b/mdl/executor/validate_widget_expression_list_test.go @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" +) + +// mendixlabs/mxcli#750 proposes `dynamicclasses: [ if … then 'a' else 'b' ]` as +// the first-class spelling of an expression property. That text already parses +// — as `propertyValueV3`'s ARRAY alternative — so the value reaches the AST as a +// []string. Both readers take only a string (WidgetV3.GetDynamicClasses via +// GetStringProp, and the column builder's `columnClass` via `v.(string)`), so +// `check` passed, `exec` reported success and the widget was written with no +// dynamic class at all: #999's silent drop, reached by the spelling an issue +// proposes. An empty `[]` is already MDL-WIDGET27; this is the non-empty case. +func TestMDLWIDGET32_ExpressionPropertyWrittenAsList(t *testing.T) { + cases := []struct { + name string + src string + want int + }{ + { + // The shape #750 proposes, verbatim. + name: "issue shape: dynamicclasses in brackets", + src: `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + container c1 (dynamicclasses: [ if $currentObject/Featured then 'is-featured' else '' ]) { } +}`, + want: 1, + }, + { + name: "canonical casing", + src: `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + container c1 (DynamicClasses: ['a']) { } +}`, + want: 1, + }, + { + name: "datagrid column DynamicCellClass in brackets", + src: `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + datagrid dg (datasource: database M.Thing) { + column c1 (attribute: Name, caption: 'N', DynamicCellClass: [ if $currentObject/Featured then 'hot' else '' ]) + } +}`, + want: 1, + }, + { + // The quoted expression is how the property is written today, and + // is what reaches storage: the control. + name: "control: quoted expression", + src: `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + container c1 (dynamicclasses: 'if $currentObject/Featured then ''is-featured'' else ''''') { } +}`, + want: 0, + }, + { + name: "control: quoted column expression", + src: `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + datagrid dg (datasource: database M.Thing) { + column c1 (attribute: Name, caption: 'N', DynamicCellClass: 'if $currentObject/Featured then ''hot'' else ''''') + } +}`, + want: 0, + }, + { + // Brackets are how MDL spells OTHER properties; the rule is keyed on + // the property, never on the brackets. + name: "control: visible takes a bracketed condition", + src: `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + container c1 (visible: [Active = true]) { } +}`, + want: 0, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + got := widgetViolations(t, tc.src, "MDL-WIDGET32") + if len(got) != tc.want { + t.Fatalf("MDL-WIDGET32: got %d violation(s), want %d: %#v", len(got), tc.want, got) + } + if tc.want == 0 { + return + } + // The message has to carry its own remedy: the quoted spelling that + // does reach storage. + for _, s := range []string{"discarded", "quoted"} { + if !strings.Contains(got[0].Message+got[0].Suggestion, s) { + t.Errorf("message should mention %q: %s / %s", s, got[0].Message, got[0].Suggestion) + } + } + }) + } +} diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index 93bb98cc84..54cb78e9df 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -168,6 +168,9 @@ func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, loc // every widget kind and needs no definition: the SHAPE is wrong whatever // the widget declares. out = append(out, validateObjectEntryProperties(w, registry, locationPrefix)...) + // An expression property written in brackets — the spelling #750 + // proposes — parses as a list and was discarded on write. + out = append(out, validateExpressionPropertyLists(w, locationPrefix)...) // #1062: an action slot holding something that is not an action, which // used to check clean, exec clean, build clean and render dead. Runs for // every widget kind and needs no definition, for the same reason as the From 0b7143f023b86b8f8a5f6d92266a222ea5e1933f Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 01:22:53 +0000 Subject: [PATCH 19/47] docs(changelog): OData client quoting, proxy and credential fixes Entries for 26880a04, dad9be6d, 6a48ed72 and dcd6f71e, which shipped without them. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 711533306e..9989d6ab7f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed - **An expression property written in brackets was silently dropped** (mendixlabs/mxcli#750) — `dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ]`, the spelling #750 proposes, parsed as a list that no writer reads: `check` was clean, `exec` said `Created page`, and the widget was stored with no dynamic class. `alter page … set DynamicClasses = [ … ]` said `Altered page` and changed nothing, and a column's `DynamicCellClass` stored the list's text — tokens fused, `[if$x/Ythen'a'else'b']` — as its expression. Measured on a copy of a Mendix 11.14.0 project with the pre-fix binary. `mxcli check` now reports **MDL-WIDGET32** for `DynamicClasses` and `DynamicCellClass` written as a list (no project needed), and ALTER refuses it, so `check -p` reports that too. Write the expression quoted. +- **`describe odata client` lost a quote level on a literal credential** — Studio Pro stores a literal user name as the expression `'abc'`, quotes included. `describe` printed `HttpUsername: 'abc'`, and re-executing that output stored `abc`, an identifier. `ClientCertificate`, header keys, `Version`, `MetadataUrl` and `Folder` were printed unescaped and did not re-parse when they held a quote. Every value is now quoted so a re-exec stores exactly what was read; measured against a Studio Pro-authored client decoded before and after a round trip. +- **An OData client's proxy constant written `@Module.Const` was stored with the `@`** — `ProxyHost` / `ProxyPort` / `ProxyUsername` / `ProxyPassword` are by-name references to a constant, and Studio Pro stores the bare name (with `ProxyType: Override`). `"@Module.Const"` named no constant, so the proxy resolved to nothing. `create`, `create or modify` and `alter` now store the bare name for the bare, `@` and quoted-`@` spellings. The constant may be a String or an Integer. +- **The design-time `$metadata` fetch sent a literal credential with its quotes** — `HttpUsername: '''MxAdmin'''`, the spelling that stores Studio Pro's `'MxAdmin'`, was sent as `'MxAdmin'`: a 401 and an empty client. A credential that is a single string literal now sends its content; a compound expression (`'Key ' + @M.C`) is reported unresolved instead of sent as text. +- **The `odata-data-sharing` skill taught credentials that store the wrong value** — its walkthrough wrote `HttpUsername: 'MxAdmin'`, which stores the identifier `MxAdmin`. It now writes `'''MxAdmin'''` and explains why. ## [0.24.0] - 2026-09-24 From 88630b9dd5ca9d15dd60cc36f1cb45524ffeb20f Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 01:23:23 +0000 Subject: [PATCH 20/47] test(odata): detach the file's explanatory comment from the helper The package-level explanation ran straight into describeAndReparse's doc comment, so godoc attached both to the helper. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- mdl/executor/cmd_odata_client_describe_quoting_test.go | 1 + 1 file changed, 1 insertion(+) diff --git a/mdl/executor/cmd_odata_client_describe_quoting_test.go b/mdl/executor/cmd_odata_client_describe_quoting_test.go index c657e749e9..5fdab30e06 100644 --- a/mdl/executor/cmd_odata_client_describe_quoting_test.go +++ b/mdl/executor/cmd_odata_client_describe_quoting_test.go @@ -19,6 +19,7 @@ import ( // an identifier, not a string. Measured on a Studio Pro-authored client // (ako/TestApp@37e0cc0, Odata.Bug1073). ClientCertificate, header keys and the // plain string properties were printed as a raw '%s', unescaped. + // describeAndReparse runs DESCRIBE on stored and parses the output with the real // visitor, returning what a re-exec would store. func describeAndReparse(t *testing.T, stored *model.ConsumedODataService, folder string) (*ast.CreateODataClientStmt, string) { From c7ffc0fecfdf1a09a597bccdd186fd079f25940c Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 06:08:04 +0000 Subject: [PATCH 21/47] feat(pages): author Studio Pro's Label widget; describe emits it named MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit describe → exec of Administration.Account_Edit and FeedbackModule.ShareFeedback(_Logo) failed to parse: statictext (Content: 'Mendix AppCloud users are provisioned by ...') line 14:21 extraneous input '(' expecting the start of a statement The stored widget is a Forms$Label named `label4` (bson dump). The emitter hard-coded `statictext (Content: …)`, dropping the name and appearance. Even named, `statictext` writes Forms$Text, which Mendix 11 cannot load (MDL-WIDGET29), and nothing could write a Forms$Label. - `label (Content: …)` is a built-in widget keyword (LABEL in widgetTypeV3; as a generic type `check -p` refused it as MDL-WIDGET25). - It builds pages.Label and writes Forms$Label with Studio Pro's key set: Appearance, Caption, null ConditionalVisibilitySettings, Name and TabIndex. gen's extra top-level Class/Style/AccessibilitySettings stay unwritten. - DESCRIBE emits `label ` with its content, class, style and design properties. The legacy Forms$Text case keeps its name too, so the output parses and MDL-WIDGET29 is what the reader sees. Round trip of the stock project: Account_Edit and ShareFeedback now exec, and the label BSON matches except for two pre-existing all-widget gaps (attribute-condition visibility and a design-property list marker). The bug-test MDL gives `mx check` 0 errors and an idempotent round trip. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../mendix/create-page/reference/widgets.md | 13 +++ docs-site/src/reference/page/create-page.md | 1 + .../bug-tests/describe-label-widget.mdl | 34 +++++++ mdl/backend/modelsdk/widget_write.go | 18 ++++ .../modelsdk/widget_write_label_test.go | 59 ++++++++++++ mdl/executor/cmd_pages_builder_v3.go | 2 + mdl/executor/cmd_pages_builder_v3_widgets.go | 32 +++++++ mdl/executor/cmd_pages_describe_output.go | 20 +++- mdl/executor/cmd_pages_label_widget_test.go | 91 +++++++++++++++++++ mdl/grammar/domains/MDLPage.g4 | 1 + mdl/visitor/visitor_page_label_widget_test.go | 45 +++++++++ 12 files changed, 315 insertions(+), 2 deletions(-) create mode 100644 mdl-examples/bug-tests/describe-label-widget.mdl create mode 100644 mdl/backend/modelsdk/widget_write_label_test.go create mode 100644 mdl/executor/cmd_pages_label_widget_test.go create mode 100644 mdl/visitor/visitor_page_label_widget_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 01d5e84aee..d632bcb873 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -695,3 +695,4 @@ {"area":"mdl/executor","date":"2026-09-24","symptom":"`alter page M.P { insert into dv { use fragment SaveCancelFooter } }` passed `check` and failed in exec: \"failed to insert: failed to build widgets: failed to build widget SaveCancelFooter: unsupported widget type: USE_FRAGMENT\". Same for REPLACE, and for a fragment nested inside an inserted container.","cause":"Fragment/building-block expansion (pageBuilder.expandFragments) was called only on the CREATE PAGE / snippet / layout paths. ALTER PAGE's applyInsertWidgetMutator / applyReplaceWidgetMutator passed op.Widgets straight to buildWidgetsFromAST, whose pageBuilder even carried ctx.Fragments — the registry was wired, the expansion call was not. Second defect found on the way: cloneWidget copied only Type/Name/Properties/Children, dropping Specialization and TypeIsGeneric, so a cloned `template for` lost its routing.","file":"`mdl/executor/cmd_alter_page.go` (expandAlterFragments, called first in the INSERT and REPLACE mutators), `mdl/executor/cmd_pages_builder_v3.go` (cloneWidget copies the whole struct)","insight":"Expand before ANYTHING inspects the widget list, not just before the build: the duplicate-name check, allColumns and allListViewTemplates all read op.Widgets, and seeing the sentinel they check the fragment's name instead of its widgets' names. When a registry is threaded into a builder, grep for the call that consumes it (`expandFragments`), not for the field — the field being set on every pageBuilder is what made the ALTER path look covered. A field-by-field clone is a silent-drop hazard; `c := *w` then deep-copy the reference fields.","refs":["#572"]} {"area":"mdl/executor","date":"2026-09-24","symptom":"`describe layout Atlas_Core.Phone_BottomBar` emitted `-- Forms$SimpleMenuBar (simpleMenuBar1) -- NOT re-executable: mxcli cannot author this widget, so re-running this script would drop it`, so a phone layout written in MDL had no bottom bar and a describe -> exec copy of Atlas's lost it","cause":"No keyword, builder, writer or describer for Forms$SimpleMenuBar, although modelsdk/gen already had SimpleMenuBar and MenuDocumentSource. And the widget's whole point is WHICH menu document it renders (a Forms$MenuDocumentSource in MenuSource), a source kind no menu widget could express — menubar/navigationtree always wrote a Forms$NavigationSource, so one pointed at a menu document described as `menubar m` and replayed onto the Responsive profile. Layouts were also never reference-checked, so a new `Menu:` typo would have passed --references and hit CE1613 at build","file":"mdl/grammar/MDLLexer.g4 + domains/MDLPage.g4 + domains/MDLSettings.g4 (SIMPLEMENUBAR), mdl/executor/cmd_pages_builder_v3.go (buildSimpleMenuBarV3, menuSourceV3), mdl/backend/modelsdk/widget_write.go (menuSourceToGen), mdl/executor/cmd_pages_describe_parse.go + cmd_pages_describe_output.go, mdl/executor/helpers.go + validate.go (menu refs, CreateLayoutStmt), sdk/pages/pages_widgets_advanced.go","insight":"**Measure the stored shape before designing the syntax** — dumping the widget from a blank project (`mxcli new --version 11.14.0`, `bson dump --type layout`) showed the source was a menu DOCUMENT, not a profile, which is what made this a source-kind feature rather than a one-keyword copy of `menubar`. Treat MenuSource as one slot with two subtypes on all three menu widgets (one helper each side), or the siblings keep silently rewriting a menu-document source to Responsive. The control that justified the reference check: `menu: Bug573.No_Such_Menu` -> `mxcli check --references` passed, `mx check` CE1613 'The selected menu ... no longer exists.' at Simple menu bar. MDL has no `show menus`, so the not-found message lists the menus that exist. Adding reference validation to CreateLayoutStmt is new coverage for EVERY widget reference in a layout, not just menus","refs":["ako/mxcli#573"],"ce":["CE1613"]} {"area": "mdl/executor", "symptom": "`DESCRIBE PAGE Administration.Account_New` → `exec` → mx check: **CE1613** \"The selected association 'Administration.UserRoles' no longer exists\" (also User_Language, User_TimeZone, and a DataGrid2 column `Administration.Account.UserRoles/Name`). `check --references` and `exec` both report success; describe → exec → describe is byte-identical", "cause": "A bare association name was qualified with the MODULE of an entity instead of looked up. Administration.Account extends System.User, which declares UserRoles, so the page entity's module named a nonexistent association. The describer always emitted the bare name (shortAttributeName, since 41d01f01); the regression was f0d1aea80 (issuetracker #19), which switched the combobox writer from the option list's module (right here by coincidence) to the page entity's. resolveAssociationAttributePath also qualified every hop against the path's START entity", "file": "`mdl/executor/cmd_pages_builder_input.go` (`resolveAssociationPathIn` → `declaredAssociationQN`), `cmd_pages_builder_v3.go` (`resolveAssociationAttributePath` per-hop context), `cmd_pages_describe_pluggable.go` (`associationRefForContext`)", "insight": "**Two wrong heuristics each fixed the other's case**: 'module of the option list' broke issuetracker #19, 'module of the context entity' broke #662 — both guess a module from an entity name where the model can be asked. Resolve by lookup: the association with that name having an end on the context entity or a generalization, nearest first, qualified with its DECLARING module; ambiguous or unknown keeps the old guess so the validator reports the author's spelling. To find which change regressed it, build the suspect commit and its parent side by side and diff `mx check` on a copy of the project — describe output was identical on all three builds, so the writer changed, not the describer. The bisect subject must be an INHERITED association from a DIFFERENT module whose option list lives in the declaring module; a same-module fixture passes both old rules. A full-project round trip (every page, describe → exec → mx check) found the DataGrid2 `UserRoles/Name` column the issue did not name — the resolver has four call sites, fix it once there", "refs": ["ako/mxcli#662", "issuetracker #19", "ako/mxcli#664"], "ce": ["CE1613"], "date": "2026-09-25"} +{"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_Edit / FeedbackModule.ShareFeedback(_Logo) fails `Parse error: line 14:21 extraneous input '(' expecting the start of a statement`; describe emitted `statictext (Content: '…')` with no widget name", "cause": "The stored widget is Studio Pro's Label (Forms$Label), and it is NAMED (`label4`). The describe emitter hard-coded `statictext (Content: %s)`, dropping name and appearance. MDL had no widget that writes Forms$Label; `statictext` writes Forms$Text, which Mendix 11 cannot load (MDL-WIDGET29), so neither 'make the name optional' nor 'synthesize a name' could have produced a correct round trip", "file": "`mdl/executor/cmd_pages_describe_output.go` (Forms$Label case), `mdl/grammar/domains/MDLPage.g4` (`LABEL` in widgetTypeV3), `mdl/executor/cmd_pages_builder_v3_widgets.go` (`buildLabelV3`), `mdl/backend/modelsdk/widget_write.go` (`*pages.Label` case + Forms$Label TypeDefaults)", "insight": "**Dump the stored widget before accepting the report's diagnosis**: 'the stored Name is empty' was an inference from the output, and one `bson dump --format ndsl` showed a named Forms$Label — which also made both proposed fixes wrong, since the keyword itself wrote an unloadable type. A generic (IDENTIFIER) widget type parses but `check -p` requires it to resolve to a pluggable definition (MDL-WIDGET25); a built-in widget needs its token in widgetTypeV3, and a visitor test must assert `!TypeIsGeneric` or it passes against the unfixed grammar. gen's Label declares top-level Class/Style/AccessibilitySettings that Studio Pro 11 does not store — assert the encoded key set with encodeToD, and register NullFields for ConditionalVisibilitySettings or the key is omitted. Round-tripping a page that previously failed to parse EXPOSES older write gaps on the same page: here attribute-condition visibility (8 Enumerations$Condition → 0, all widgets), a nanoflow data-view source shape (CE2633), and compound design-property list markers (2 → 3) — diff the stored BSON before/after, not just mx check", "refs": ["Administration.Account_Edit", "FeedbackModule.ShareFeedback"], "rules": ["MDL-WIDGET25", "MDL-WIDGET29"], "date": "2026-09-25"} diff --git a/.claude/skills/mendix/create-page/reference/widgets.md b/.claude/skills/mendix/create-page/reference/widgets.md index c0d3911e38..4553df8735 100644 --- a/.claude/skills/mendix/create-page/reference/widgets.md +++ b/.claude/skills/mendix/create-page/reference/widgets.md @@ -66,6 +66,19 @@ dynamictext day (content: '{1}', contentparams: [{1} = DueOn format (dateFormat > NullReferenceException when the widget is opened. Bind every placeholder, or use > a plain static `content: 'text'`. +### LABEL Widget + +Studio Pro's Label widget (`Forms$Label`) — a fixed caption. Stock marketplace +modules still carry it (Administration's `Account_Edit`, the Feedback module), so +`describe page` emits it; for new text prefer `dynamictext`. + +```sql +label label4 (content: 'Attachment', class: 'text-semibold') +``` + +Do not write `statictext`: it stores `Forms$Text`, a type Mendix 11 cannot load +(MDL-WIDGET29). + ### ACTIONBUTTON Widget Create a button with action binding: diff --git a/docs-site/src/reference/page/create-page.md b/docs-site/src/reference/page/create-page.md index 43766d97bb..4cbd52c384 100644 --- a/docs-site/src/reference/page/create-page.md +++ b/docs-site/src/reference/page/create-page.md @@ -78,6 +78,7 @@ The widget tree inside `{ ... }` defines the page content. Widgets are nested hi | Widget | Description | Key Properties | |--------|-------------|----------------| | `DYNAMICTEXT` | Display-only text bound to an attribute | `Attribute` | +| `LABEL` | Studio Pro's Label widget (`Forms$Label`): a fixed, translatable caption. `describe` emits it for existing labels; prefer `DYNAMICTEXT` for new text | `Content` | | `IMAGE` | Generic image | `Width`, `Height` | | `STATICIMAGE` | Fixed image from project resources | `Width`, `Height` | | `DYNAMICIMAGE` | Image from an entity attribute | `Width`, `Height` | diff --git a/mdl-examples/bug-tests/describe-label-widget.mdl b/mdl-examples/bug-tests/describe-label-widget.mdl new file mode 100644 index 0000000000..d5dd628641 --- /dev/null +++ b/mdl-examples/bug-tests/describe-label-widget.mdl @@ -0,0 +1,34 @@ +-- ============================================================================ +-- DESCRIBE emitted a Forms$Label as a nameless `statictext` +-- ============================================================================ +-- +-- Symptom: describe → exec of Administration.Account_Edit (Administration +-- v4.3.2) and FeedbackModule.ShareFeedback / ShareFeedback_Logo (Feedback +-- v4.0.2), Mendix 11.13.0: +-- statictext (Content: 'Mendix AppCloud users are provisioned by ...') +-- Parse error: line 14:21 extraneous input '(' expecting the start of a statement +-- +-- Cause: the stored widget is Studio Pro's Label (Forms$Label), and it IS +-- named (`label4`). The emitter hard-coded `statictext (Content: …)`, dropping +-- the name and the appearance. And even named, `statictext` writes Forms$Text, +-- a type Mendix 11 cannot load (MDL-WIDGET29) — MDL had no way to write a +-- Forms$Label at all. +-- +-- Fix: a `label` widget that writes Forms$Label (Appearance, Caption, null +-- ConditionalVisibilitySettings, Name, TabIndex — Studio Pro's key set), and +-- DESCRIBE emits it with its name, class, style and design properties. +-- +-- Verify: exec, then describe → exec again (expect "Unchanged page"), then +-- `mxcli docker check` — 0 errors. +-- ============================================================================ + +create or replace page MyFirstModule.LabelWidget_Test +( Title: 'Labels', Layout: Atlas_Core.Atlas_Default ) +{ + label lblPlain (Content: 'Attachment') + label lblStyled (Content: 'Provisioned elsewhere', Class: 'alert alert-warning', Style: 'width:100%;') + label lblSpaced (Content: 'Spaced', Class: 'text-semibold', DesignProperties: ['Spacing': ['margin-bottom': 'None']]) +} +/ + +describe page MyFirstModule.LabelWidget_Test; diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index bd297391e3..ec2e23a9df 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -107,6 +107,13 @@ func init() { NullFields: []string{"ConditionalVisibilitySettings", "NativeAccessibilitySettings"}, }) codec.RegisterListMarker("Forms$Title", 2) + // Label (Studio Pro's Label widget): a null visibility slot — the one it has; + // no NativeAccessibilitySettings — and marker 2 as a widget. Measured on the + // three Forms$Label in a stock Administration + Feedback project (11.13.0). + codec.RegisterTypeDefaults("Forms$Label", codec.TypeDefaults{ + NullFields: []string{"ConditionalVisibilitySettings"}, + }) + codec.RegisterListMarker("Forms$Label", 2) // Conditional visibility/editability settings (issue #627). When a widget // carries one, applyWidgetBase emits the node; these defaults fill the // sub-fields Studio Pro writes: empty-string Attribute, null SourceVariable, and @@ -417,6 +424,17 @@ func widgetToGen(w pages.Widget) (element.Element, error) { g.SetCaption(captionToGen(x.Caption)) return g, nil + case *pages.Label: + // Studio Pro's Label widget. Stored with exactly Appearance, Caption, + // ConditionalVisibilitySettings, Name and TabIndex — measured on the + // three in a stock Administration v4.3.2 + Feedback v4.0.2 project at + // 11.13.0. gen's Label also declares top-level Class/Style and + // AccessibilitySettings; they are left unset, so not written. + g := genPg.NewLabel() + applyWidgetBase(g, &x.BaseWidget) + g.SetCaption(captionToGen(x.Caption)) + return g, nil + case *pages.TextBox: g := genPg.NewTextBox() applyWidgetBase(g, &x.BaseWidget) diff --git a/mdl/backend/modelsdk/widget_write_label_test.go b/mdl/backend/modelsdk/widget_write_label_test.go new file mode 100644 index 0000000000..927cde7b2f --- /dev/null +++ b/mdl/backend/modelsdk/widget_write_label_test.go @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "sort" + "testing" + + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// A Forms$Label as Studio Pro stores it — measured on the three in a stock +// Administration v4.3.2 + Feedback v4.0.2 project at Mendix 11.13.0 — has +// exactly these top-level keys. gen's Label type ALSO declares top-level Class, +// Style and AccessibilitySettings (older metamodel versions); writing those +// would be keys Studio Pro does not store. +func TestWidgetToGen_Label(t *testing.T) { + el, err := widgetToGen(&pages.Label{ + BaseWidget: pages.BaseWidget{ + BaseElement: model.BaseElement{ID: "lbl-id", TypeName: "Forms$Label"}, + Name: "label4", + Class: "text-semibold", + }, + Caption: &model.Text{Translations: map[string]string{"en_US": "Attachment"}}, + }) + if err != nil { + t.Fatalf("widgetToGen refused a Label: %v", err) + } + d := encodeToD(t, el) + var keys []string + for _, e := range d { + if e.Key != "$ID" { + keys = append(keys, e.Key) + } + } + sort.Strings(keys) + want := []string{"$Type", "Appearance", "Caption", "ConditionalVisibilitySettings", "Name", "TabIndex"} + if len(keys) != len(want) { + t.Fatalf("keys = %v, want %v", keys, want) + } + for i := range want { + if keys[i] != want[i] { + t.Fatalf("keys = %v, want %v", keys, want) + } + } + for _, e := range d { + switch e.Key { + case "$Type": + if e.Value != "Forms$Label" { + t.Errorf("$Type = %v", e.Value) + } + case "Name": + if e.Value != "label4" { + t.Errorf("Name = %v", e.Value) + } + } + } +} diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index 9d9d8708d3..e1d520052c 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -400,6 +400,8 @@ func (pb *pageBuilder) buildWidgetV3(w *ast.WidgetV3) (pages.Widget, error) { widget, err = pb.buildDynamicTextV3(w) case "title": widget, err = pb.buildTitleV3(w) + case "label": + widget, err = pb.buildLabelV3(w) case "button", "actionbutton", "linkbutton": widget, err = pb.buildButtonV3(w) case "tabcontainer": diff --git a/mdl/executor/cmd_pages_builder_v3_widgets.go b/mdl/executor/cmd_pages_builder_v3_widgets.go index 374f58cd9f..9d0d0a3c12 100644 --- a/mdl/executor/cmd_pages_builder_v3_widgets.go +++ b/mdl/executor/cmd_pages_builder_v3_widgets.go @@ -921,6 +921,38 @@ func (pb *pageBuilder) buildTitleV3(w *ast.WidgetV3) (*pages.Title, error) { return title, nil } +// buildLabelV3 builds Studio Pro's Label widget (Forms$Label): a name, a +// translatable caption and an appearance. +// +// DESCRIBE emits it for the Forms$Label that stock marketplace modules still +// carry (Administration.Account_Edit, FeedbackModule.ShareFeedback). Before it +// existed describe wrote `statictext`, which writes Forms$Text — a type Mendix +// 11 cannot load — so the round trip had nothing correct to write back. +func (pb *pageBuilder) buildLabelV3(w *ast.WidgetV3) (*pages.Label, error) { + label := &pages.Label{ + BaseWidget: pages.BaseWidget{ + BaseElement: model.BaseElement{ + ID: model.ID(types.GenerateID()), + TypeName: "Forms$Label", + }, + Name: w.Name, + }, + } + if content := w.GetContent(); content != "" { + label.Caption = &model.Text{ + BaseElement: model.BaseElement{ + ID: model.ID(types.GenerateID()), + TypeName: "Texts$Text", + }, + Translations: map[string]string{pb.textLang(): content}, + } + } + if err := pb.registerWidgetName(w.Name, label.ID); err != nil { + return nil, err + } + return label, nil +} + func (pb *pageBuilder) buildButtonV3(w *ast.WidgetV3) (*pages.ActionButton, error) { btn := &pages.ActionButton{ BaseWidget: pages.BaseWidget{ diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index c0c5c0b5e4..ede1c21f7c 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -485,7 +485,11 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { props = append(props, fmt.Sprintf("Content: %s", mdlQuote(w.Content))) } props = appendAppearanceProps(props, w) - formatWidgetProps(ctx.Output, prefix, "statictext", props, "\n") + // Forms$Text only survives in a project converted up from an old + // Mendix; writing one is refused (MDL-WIDGET29). Keep the name anyway, + // so the output parses and that refusal — not a parse error — is what + // the reader sees. + formatWidgetProps(ctx.Output, prefix, fmt.Sprintf("statictext %s", mdlIdent(w.Name)), props, "\n") case "Forms$Title", "Pages$Title": header := fmt.Sprintf("title %s", mdlIdent(w.Name)) @@ -864,7 +868,19 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { fmt.Fprintf(ctx.Output, "%s}\n", prefix) case "Forms$Label", "Pages$Label": - fmt.Fprintf(ctx.Output, "%sstatictext (Content: %s)\n", prefix, mdlQuote(w.Content)) + // Studio Pro's Label widget. This used to print `statictext (Content: …)` + // — no name, so the output did not parse, and `statictext` writes + // Forms$Text, which Mendix 11 cannot load (MDL-WIDGET29). Measured on + // Administration.Account_Edit and FeedbackModule.ShareFeedback(_Logo): + // each Forms$Label there is named (`label4`), so the name was dropped + // here, not missing in the model. + header := fmt.Sprintf("label %s", mdlIdent(w.Name)) + props := []string{} + if w.Content != "" { + props = append(props, fmt.Sprintf("Content: %s", mdlQuote(w.Content))) + } + props = appendAppearanceProps(props, w) + formatWidgetProps(ctx.Output, prefix, header, props, "\n") case "Forms$Gallery", "Pages$Gallery": header := fmt.Sprintf("gallery %s", mdlIdent(w.Name)) diff --git a/mdl/executor/cmd_pages_label_widget_test.go b/mdl/executor/cmd_pages_label_widget_test.go new file mode 100644 index 0000000000..d7fb55c383 --- /dev/null +++ b/mdl/executor/cmd_pages_label_widget_test.go @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "context" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// A Studio Pro Forms$Label, as FeedbackModule.ShareFeedback stores it +// (Feedback v4.0.2): a name, a Texts$Text caption, and an appearance. +func storedLabelWidget() map[string]any { + return map[string]any{ + "$Type": "Forms$Label", + "Name": "label4", + "Appearance": map[string]any{ + "$Type": "Forms$Appearance", + "Class": "alert alert-warning", + "Style": "width:100%;", + }, + "Caption": map[string]any{ + "$Type": "Texts$Text", + "Items": []any{int32(3), map[string]any{ + "$Type": "Texts$Translation", "LanguageCode": "en_US", "Text": "Attachment", + }}, + }, + "TabIndex": int32(0), + } +} + +// DESCRIBE wrote `statictext (Content: 'Attachment')` — no name, so exec of +// the output failed `extraneous input '(' expecting the start of a statement` +// (3 of 17 pages of a stock Administration + Feedback project). It must emit a +// named `label` that parses and keeps the appearance. +func TestDescribeLabelWidget_EmitsNamedLabel(t *testing.T) { + var buf bytes.Buffer + ctx := (&Executor{}).newExecContext(context.Background()) + ctx.Output = &buf + raw := parseRawWidget(ctx, storedLabelWidget()) + if len(raw) != 1 { + t.Fatalf("parseRawWidget returned %d widgets", len(raw)) + } + outputWidgetMDLV3(ctx, raw[0], 1) + got := buf.String() + for _, want := range []string{"label label4", "Content: 'Attachment'", "Class: 'alert alert-warning'", "Style: 'width:100%;'"} { + if !strings.Contains(got, want) { + t.Errorf("describe output lacks %q:\n%s", want, got) + } + } + src := "create page M.P (Title: 'x', Layout: A.L) {\n" + got + "}\n" + if _, errs := visitor.Build(src); len(errs) > 0 { + t.Fatalf("describe output does not parse: %v\n%s", errs, src) + } +} + +// `label` builds a Forms$Label — not Forms$Text, which Mendix 11 cannot load. +func TestBuildLabelWidget_WritesFormsLabel(t *testing.T) { + pb := newTestPageBuilderForLabel() + w := &ast.WidgetV3{Type: "label", Name: "label4", Properties: map[string]any{"Content": "Attachment", "Class": "text-semibold"}} + built, err := pb.buildWidgetV3(w) + if err != nil { + t.Fatalf("build: %v", err) + } + lbl, ok := built.(*pages.Label) + if !ok { + t.Fatalf("built %T, want *pages.Label", built) + } + if lbl.TypeName != "Forms$Label" || lbl.Name != "label4" { + t.Errorf("TypeName/Name = %q/%q", lbl.TypeName, lbl.Name) + } + if lbl.Caption == nil || lbl.Caption.Translations["en_US"] != "Attachment" { + t.Errorf("Caption = %+v", lbl.Caption) + } + if lbl.Class != "text-semibold" { + t.Errorf("Class = %q", lbl.Class) + } +} + +func newTestPageBuilderForLabel() *pageBuilder { + return &pageBuilder{ + paramEntityNames: map[string]string{}, + widgetScope: map[string]model.ID{}, + } +} diff --git a/mdl/grammar/domains/MDLPage.g4 b/mdl/grammar/domains/MDLPage.g4 index d2d6190e00..70e06ee280 100644 --- a/mdl/grammar/domains/MDLPage.g4 +++ b/mdl/grammar/domains/MDLPage.g4 @@ -392,6 +392,7 @@ widgetTypeV3 | ACTIONBUTTON | LINKBUTTON | TITLE + | LABEL // Forms$Label (Studio Pro's Label widget) | DYNAMICTEXT | STATICTEXT | SNIPPETCALL diff --git a/mdl/visitor/visitor_page_label_widget_test.go b/mdl/visitor/visitor_page_label_widget_test.go new file mode 100644 index 0000000000..633d9a3b7e --- /dev/null +++ b/mdl/visitor/visitor_page_label_widget_test.go @@ -0,0 +1,45 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// `label` is the widget keyword for Forms$Label — Studio Pro's Label widget, +// still loadable on Mendix 11 and carried by stock marketplace modules +// (Administration.Account_Edit, FeedbackModule.ShareFeedback). DESCRIBE had no +// keyword to emit for it and wrote `statictext (Content: …)` with no name, +// which does not parse (and `statictext` writes Forms$Text, which Mendix 11 +// cannot load). +func TestLabelWidgetParses(t *testing.T) { + src := "create page M.P (Title: 'x', Layout: A.L) { label label4 (Content: 'Attachment', Class: 'text-semibold') };" + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse: %v", errs) + } + w := prog.Statements[0].(*ast.CreatePageStmtV3).Widgets[0] + if !strings.EqualFold(w.Type, "label") || w.Name != "label4" { + t.Errorf("widget = %s %q, want label label4", w.Type, w.Name) + } + // A built-in keyword, not a generic type: a generic one must resolve to a + // pluggable widget definition, and `check -p` refused `label` as + // MDL-WIDGET25 "not a widget in this project". + if w.TypeIsGeneric { + t.Error("label parsed as a generic widget type; it must be an enumerated widgetTypeV3 token") + } + if w.GetContent() != "Attachment" { + t.Errorf("Content = %q", w.GetContent()) + } +} + +// `Label:` stays a property on other widgets — the keyword is shared. +func TestLabelPropertyStillParses(t *testing.T) { + src := "create page M.P (Title: 'x', Layout: A.L) { textbox tb (Label: 'Name', Attribute: Name) };" + if _, errs := Build(src); len(errs) > 0 { + t.Fatalf("parse: %v", errs) + } +} From 51d8963d56694474c4bdd6aac9ff162478620e38 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 06:32:30 +0000 Subject: [PATCH 22/47] feat(odata): write client credentials and header values as expressions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Decided in PROPOSAL_first_class_expressions.md §6.4 (question 5): HttpUsername, HttpPassword, ClientCertificate and header values hold a Mendix expression, and MDL now writes it as-is. `HttpUsername: 'admin'` is the string 'admin' - what Studio Pro stores - where it used to store the identifier `admin` and a string needed `'''admin'''`. `@Mod.C` reads a constant, and `'Bearer ' + @Mod.Token` concatenates. - grammar: odataPropertyAssignment, odataAlterAssignment and odataHeaderEntry accept `expression` after the plain value forms, so existing plain values keep their parse. - visitor: the four slots store the expression's source text (whitespace kept; GetText() would fuse `'a' + @M.C`). An expression in any other OData property - client, service, business event, alter - is an error, so the wider rule cannot open a silent empty value. - describe prints the stored expression as-is; Studio Pro's client (ako/TestApp Odata.Bug1073) now reads `HttpUsername: 'abc'`. - MDL-ODATA07 refuses the two older spellings whose meaning changed: doubled quotes (would now store quote characters) and a quoted `'@Mod.C'` (would now store the literal text). check and exec both refuse; nothing is written. Measured on a copy of ako/TestApp: `'admin'`, `@Q5.ApiKey`, `'it''s'` and a concatenated header store exactly those expressions; describe prints the same MDL back; a describe -> exec round trip stores identical values. The design-time $metadata fetch sends `admin` for `'admin'` (resolveCredential already evaluates a string literal). The examples, the odata-data-sharing skill, the MxGraphStudio case study and the regression MDL move to the new spelling. ServiceUrl is unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../reference/walkthroughs.md | 23 ++- CHANGELOG.md | 4 + .../CASE_STUDY_MxGraphStudioDemo.md | 4 +- .../PROPOSAL_first_class_expressions.md | 8 +- ...-odata-credential-legacy-spelling.fail.mdl | 21 +++ ...a-client-describe-requotes-expressions.mdl | 17 ++- .../doctype-tests/10-odata-examples.mdl | 20 ++- mdl/executor/cmd_odata.go | 33 +++-- .../cmd_odata_client_describe_quoting_test.go | 5 + mdl/executor/cmd_odata_metadata_auth_test.go | 16 +-- .../odata_client_expression_slots_test.go | 131 ++++++++++++++++++ mdl/executor/validate_odata_properties.go | 73 ++++++++++ mdl/grammar/domains/MDLService.g4 | 10 ++ mdl/visitor/visitor_alter.go | 5 + mdl/visitor/visitor_odata.go | 30 +--- mdl/visitor/visitor_odata_expression.go | 90 ++++++++++++ 16 files changed, 410 insertions(+), 80 deletions(-) create mode 100644 mdl-examples/bug-tests/750-odata-credential-legacy-spelling.fail.mdl create mode 100644 mdl/executor/odata_client_expression_slots_test.go create mode 100644 mdl/visitor/visitor_odata_expression.go diff --git a/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md b/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md index fa9d201496..29d18996b1 100644 --- a/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md +++ b/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md @@ -186,12 +186,11 @@ create odata client ProductClient.ProductDataApiClient ( timeout: 300, ServiceUrl: '@ProductClient.ProductDataApiLocation', UseAuthentication: Yes, - -- HttpUsername/HttpPassword hold a Mendix EXPRESSION. A literal credential - -- is a string literal inside the MDL string, so its quotes are doubled; - -- 'MxAdmin' alone would store the identifier MxAdmin. A constant needs no - -- extra quotes: HttpPassword: @ProductClient.ApiPassword - HttpUsername: '''MxAdmin''', - HttpPassword: '''1''' + -- HttpUsername/HttpPassword hold a Mendix expression, written as-is: + -- 'MxAdmin' is the string, @ProductClient.ApiPassword (no quotes) reads a + -- constant. The old doubled-quote form '''MxAdmin''' is refused (MDL-ODATA07). + HttpUsername: 'MxAdmin', + HttpPassword: '1' ); -- OData client with local file - relative path (offline development) @@ -202,8 +201,8 @@ CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( Timeout: 300, ServiceUrl: '@ProductClient.ProductDataApiLocation', UseAuthentication: Yes, - HttpUsername: '''MxAdmin''', - HttpPassword: '''1''' + HttpUsername: 'MxAdmin', + HttpPassword: '1' ); -- OData client with local file - relative path without ./ @@ -213,8 +212,8 @@ CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( Timeout: 300, ServiceUrl: '@ProductClient.ProductDataApiLocation', UseAuthentication: Yes, - HttpUsername: '''MxAdmin''', - HttpPassword: '''1''' + HttpUsername: 'MxAdmin', + HttpPassword: '1' ); -- OData client with local file - absolute file:// URI @@ -224,8 +223,8 @@ CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( Timeout: 300, ServiceUrl: '@ProductClient.ProductDataApiLocation', UseAuthentication: Yes, - HttpUsername: '''MxAdmin''', - HttpPassword: '''1''' + HttpUsername: 'MxAdmin', + HttpPassword: '1' ); -- External entities (mapped from published service) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9989d6ab7f..a487522972 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +### Changed + +- **An OData client's credentials and header values are written as Mendix expressions** (mendixlabs/mxcli#750) — `HttpUsername`, `HttpPassword`, `ClientCertificate` and every `headers (…)` value hold an expression, and MDL now writes it as-is: `HttpUsername: 'admin'` is the string `'admin'`, `@Module.Const` reads a constant, and `'Bearer ' + @Module.Token` concatenates. Before, a quoted value was the expression's *text*, so `'admin'` stored the identifier `admin` and a string needed `'''admin'''`. `describe` prints the stored expression as-is, so Studio Pro's `'abc'` now reads `HttpUsername: 'abc'`; measured against a Studio Pro-authored client, and a describe → exec round trip stores identical values. **Migrating a script:** `'''admin'''` becomes `'admin'`, and a quoted constant `'@Module.Const'` becomes `@Module.Const` — both old forms still parse but would now store something else, so `check` and `exec` refuse them as **MDL-ODATA07**. A compound expression in any other OData property (`Path: 'a' + 'b'`) is an error rather than an empty value. `ServiceUrl` is unchanged. + ### Fixed - **An expression property written in brackets was silently dropped** (mendixlabs/mxcli#750) — `dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ]`, the spelling #750 proposes, parsed as a list that no writer reads: `check` was clean, `exec` said `Created page`, and the widget was stored with no dynamic class. `alter page … set DynamicClasses = [ … ]` said `Altered page` and changed nothing, and a column's `DynamicCellClass` stored the list's text — tokens fused, `[if$x/Ythen'a'else'b']` — as its expression. Measured on a copy of a Mendix 11.14.0 project with the pre-fix binary. `mxcli check` now reports **MDL-WIDGET32** for `DynamicClasses` and `DynamicCellClass` written as a list (no project needed), and ALTER refuses it, so `check -p` reports that too. Write the expression quoted. diff --git a/docs/01-project/CASE_STUDY_MxGraphStudioDemo.md b/docs/01-project/CASE_STUDY_MxGraphStudioDemo.md index 2ee46b1104..1b75abc8af 100644 --- a/docs/01-project/CASE_STUDY_MxGraphStudioDemo.md +++ b/docs/01-project/CASE_STUDY_MxGraphStudioDemo.md @@ -112,8 +112,8 @@ create odata client OdataPlm.MxPlmOdataApiClient ( timeout: 300, ServiceUrl: '@OdataPlm.MxPlmOdataApiClient_Location', UseAuthentication: Yes, - HttpUsername: '@Main.MxPlmGraphClient_username', - HttpPassword: '@Main.MxPlmGraphClient_password' + HttpUsername: @Main.MxPlmGraphClient_username, + HttpPassword: @Main.MxPlmGraphClient_password ); / ``` diff --git a/docs/11-proposals/PROPOSAL_first_class_expressions.md b/docs/11-proposals/PROPOSAL_first_class_expressions.md index e59375ddb9..dc9a87ec68 100644 --- a/docs/11-proposals/PROPOSAL_first_class_expressions.md +++ b/docs/11-proposals/PROPOSAL_first_class_expressions.md @@ -1,6 +1,6 @@ --- title: First-class expressions for expression-typed MDL properties -status: draft +status: partial date: 2026-09-08 revised: 2026-09-25 related: @@ -241,6 +241,10 @@ proposals compose rather than compete. place this change widens a generic rule, so it wants a maintainer decision. 5. **Flip the meaning of a quoted OData credential/header (§6.4 option a)?** + **Decided 2026-09-25: (a).** `HttpUsername: 'admin'` is the string + 'admin'. Implemented with MDL-ODATA07 refusing both older spellings + (doubled quotes, and a quoted `@Mod.C`) as errors. + *Measured 2026-09-24* — grep of `mdl-examples/`, `.claude/skills/` and `docs-site/src/` for OData `Http*` and `headers` values: @@ -490,7 +494,7 @@ So today the same HTTP header is written `'Accept' = 'application/json'` on a REST client and `'Accept': '''application/json'''` on an OData client. Option (a) removes that inconsistency; (c) keeps it. -Recommendation: **(a)**, scoped to these four OData slots. Unlike +**Decided: (a)** (2026-09-25), scoped to these four OData slots. Unlike `dynamicclasses` — where a plain class name is rare and an `if` is the norm — an OData credential or header is almost always a literal or a constant, so the quoted form *is* the common case and has to read correctly. The trade-off is a diff --git a/mdl-examples/bug-tests/750-odata-credential-legacy-spelling.fail.mdl b/mdl-examples/bug-tests/750-odata-credential-legacy-spelling.fail.mdl new file mode 100644 index 0000000000..d991a63d1e --- /dev/null +++ b/mdl-examples/bug-tests/750-odata-credential-legacy-spelling.fail.mdl @@ -0,0 +1,21 @@ +-- mendixlabs/mxcli#750, PROPOSAL_first_class_expressions.md §6.4 (decided): +-- an OData client's HttpUsername / HttpPassword / ClientCertificate and header +-- values are Mendix expressions written as-is. `HttpUsername: 'admin'` is the +-- string 'admin' — what Studio Pro stores — and `@Module.Const` reads a +-- constant. +-- +-- Two older spellings still parse but would now store something else, so check +-- refuses them as MDL-ODATA07: +-- +-- '''admin''' was the string 'admin'; now a string containing the quotes +-- '@Mod.C' was a constant reference; now the literal text @Mod.C +-- +-- This file must FAIL `mxcli check`. + +create odata client Legacy.Api ( + ODataVersion: OData4, + MetadataUrl: 'https://api.example.com/odata/v4/$metadata', + UseAuthentication: Yes, + HttpUsername: '''admin''', + HttpPassword: '@Legacy.ApiPassword' +); diff --git a/mdl-examples/bug-tests/odata-client-describe-requotes-expressions.mdl b/mdl-examples/bug-tests/odata-client-describe-requotes-expressions.mdl index f7d5352b17..5f2b4a5a28 100644 --- a/mdl-examples/bug-tests/odata-client-describe-requotes-expressions.mdl +++ b/mdl-examples/bug-tests/odata-client-describe-requotes-expressions.mdl @@ -16,8 +16,11 @@ -- -- Fix: always mdlQuote — the inverse of the visitor's unquoteString. -- --- This is the form DESCRIBE now prints for that client. Each value re-stores the --- expression Studio Pro stored: 'abc', @Module.Const, 'Key ' + @Module.Const. +-- Since then these properties became first-class expressions (#750, §6.4 of +-- PROPOSAL_first_class_expressions.md): MDL writes the expression as-is, and +-- DESCRIBE prints it that way. This is the form DESCRIBE now prints for that +-- client. Each value re-stores the expression Studio Pro stored: 'abc', +-- @Module.Const, 'Key ' + @Module.Const. create constant OdQuote.ApiLocation type string @@ -33,12 +36,12 @@ create odata client OdQuote.QuotedApi ( MetadataUrl: 'https://api.example.com/odata/v4/$metadata', ServiceUrl: '@OdQuote.ApiLocation', UseAuthentication: Yes, - HttpUsername: '''abc''', - HttpPassword: '@OdQuote.ApiKey', - ClientCertificate: '''my-cert''' + HttpUsername: 'abc', + HttpPassword: @OdQuote.ApiKey, + ClientCertificate: 'my-cert' ) headers ( - 'X-Api-Key': '''Key '' + @OdQuote.ApiKey', - 'X-O''Key': '''it''''s''' + 'X-Api-Key': 'Key ' + @OdQuote.ApiKey, + 'X-O''Key': 'it''s' ); / diff --git a/mdl-examples/doctype-tests/10-odata-examples.mdl b/mdl-examples/doctype-tests/10-odata-examples.mdl index 0f1c413cd9..4cc5f933da 100644 --- a/mdl-examples/doctype-tests/10-odata-examples.mdl +++ b/mdl-examples/doctype-tests/10-odata-examples.mdl @@ -298,24 +298,22 @@ create odata client OdTest.FullConfigAPI ( timeout: 300, ServiceUrl: @OdTest.FullConfigAPIServiceUrl, UseAuthentication: Yes, - -- HttpUsername/HttpPassword/ClientCertificate are Mendix expression fields: - -- the stored value must be a Mendix expression string, so a string literal - -- needs single quotes inside the MDL string (use doubled '' to escape). - HttpUsername: '''admin''', - HttpPassword: '''secret''', - ClientCertificate: '''my-cert''', + -- HttpUsername/HttpPassword/ClientCertificate hold a Mendix expression, + -- written as-is: 'admin' is the string, @Module.Const reads a constant, + -- and 'Basic ' + @Module.Const concatenates. + HttpUsername: 'admin', + HttpPassword: 'secret', + ClientCertificate: 'my-cert', ErrorHandlingMicroflow: microflow OdTest.HandleError -- ProxyHost / ProxyPort / ProxyUsername / ProxyPassword are by-name -- references to a constant, and need `ProxyType: Override` alongside them — -- the shape Studio Pro stores for a custom proxy. `OdTest.X` and `@OdTest.X` -- both store the bare name; see bug-tests/odata-client-proxy-constant-at-prefix.mdl. ) --- Header values are Mendix expression fields too; wrap literal values in --- single quotes (escaped as doubled '') so the BSON stores a valid --- string-literal expression rather than a bare identifier. +-- Header values are Mendix expressions too, written the same way. headers ( - 'X-Api-Key': '''abc123''', - 'Accept': '''application/json''' + 'X-Api-Key': 'Key ' + @OdTest.FullConfigAPIServiceUrl, + 'Accept': 'application/json' ); / diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index 99517b6bc8..08cada3516 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -160,17 +160,21 @@ func outputConsumedODataServiceMDL(ctx *ExecContext, svc *model.ConsumedODataSer if cfg.OverrideLocation && cfg.CustomLocation != "" { props = append(props, fmt.Sprintf(" ServiceUrl: %s", formatExprValue(cfg.CustomLocation))) } + // HttpUsername / HttpPassword / ClientCertificate and header values are + // Mendix expressions, and MDL writes an expression as-is: the stored + // `'abc'` prints as `'abc'`, the string (PROPOSAL_first_class_expressions.md + // §6.4). ServiceUrl above still takes the older quoted form. if cfg.UseAuthentication { props = append(props, " UseAuthentication: Yes") if cfg.Username != "" { - props = append(props, fmt.Sprintf(" HttpUsername: %s", formatExprValue(cfg.Username))) + props = append(props, fmt.Sprintf(" HttpUsername: %s", cfg.Username)) } if cfg.Password != "" { - props = append(props, fmt.Sprintf(" HttpPassword: %s", formatExprValue(cfg.Password))) + props = append(props, fmt.Sprintf(" HttpPassword: %s", cfg.Password)) } } if cfg.ClientCertificate != "" { - props = append(props, fmt.Sprintf(" ClientCertificate: %s", formatExprValue(cfg.ClientCertificate))) + props = append(props, fmt.Sprintf(" ClientCertificate: %s", cfg.ClientCertificate)) } } @@ -212,7 +216,7 @@ func outputConsumedODataServiceMDL(ctx *ExecContext, svc *model.ConsumedODataSer if i == len(cfg.HeaderEntries)-1 { comma = "" } - fmt.Fprintf(ctx.Output, " %s: %s%s\n", mdlQuote(h.Key), formatExprValue(h.Value), comma) + fmt.Fprintf(ctx.Output, " %s: %s%s\n", mdlQuote(h.Key), h.Value, comma) } fmt.Fprintln(ctx.Output, ");") } else { @@ -2100,18 +2104,19 @@ func metadataAuthFromStmt(ctx *ExecContext, stmt *ast.CreateODataClientStmt) *me // resolveCredential turns an MDL property value into the string to send on the // design-time fetch. // -// Three spellings reach here and all three have to work, because the shape MDL -// pushes users towards is the constant reference — mxcli requires a constant for -// ServiceUrl, so a client written the documented way has constants for its -// credentials too (mxcli-formula1 #23 follow-up): +// The value is the Mendix expression the property holds, as written — these +// properties are first-class expressions (PROPOSAL_first_class_expressions.md +// §6.4): // -// HttpUsername: 'f1api' a literal -// HttpUsername: @Module.ApiUser a constant reference -// HttpUsername: '@Module.ApiUser' the same reference, quoted +// HttpUsername: 'f1api' a string: sends f1api +// HttpUsername: @Module.ApiUser a constant: sends its design-time default +// HttpUsername: 'Key ' + @M.C compound: cannot be evaluated, reported unresolved // -// The quoted form is the trap: it is a STRING_LITERAL, so the isLiteral flag says -// "literal" and the naive reading sends the eleven characters `@Module.ApiUser` -// as the username. Worse than a 401, because it looks like it tried. +// The constant reference matters because mxcli requires one for ServiceUrl, so a +// client written the documented way has constants for its credentials too +// (mxcli-formula1 #23 follow-up). The old quoted form `'@Module.ApiUser'` is now +// the literal text and is refused at check time (MDL-ODATA07); the branches +// below still resolve the bare-value inputs older statements produced. // // A constant's design-time default is exactly what Studio Pro uses for the same // fetch, so resolving it here is not a workaround — it is the value. diff --git a/mdl/executor/cmd_odata_client_describe_quoting_test.go b/mdl/executor/cmd_odata_client_describe_quoting_test.go index 5fdab30e06..d784de10e4 100644 --- a/mdl/executor/cmd_odata_client_describe_quoting_test.go +++ b/mdl/executor/cmd_odata_client_describe_quoting_test.go @@ -19,6 +19,11 @@ import ( // an identifier, not a string. Measured on a Studio Pro-authored client // (ako/TestApp@37e0cc0, Odata.Bug1073). ClientCertificate, header keys and the // plain string properties were printed as a raw '%s', unescaped. +// +// Since these properties became first-class expressions (§6.4 of +// PROPOSAL_first_class_expressions.md) describe prints the stored expression +// as-is and the visitor stores it as written, so `'abc'` round-trips as `'abc'`. +// The contract these tests hold — exec(describe(x)) stores x — is unchanged. // describeAndReparse runs DESCRIBE on stored and parses the output with the real // visitor, returning what a re-exec would store. diff --git a/mdl/executor/cmd_odata_metadata_auth_test.go b/mdl/executor/cmd_odata_metadata_auth_test.go index c1ebebcafc..3bce80fc95 100644 --- a/mdl/executor/cmd_odata_metadata_auth_test.go +++ b/mdl/executor/cmd_odata_metadata_auth_test.go @@ -133,7 +133,7 @@ func TestResolveCredential(t *testing.T) { {"a dotted literal stays a literal", "s3.cret", true, "s3.cret", true}, // The value is a Mendix EXPRESSION. Studio Pro stores a literal // credential as the string literal `'MxAdmin'`, quotes included, and so - // does MDL's `HttpUsername: '''MxAdmin'''`. The fetch must send its + // does MDL's `HttpUsername: 'MxAdmin'`. The fetch must send its // content, not the quotes — sending `'MxAdmin'` is a 401 against the // very service the odata-data-sharing walkthrough imports from. {"a string-literal expression sends its content", "'MxAdmin'", true, "MxAdmin", true}, @@ -157,18 +157,18 @@ func TestResolveCredential(t *testing.T) { } } -// The spelling the odata-data-sharing skill now teaches, end to end: parse the -// MDL, then build the credentials the design-time fetch will send. The stored -// value must be the expression `'MxAdmin'` (what Studio Pro stores) and the -// fetch must send `MxAdmin` — before the fix, fixing the skill traded a runtime -// defect for a 401 at design time. +// The spelling the odata-data-sharing skill teaches, end to end: parse the MDL, +// then build the credentials the design-time fetch will send. The stored value +// must be the expression `'MxAdmin'` (what Studio Pro stores) and the fetch must +// send `MxAdmin`. Since these properties became first-class expressions the MDL +// is `'MxAdmin'` itself; before, it was the doubled-quote form. func TestMetadataAuth_StringLiteralCredentialFromMDL(t *testing.T) { prog := parseMDL(t, `create odata client M.Api ( ODataVersion: OData4, MetadataUrl: 'http://localhost:8080/odata/api/v1/$metadata', UseAuthentication: Yes, - HttpUsername: '''MxAdmin''', - HttpPassword: '''1''' + HttpUsername: 'MxAdmin', + HttpPassword: '1' );`) stmt := prog.Statements[0].(*ast.CreateODataClientStmt) if stmt.HttpUsername != "'MxAdmin'" { diff --git a/mdl/executor/odata_client_expression_slots_test.go b/mdl/executor/odata_client_expression_slots_test.go new file mode 100644 index 0000000000..5a654f7572 --- /dev/null +++ b/mdl/executor/odata_client_expression_slots_test.go @@ -0,0 +1,131 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// PROPOSAL_first_class_expressions.md §6.4, decided: HttpUsername, HttpPassword, +// ClientCertificate and header values hold a Mendix EXPRESSION, and MDL writes +// that expression as-is. `HttpUsername: 'admin'` is the string 'admin' — the +// value Studio Pro stores, quotes included — where it used to store the +// identifier `admin` and a string needed `'''admin'''`. + +func odataClientFrom(t *testing.T, src string) *ast.CreateODataClientStmt { + t.Helper() + for _, s := range parseMDL(t, src).Statements { + if c, ok := s.(*ast.CreateODataClientStmt); ok { + return c + } + } + t.Fatalf("no create odata client statement in %q", src) + return nil +} + +func TestODataClientExpressionSlots_StoreTheExpressionAsWritten(t *testing.T) { + stmt := odataClientFrom(t, `create odata client M.Api ( + ODataVersion: OData4, + MetadataUrl: 'https://example.com/$metadata', + UseAuthentication: Yes, + HttpUsername: 'admin', + HttpPassword: @M.ApiPassword, + ClientCertificate: 'it''s' +) +headers ( + 'X-Api-Key': 'Key ' + @M.ApiKey, + 'Accept': 'application/json' +);`) + + for _, c := range []struct{ field, got, want string }{ + {"HttpUsername", stmt.HttpUsername, "'admin'"}, + {"HttpPassword", stmt.HttpPassword, "@M.ApiPassword"}, + {"ClientCertificate", stmt.ClientCertificate, "'it''s'"}, + } { + if c.got != c.want { + t.Errorf("%s stored %q, want the expression %q", c.field, c.got, c.want) + } + } + want := map[string]string{"X-Api-Key": "'Key ' + @M.ApiKey", "Accept": "'application/json'"} + for _, h := range stmt.Headers { + if h.Value != want[h.Key] { + t.Errorf("header %s stored %q, want %q", h.Key, h.Value, want[h.Key]) + } + } + // Control: a plain-value property keeps its unquoted value. + if stmt.MetadataUrl != "https://example.com/$metadata" { + t.Errorf("MetadataUrl = %q, want the unquoted URL", stmt.MetadataUrl) + } +} + +func TestODataClientExpressionSlots_AlterStoresTheExpression(t *testing.T) { + prog := parseMDL(t, `alter odata client M.Api set HttpUsername = 'admin', HttpPassword = 'a' + @M.C;`) + stmt := prog.Statements[0].(*ast.AlterODataClientStmt) + if got := stmt.Changes["HttpUsername"]; got != "'admin'" { + t.Errorf("HttpUsername set to %q, want %q", got, "'admin'") + } + if got := stmt.Changes["HttpPassword"]; got != "'a' + @M.C" { + t.Errorf("HttpPassword set to %q, want %q", got, "'a' + @M.C") + } +} + +// A compound expression in a property that takes a plain value must be refused, +// not read as an empty value: widening the value rule for the four slots must +// not open a silent drop everywhere else. +func TestODataExpressionInAPlainProperty_IsAnError(t *testing.T) { + for _, src := range []string{ + `create odata client M.Api (ODataVersion: OData4, MetadataUrl: 'https://x/' + '$metadata');`, + `create odata service M.S (Path: 'odata/' + 'v1', Version: '1.0') {};`, + `alter odata client M.Api set MetadataUrl = 'a' + 'b';`, + } { + _, errs := visitor.Build(src) + if len(errs) == 0 { + t.Errorf("accepted an expression in a plain-value property: %s", src) + continue + } + if !strings.Contains(errs[0].Error(), "expression") { + t.Errorf("error %q should say the property does not take an expression", errs[0]) + } + } +} + +// MDL-ODATA07: the two spellings whose meaning changed. Both stay parseable, and +// both would now silently store something else, so check refuses them and names +// the new spelling. +func TestMDLODATA07_LegacyCredentialSpelling(t *testing.T) { + cases := []struct { + name, value string + want int + }{ + {"doubled quotes, the old string spelling", `'''admin'''`, 1}, + {"quoted constant reference", `'@M.ApiUser'`, 1}, + {"control: a string", `'admin'`, 0}, + {"control: a constant reference", `@M.ApiUser`, 0}, + {"control: a compound expression", `'Bearer ' + @M.Token`, 0}, + // A literal that merely starts with @ is a legitimate string. + {"control: an @ that is not a qualified name", `'@home'`, 0}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + prog := parseMDL(t, `create odata client M.Api ( + ODataVersion: OData4, MetadataUrl: 'https://x/$metadata', + UseAuthentication: Yes, HttpUsername: `+tc.value+` +) +headers ('X-Token': `+tc.value+`);`) + var got []string + for _, v := range ValidateODataProperties(prog) { + if v.RuleID == "MDL-ODATA07" { + got = append(got, v.Message) + } + } + // One for HttpUsername, one for the header. + if len(got) != 2*tc.want { + t.Fatalf("MDL-ODATA07: got %d, want %d: %v", len(got), 2*tc.want, got) + } + }) + } +} diff --git a/mdl/executor/validate_odata_properties.go b/mdl/executor/validate_odata_properties.go index ad31e060c9..069f86aa82 100644 --- a/mdl/executor/validate_odata_properties.go +++ b/mdl/executor/validate_odata_properties.go @@ -12,6 +12,8 @@ package executor import ( "fmt" + "regexp" + "sort" "strings" "github.com/mendixlabs/mxcli/mdl/ast" @@ -71,6 +73,25 @@ func ValidateODataProperties(prog *ast.Program) []linter.Violation { case *ast.CreateODataClientStmt: out = append(out, unknownODataProps( "odata client "+s.Name.String(), s.UnknownProperties, knownODataClientProps)...) + loc := "odata client " + s.Name.String() + out = append(out, legacyODataExpression(loc, "HttpUsername", s.HttpUsername)...) + out = append(out, legacyODataExpression(loc, "HttpPassword", s.HttpPassword)...) + out = append(out, legacyODataExpression(loc, "ClientCertificate", s.ClientCertificate)...) + for _, h := range s.Headers { + out = append(out, legacyODataExpression(loc, "header "+h.Key, h.Value)...) + } + case *ast.AlterODataClientStmt: + loc := "alter odata client " + s.Name.String() + names := make([]string, 0, len(s.Changes)) + for name := range s.Changes { + names = append(names, name) + } + sort.Strings(names) // map order would make two runs report differently + for _, name := range names { + if str, ok := s.Changes[name].(string); ok && isODataClientExpressionName(name) { + out = append(out, legacyODataExpression(loc, name, str)...) + } + } case *ast.CreateExternalEntityStmt: out = append(out, unknownODataProps( "external entity "+s.Name.String(), s.UnknownProperties, knownExternalEntityProps)...) @@ -146,3 +167,55 @@ func withinOneEdit(a, b string) bool { } return true } + +// quotedConstantRef matches the text of a string literal that is really a +// constant reference: `@Module.Name`. +var quotedConstantRef = regexp.MustCompile(`^@[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)+$`) + +func isODataClientExpressionName(name string) bool { + switch strings.ToLower(name) { + case "httpusername", "httppassword", "clientcertificate": + return true + } + return false +} + +// legacyODataExpression (MDL-ODATA07) reports the two spellings whose meaning +// changed when HttpUsername / HttpPassword / ClientCertificate / header values +// became first-class expressions (PROPOSAL_first_class_expressions.md §6.4): +// +// '''admin''' was the string 'admin'; now a string that CONTAINS the quotes +// '@Mod.C' was a constant reference; now the literal text @Mod.C +// +// Both still parse, and both would now store something else without a word — +// a credential with stray quote characters, or a constant's name sent as the +// password. So they are errors that name the new spelling. The false positive +// is a real credential that begins and ends with a quote, or is shaped exactly +// like a qualified name after an @; the message says how to write either. +func legacyODataExpression(location, prop, expr string) []linter.Violation { + content, isLiteral := mendixStringLiteral(expr) + if !isLiteral { + return nil + } + var msg, fix string + switch { + case len(content) >= 2 && strings.HasPrefix(content, "'") && strings.HasSuffix(content, "'"): + msg = fmt.Sprintf("%s: %s is written %s — the doubled quotes are the old spelling of the string %s, "+ + "and now store the quote characters as part of the value", location, prop, expr, content) + fix = fmt.Sprintf("Write %s: %s. A value that really does begin and end with a quote character "+ + "is written as a concatenation, which this check does not flag: %s", + prop, content, "'''' + '"+strings.Trim(content, "'")+"' + ''''") + case quotedConstantRef.MatchString(content): + msg = fmt.Sprintf("%s: %s is written %s — a quoted @-name used to mean the constant %s, "+ + "and now stores the literal text %s", location, prop, expr, content[1:], content) + fix = fmt.Sprintf("Write %s: %s (no quotes) to read the constant.", prop, content) + default: + return nil + } + return []linter.Violation{{ + RuleID: "MDL-ODATA07", + Severity: linter.SeverityError, + Message: msg, + Suggestion: fix, + }} +} diff --git a/mdl/grammar/domains/MDLService.g4 b/mdl/grammar/domains/MDLService.g4 index ddf32c3654..0ceb150bd6 100644 --- a/mdl/grammar/domains/MDLService.g4 +++ b/mdl/grammar/domains/MDLService.g4 @@ -159,12 +159,21 @@ odataPropertyValue | qualifiedName ; +// A Mendix expression is accepted after the plain value forms, so `'admin'`, +// `@Mod.Const`, `microflow Mod.F` and `OData4` keep their parse and only what +// those reject (`'Bearer ' + @Mod.Token`) reaches it. The visitor stores the +// source text for the expression-typed properties (HttpUsername, HttpPassword, +// ClientCertificate, header values) and refuses an expression anywhere else, +// so a plain-value property cannot silently read it as empty +// (PROPOSAL_first_class_expressions.md §6.4). odataPropertyAssignment : identifierOrKeyword COLON odataPropertyValue + | identifierOrKeyword COLON expression ; odataAlterAssignment : identifierOrKeyword EQUALS odataPropertyValue + | identifierOrKeyword EQUALS expression ; odataAuthenticationClause @@ -234,6 +243,7 @@ odataHeadersClause odataHeaderEntry : STRING_LITERAL COLON odataPropertyValue + | STRING_LITERAL COLON expression ; // ============================================================================= diff --git a/mdl/visitor/visitor_alter.go b/mdl/visitor/visitor_alter.go index 9003b792cb..e69ffc2ea8 100644 --- a/mdl/visitor/visitor_alter.go +++ b/mdl/visitor/visitor_alter.go @@ -66,6 +66,11 @@ func (b *Builder) ExitAlterStatement(ctx *parser.AlterStatementContext) { for _, propCtx := range ctx.AllOdataAlterAssignment() { prop := propCtx.(*parser.OdataAlterAssignmentContext) name := identifierOrKeywordText(prop.IdentifierOrKeyword()) + if ctx.CLIENT() != nil && isODataClientExpressionProp(name) { + // Expression-typed: the expression as written (see visitor_odata_expression.go). + changes[name], _ = odataExpressionValue(prop.OdataPropertyValue(), prop.Expression()) + continue + } val := prop.OdataPropertyValue() if val != nil { changes[name] = odataValueText(val.(*parser.OdataPropertyValueContext)) diff --git a/mdl/visitor/visitor_odata.go b/mdl/visitor/visitor_odata.go index 3a1d23f401..6f3a9561f6 100644 --- a/mdl/visitor/visitor_odata.go +++ b/mdl/visitor/visitor_odata.go @@ -43,14 +43,13 @@ func (b *Builder) ExitCreateODataClientStatement(ctx *parser.CreateODataClientSt stmt.ServiceUrl = value case "useauthentication": stmt.UseAuthentication = strings.EqualFold(value, "true") || strings.EqualFold(value, "yes") + // Expression-typed: the expression as written, not an unquoted value. case "httpusername": - stmt.HttpUsername = value - stmt.HttpUsernameIsLiteral = odataValueIsLiteral(prop) + stmt.HttpUsername, stmt.HttpUsernameIsLiteral = odataExpressionValue(prop.OdataPropertyValue(), prop.Expression()) case "httppassword": - stmt.HttpPassword = value - stmt.HttpPasswordIsLiteral = odataValueIsLiteral(prop) + stmt.HttpPassword, stmt.HttpPasswordIsLiteral = odataExpressionValue(prop.OdataPropertyValue(), prop.Expression()) case "clientcertificate": - stmt.ClientCertificate = value + stmt.ClientCertificate, _ = odataExpressionValue(prop.OdataPropertyValue(), prop.Expression()) case "configurationmicroflow": // "Configuration microflow" — returns System.ConsumedODataConfiguration. stmt.ConfigurationMicroflow = value @@ -308,18 +307,6 @@ func odataValueText(val *parser.OdataPropertyValueContext) string { return "" } -// odataValueIsLiteral reports whether an OData property value was written as a -// quoted string rather than a constant reference. odataValueText strips a -// literal's quotes, so this is the only thing that still tells the two apart — -// and mxcli can only use a literal for the design-time $metadata fetch. -func odataValueIsLiteral(prop *parser.OdataPropertyAssignmentContext) bool { - valCtx := prop.OdataPropertyValue() - if valCtx == nil { - return false - } - return valCtx.(*parser.OdataPropertyValueContext).STRING_LITERAL() != nil -} - // odataAssignmentValueText extracts the string value from an OData property assignment. func odataAssignmentValueText(prop *parser.OdataPropertyAssignmentContext) string { valCtx := prop.OdataPropertyValue() @@ -421,13 +408,8 @@ func parseODataHeaders(ctx parser.IOdataHeadersClauseContext) []ast.HeaderDef { for _, entryCtx := range clause.AllOdataHeaderEntry() { entry := entryCtx.(*parser.OdataHeaderEntryContext) key := unquoteString(entry.STRING_LITERAL().GetText()) - value := "" - isLiteral := false - if valCtx := entry.OdataPropertyValue(); valCtx != nil { - vc := valCtx.(*parser.OdataPropertyValueContext) - value = odataValueText(vc) - isLiteral = vc.STRING_LITERAL() != nil - } + // A header value is a Mendix expression, kept as written. + value, isLiteral := odataExpressionValue(entry.OdataPropertyValue(), entry.Expression()) headers = append(headers, ast.HeaderDef{Key: key, Value: value, ValueIsLiteral: isLiteral}) } diff --git a/mdl/visitor/visitor_odata_expression.go b/mdl/visitor/visitor_odata_expression.go new file mode 100644 index 0000000000..51c41411ba --- /dev/null +++ b/mdl/visitor/visitor_odata_expression.go @@ -0,0 +1,90 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "fmt" + "strings" + + "github.com/antlr4-go/antlr/v4" + + "github.com/mendixlabs/mxcli/mdl/grammar/parser" +) + +// The consumed OData client's expression-typed properties. Each holds ONE Mendix +// expression, which MDL writes as-is: `HttpUsername: 'admin'` is the string +// 'admin' (the value Studio Pro stores, quotes included), `@Mod.Const` reads a +// constant, and `'Bearer ' + @Mod.Token` concatenates. Header values are the +// same kind. PROPOSAL_first_class_expressions.md §6.4. +var odataClientExpressionProps = map[string]bool{ + "httpusername": true, + "httppassword": true, + "clientcertificate": true, +} + +func isODataClientExpressionProp(name string) bool { + return odataClientExpressionProps[strings.ToLower(name)] +} + +// ruleSourceText is the author's text for a rule, whitespace included — the +// characters between its first and last token in the input. GetText() would +// concatenate the tokens without the whitespace between them, turning +// `'a' + @M.C` into `'a'+@M.C` and `if $x then` into `if$xthen`. +func ruleSourceText(ctx antlr.ParserRuleContext) string { + if ctx == nil { + return "" + } + start, stop := ctx.GetStart(), ctx.GetStop() + if start == nil || stop == nil || stop.GetStop() < start.GetStart() { + return ctx.GetText() + } + return start.GetInputStream().GetText(start.GetStart(), stop.GetStop()) +} + +// odataExpressionValue returns the expression an OData client expression +// property holds, exactly as written, whichever grammar alternative matched it, +// and whether it is a single string literal. +func odataExpressionValue(valueCtx parser.IOdataPropertyValueContext, exprCtx parser.IExpressionContext) (string, bool) { + if valueCtx != nil { + vc := valueCtx.(*parser.OdataPropertyValueContext) + return ruleSourceText(vc), vc.STRING_LITERAL() != nil + } + if exprCtx != nil { + return ruleSourceText(exprCtx.(antlr.ParserRuleContext)), false + } + return "", false +} + +// ExitOdataPropertyAssignment refuses an expression where only a plain value is +// read. The grammar admits `name: ` for every OData property list +// because it cannot tell the names apart; without this, `Path: 'a' + 'b'` would +// reach a visitor that reads only the plain alternatives and store nothing. +func (b *Builder) ExitOdataPropertyAssignment(ctx *parser.OdataPropertyAssignmentContext) { + if ctx.Expression() == nil { + return + } + name := identifierOrKeywordText(ctx.IdentifierOrKeyword()) + if _, onClient := ctx.GetParent().(*parser.CreateODataClientStatementContext); onClient && isODataClientExpressionProp(name) { + return + } + b.addError(odataExpressionNotAllowed(name, ctx.Expression())) +} + +// ExitOdataAlterAssignment is the ALTER twin of ExitOdataPropertyAssignment. +func (b *Builder) ExitOdataAlterAssignment(ctx *parser.OdataAlterAssignmentContext) { + if ctx.Expression() == nil { + return + } + name := identifierOrKeywordText(ctx.IdentifierOrKeyword()) + if alter, ok := ctx.GetParent().(*parser.AlterStatementContext); ok && alter.CLIENT() != nil && isODataClientExpressionProp(name) { + return + } + b.addError(odataExpressionNotAllowed(name, ctx.Expression())) +} + +func odataExpressionNotAllowed(name string, expr parser.IExpressionContext) error { + return fmt.Errorf( + "property %s takes a plain value, not an expression: %s — "+ + "only an OData client's HttpUsername, HttpPassword, ClientCertificate and header values take an expression", + name, ruleSourceText(expr.(antlr.ParserRuleContext))) +} From 9c764e5a901595de9b45765581a19b1cc4fbcf3a Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 06:36:17 +0000 Subject: [PATCH 23/47] fix(codec): write a compound design property's Properties list with marker 2 Studio Pro stores Forms$CompoundDesignPropertyValue.Properties with BSON array marker 2 (373 of 373 across the pages, layouts, building blocks and page templates of a Mendix 11.13.0 app) and the enclosing Forms$Appearance.DesignProperties with marker 3 (1821 of 1821). mxcli wrote 3 for both, so a describe -> exec round trip of a Studio Pro page changed every nested list's marker. The codec chose a PartList's marker from the child element's $Type alone, and both lists hold Forms$DesignPropertyValue. Add codec.RegisterPropertyListMarker, keyed on the owning $Type and key and consulted before the child-type marker, and register Forms$CompoundDesignPropertyValue/Properties as 2. Pages, snippets and layouts share newAppearance, so all three are covered. Co-Authored-By: Claude Opus 5.5 --- .../skills/fix-issue/findings/modelsdk.jsonl | 1 + .../compound-design-property-marker.mdl | 38 ++++++++++++++ mdl/backend/modelsdk/design_property_test.go | 51 +++++++++++++++++++ mdl/backend/modelsdk/widget_write.go | 6 +++ modelsdk/codec/defaults.go | 24 +++++++++ modelsdk/codec/encoder.go | 26 +++++++--- 6 files changed, 140 insertions(+), 6 deletions(-) create mode 100644 mdl-examples/bug-tests/compound-design-property-marker.mdl diff --git a/.claude/skills/fix-issue/findings/modelsdk.jsonl b/.claude/skills/fix-issue/findings/modelsdk.jsonl index e58615d476..29e35108d3 100644 --- a/.claude/skills/fix-issue/findings/modelsdk.jsonl +++ b/.claude/skills/fix-issue/findings/modelsdk.jsonl @@ -20,3 +20,4 @@ {"area": "docs / modelsdk/mpr", "date": "2026-09-22", "symptom": "Four reference pages described an MPR v1 `UnitContents` table holding the BSON blobs, and a v1/v2 detection recipe that probes for it. No .mpr has ever had that table: a v1 file has exactly `Unit` and `_MetaData`, and contents are the `Unit.Contents` blob. `grep -rn UnitContents --include=*.go` is 0 hits. Reported by an outside reader building an independent format reader (mendixlabs/mxcli#1072).", "cause": "Never-measured prose. The pages also invented `UnitType` and `Name` columns on `Unit` (there are seven columns and neither is among them — type and name come out of the BSON `$Type`/`Name`), and drew `mprcontents/` flat when it is sharded `//.mxunit`. Fixed by rewriting the four pages from the SQLite catalogs of two real fixtures, and adding modelsdk/mpr/docs_schema_test.go to hold them there.", "file": "`docs-site/src/internals/mpr-format.md`, `docs-site/src/internals/mpr-v1-v2.md`, `docs-site/src/appendixes/version-compatibility.md`, `docs/05-mdl-specification/10-bson-mapping.md`; test `modelsdk/mpr/docs_schema_test.go`", "insight": "Prose cannot be type-checked but the IDENTIFIERS in it can, and the rule that makes it zero-maintenance is a prefix rule, not an allowlist: check only names BEGINNING with a real table name (`Unit`, `_MetaData`, `_Transaction`) against the union of the fixtures' tables and columns. `UnitContents` and `UnitType` are caught; the catalog tables these same pages mention (`REFS` and friends) never start with a real .mpr table name, so they need no exemption and no one has to maintain a list. The page set is discovered by content (any .md under docs-site/src or docs/05-mdl-specification mentioning `.mpr`/`mprcontents`), so a page added later is covered without anyone remembering. One consequence worth stating in the docs themselves: a page that wants to say a column does NOT exist must say it in PROSE — the first fix wrote \"there is no `UnitType` column\" and the test flagged its own remedy, which is correct, because the old pages' \"no `UnitContents`\" at mpr-v1-v2.md:35 read as a v1/v2 difference rather than as a fiction and an exemption for denials would have masked it. The control is cheap and exact here: `git stash` the doc edits with the test file kept, and the failures reproduce the reporter's line list verbatim (version-compatibility.md:31, mpr-format.md:21,23, mpr-v1-v2.md:12,35,69,73,74,84,94, 10-bson-mapping.md:30) plus the two they had not found. No Mendix tool runs in this fix's argument — the claims are about SQLite schema and are read straight off `sqlite_master`/`PRAGMA table_info`, which is the primary source, so the usual 'build two apps' rule does not apply.", "refs": ["mendixlabs/mxcli#1072"]} {"area": "docs / modelsdk/mpr", "date": "2026-09-22", "symptom": "The MPR reference pages' \"Unit Types\" tables mapped BSON `$Type` to document kinds, and 15 of the rows named a spelling no unit carries: `Pages$Page`/`Pages$Layout`/`Pages$Snippet`/`Pages$BuildingBlock` (real units say `Forms$*`), and docs/05-mdl-specification/10-bson-mapping.md lowercased eleven more (`microflows$microflow`, `pages$page`, `security$ProjectSecurity`…). It also listed `CustomWidgets$customwidget` as a document type. Found while fixing mendixlabs/mxcli#1072, filed and fixed separately.", "cause": "The tables were written from the TypeScript SDK's QUALIFIED names rather than the storage names Mendix writes — the same split CLAUDE.md documents for `ShowPageAction`/`ShowFormAction`, never applied here. `CustomWidgets$CustomWidget` is a widget element inside a page's tree (mdl/catalog/builder_widget_refs.go), never a unit, so that row was removed rather than corrected.", "file": "`docs-site/src/internals/mpr-format.md`, `docs/05-mdl-specification/10-bson-mapping.md`; test `modelsdk/mpr/docs_schema_test.go` (TestDocumentedUnitTypesUseStorageNames)", "insight": "Measuring the real set is one command and settles the whole table at once: decode every `mprcontents/*/*/*.mxunit` (and every v1 `Unit.Contents` blob) and count `$Type` — 28 distinct values across a blank 11.6.6 app and a 9.24.30 one. Do NOT try to verify rows one at a time against gen, which carries BOTH spellings: `model/types.go` defines `DocumentTypePage = \"Pages$Page\"` and mdl/catalog/builder_xpath.go defensively matches `Forms$Page` AND `Pages$Page`, so grepping the codebase 'confirms' the wrong name. The fixture is the arbiter; the codebase is not. The test rule that makes this checkable without a maintenance burden keys on the LOCAL name after the `$`, case-insensitively: a fixture cannot prove a type ABSENT (a blank project has no business-event service), so demanding every documented type be present would fail correct rows — but when the fixture has a type with the same local name, the documented row must equal it exactly. That catches all four `Pages$` rows and all eleven lowercase ones with zero false positives. Its stated limit is real and cost a manual fix: a row whose local name appears nowhere in the fixtures is not checked at all, which is how `CustomWidgets$customwidget` slipped past and had to be removed by hand. One editing trap, not a Mendix one: anchoring a section replacement on `'---'` matches a markdown TABLE SEPARATOR (`|---|---|`) long before the horizontal rule you meant — the edit silently no-ops on the table you were replacing. Anchor on `'\\n---\\n'`.", "refs": ["mendixlabs/mxcli#1072"]} {"area": "modelsdk/canon", "date": "2026-09-23", "symptom": "The storage-GUID write guard (`canon.StorageGUIDChanges`) stopped refusing the MOVE ENTITY data loss it had exposed (ako/mxcli#503). With MoveEntity's carries removed, moving an association's TO side re-minted the in-place converted cross-association's GUID and the write went through silently, where the issue records a refusal.", "cause": "`sameMember` (added in 86927852 to stop the guard refusing transplant mis-pairings) required an equal `$Type` as well as an equal `Name`. MoveEntity converts `DomainModels$Association` to `DomainModels$CrossAssociation` IN PLACE, keeping `$ID` and `Name`, so the type clause made the guard skip the pair. The type clause excluded nothing the transplant can produce: `pairDoc` stops at a `$Type` mismatch (TestTransplantIgnoresMismatchedTypes).", "file": "`modelsdk/canon/storageguid.go` (`sameMember`, the note above `GUIDChange`)", "insight": "Before adding a clause to an identity test that sits on an approximate pairing, ask what error of THAT pairing the clause excludes. The transplant only mis-pairs same-type, different-name elements, so `$Type` excluded none of its errors. Its only effect was to exclude the one writer that keeps an `$ID` across a type change deliberately. Rule now: Name when both sides have one; `$Type` only when neither does; a pair with a name on one side only is not a match. How the gap was found: stub the three MoveEntity carries on main and run TestIssue503. The child-side case returned no error where the issue quotes a refusal. That mismatch between the recorded refusal and the observed silence was the tell. A guard's quiet is not evidence of a clean write, so a guard's comment must list every hole it leaves; this one listed only renames. Controls: (1) the new canon test fails on the old `sameMember` with `got 0 change(s)`; (2) with the MoveEntity carries stubbed the child-side move is refused again with the issue's exact message, and the parent-side move still goes through, because the moved element changes unit and pairs with nothing (a documented hole); (3) the 86927852 false positive does not return: `marketplace install --file mx-modules/BusinessEvents_3.12.0.mpk` into a copy of testdata/expr-checker, then `create or modify persistent entity BusinessEvents.PublishedBusinessEvent (EventId: long)` is accepted, while a build with an `$ID`-only rule refuses it (EventId paired with a removed attribute). The existing table case `DifferentType_NotAChange` pinned the wrong decision with the justification 'nothing authors this today', which was false the day it was written. Grep for the writers (`SetID(x.ID())` next to `New()`) before claiming nothing authors a shape.", "refs": ["ako/mxcli#503", "mendixlabs/mxcli#1119"]} +{"area":"modelsdk/codec","date":"2026-09-25","symptom":"A compound design property (Atlas `Spacing` → `margin-bottom`, or a multiSelect toggle group) writes its `Forms$CompoundDesignPropertyValue.Properties` list with BSON array marker 3 where Studio Pro writes 2. `check`, `exec` and `mx check` all pass; a describe → exec round trip of FeedbackModule.ShareFeedback (Feedback v4.0.2, 11.13.0) turned every nested marker-2 list into 3","cause":"The codec picks a PartList's marker from the CHILD element's `$Type` only (`partListMarker` → `lookupListMarker`). The nested list and the enclosing `Forms$Appearance.DesignProperties` list (marker 3) both hold `Forms$DesignPropertyValue`, so no `RegisterListMarker` on the child type could tell them apart and both fell to the default 3","file":"`modelsdk/codec/defaults.go` (`RegisterPropertyListMarker`), `modelsdk/codec/encoder.go` (`propertyListMarker`), `mdl/backend/modelsdk/widget_write.go` (init)","insight":"When one child `$Type` sits in two lists with different markers, the marker belongs to the owner+key, not the child: `RegisterPropertyListMarker(owner, key, m)` is consulted first, for an empty list too and in the selective-rebuild path. Establish the marker by counting Studio Pro-authored BSON before changing anything: walking every mxunit gave Compound.Properties 373/373 marker 2 and Appearance.DesignProperties 1821/1821 marker 3 across pages, layouts, building blocks and page templates. Count per (owner $Type, key) — a flat grep of `Properties [marker=2]` in ndsl also matches unrelated lists. Pages, snippets and layouts share `newAppearance`, so one registration covers all. Test `TestAppearanceCompoundDesignPropertyMarkers`; bug-test `mdl-examples/bug-tests/compound-design-property-marker.mdl`. Same class, not fixed: the selective-rebuild branch of `encodeEntry` still hard-codes 3 for lists with no owner registration, ignoring a child-type `RegisterListMarker`","refs":["#668"]} diff --git a/mdl-examples/bug-tests/compound-design-property-marker.mdl b/mdl-examples/bug-tests/compound-design-property-marker.mdl new file mode 100644 index 0000000000..79d35eb1bc --- /dev/null +++ b/mdl-examples/bug-tests/compound-design-property-marker.mdl @@ -0,0 +1,38 @@ +-- @version: 10.0+ +-- ============================================================================ +-- Compound design property's nested Properties list written with marker 3. +-- +-- Symptom: a compound (nested) design property such as Atlas 'Spacing' → +-- margin-bottom serialized its Forms$CompoundDesignPropertyValue.Properties +-- list with BSON typed-array marker 3. Studio Pro writes 2 — measured 373 of +-- 373 such lists across the pages, layouts, building blocks and page templates +-- of a Mendix 11.13.0 app, against 1821 of 1821 marker 3 for the enclosing +-- Forms$Appearance.DesignProperties list. A describe → exec round trip of +-- FeedbackModule.ShareFeedback (Feedback v4.0.2) turned every nested list from +-- 2 to 3. mx check does not complain. +-- +-- Cause: the codec picked a PartList's marker from the CHILD element's $Type +-- alone. Both lists hold Forms$DesignPropertyValue, so no child-type +-- registration could give one 3 and the other 2; both fell to the default 3. +-- +-- Fix: codec.RegisterPropertyListMarker keys a marker on the owning $Type and +-- property; Forms$CompoundDesignPropertyValue / Properties is registered as 2. +-- Pages, snippets and layouts share the one Appearance writer, so all three +-- are covered. +-- +-- Verify: exec this, then +-- mxcli bson dump -p app.mpr --type page --object BugCompoundDPMarker.SpacingPage --format ndsl +-- shows `DesignProperties [marker=3]` and, under the Spacing value, +-- `Properties [marker=2]`. +-- ============================================================================ + +create module BugCompoundDPMarker; + +create page BugCompoundDPMarker.SpacingPage ( + title: 'Spacing Page', + layout: Atlas_Core.Atlas_Default +) { + container container1 (designproperties: ['Spacing': ['margin-bottom': 'None']]) { + dynamictext text1 (content: 'Spaced') + } +} diff --git a/mdl/backend/modelsdk/design_property_test.go b/mdl/backend/modelsdk/design_property_test.go index 7528fd6d57..15c177826c 100644 --- a/mdl/backend/modelsdk/design_property_test.go +++ b/mdl/backend/modelsdk/design_property_test.go @@ -83,3 +83,54 @@ func TestAppearanceAlwaysEmitsDesignProperties(t *testing.T) { t.Errorf("DesignProperties = %v, want the empty typed-array marker [3]", val) } } + +// A compound design property's nested Properties list carries typed-array marker +// 2, while the Appearance's own DesignProperties list carries 3 — both hold +// Forms$DesignPropertyValue children, so the marker cannot be keyed on the child +// $Type. Measured across every Studio Pro-authored document in a Mendix 11.13.0 +// app: 373 of 373 CompoundDesignPropertyValue.Properties lists are marker 2, +// 1821 of 1821 Appearance.DesignProperties lists marker 3. mxcli wrote 3 for +// both, so a describe → exec round trip of FeedbackModule.ShareFeedback +// (Feedback v4.0.2) turned every nested marker-2 list into 3. +func TestAppearanceCompoundDesignPropertyMarkers(t *testing.T) { + dps := []pages.DesignPropertyValue{ + {Key: "Spacing", ValueType: "compound", Compound: []pages.DesignPropertyValue{ + {Key: "margin-bottom", ValueType: "option", Option: "None"}, + }}, + } + out, err := (&codec.Encoder{}).Encode(newAppearance("", "", "", dps)) + if err != nil { + t.Fatalf("encode appearance: %v", err) + } + var doc bson.D + if err := bson.Unmarshal(out, &doc); err != nil { + t.Fatalf("unmarshal: %v", err) + } + + val, _ := lookupKey(doc, "DesignProperties") + outer, ok := val.(bson.A) + if !ok || len(outer) != 2 { + t.Fatalf("DesignProperties = %v, want marker + one entry", val) + } + if outer[0] != int32(3) { + t.Errorf("DesignProperties marker = %v, want 3 (Studio Pro)", outer[0]) + } + + entry, ok := outer[1].(bson.D) + if !ok { + t.Fatalf("DesignProperties[1] = %T, want a document", outer[1]) + } + v, _ := lookupKey(entry, "Value") + compound, ok := v.(bson.D) + if !ok { + t.Fatalf("Spacing Value = %T, want a document", v) + } + p, _ := lookupKey(compound, "Properties") + inner, ok := p.(bson.A) + if !ok || len(inner) != 2 { + t.Fatalf("CompoundDesignPropertyValue.Properties = %v, want marker + one entry", p) + } + if inner[0] != int32(2) { + t.Errorf("CompoundDesignPropertyValue.Properties marker = %v, want 2 (Studio Pro)", inner[0]) + } +} diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index bd297391e3..92eee25c17 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -49,6 +49,12 @@ func init() { EmptyStringFields: []string{"LocalVariable", "PageParameter", "SnippetParameter", "SubKey", "Widget"}, FalseFields: []string{"UseAllPages"}, }) + // A compound design property's nested Properties list is marker 2, though it + // holds the same Forms$DesignPropertyValue children as the Appearance's + // DesignProperties list (marker 3) — so it is keyed on the owner, not the + // child type. Measured 373 of 373 across Studio Pro-authored pages, layouts, + // building blocks and page templates in a Mendix 11.13.0 app. + codec.RegisterPropertyListMarker("Forms$CompoundDesignPropertyValue", "Properties", 2) // A ClientTemplate's Parameters list is always emitted with marker 2, even empty // (unusual — most empty lists are marker 3). codec.RegisterTypeDefaults("Forms$ClientTemplate", codec.TypeDefaults{ diff --git a/modelsdk/codec/defaults.go b/modelsdk/codec/defaults.go index 3349d1466d..bba6019a22 100644 --- a/modelsdk/codec/defaults.go +++ b/modelsdk/codec/defaults.go @@ -93,6 +93,30 @@ func RegisterListMarker(childType string, marker int32) { listMarkers[childType] = marker } +// propertyListMarkers maps an owning $Type and PartList key to its marker. It +// exists for lists whose marker depends on WHERE the list sits rather than on +// what it holds: a Forms$DesignPropertyValue list is marker 3 as a +// Forms$Appearance's DesignProperties and marker 2 as a +// Forms$CompoundDesignPropertyValue's Properties (measured: 1821/1821 and +// 373/373 across Studio Pro-authored documents in a Mendix 11.13.0 app), so a +// RegisterListMarker on the child type cannot express it. +var propertyListMarkers = map[string]map[string]int32{} + +// RegisterPropertyListMarker declares the typed-array marker for the PartList +// stored under key on elements of ownerType. It takes precedence over a +// child-type RegisterListMarker and also covers the list when it is empty. +func RegisterPropertyListMarker(ownerType, key string, marker int32) { + if propertyListMarkers[ownerType] == nil { + propertyListMarkers[ownerType] = map[string]int32{} + } + propertyListMarkers[ownerType][key] = marker +} + +func lookupPropertyListMarker(ownerType, key string) (int32, bool) { + m, ok := propertyListMarkers[ownerType][key] + return m, ok +} + func lookupListMarker(childType string) int32 { if m, ok := listMarkers[childType]; ok { return m diff --git a/modelsdk/codec/encoder.go b/modelsdk/codec/encoder.go index 943878158a..de03353d8d 100644 --- a/modelsdk/codec/encoder.go +++ b/modelsdk/codec/encoder.go @@ -162,7 +162,7 @@ func (e *Encoder) buildDoc(elem element.Element) (bson.D, error) { if idx < 0 { continue } - val, err := e.encodeEntry(rebuild[idx]) + val, err := e.encodeEntry(elem.TypeName(), rebuild[idx]) if err != nil { return nil, err } @@ -253,7 +253,7 @@ func (e *Encoder) buildDoc(elem element.Element) (bson.D, error) { continue } // Dirty field: encode new value. - val, err := e.encodeEntry(rebuild[idx]) + val, err := e.encodeEntry(elem.TypeName(), rebuild[idx]) if err != nil { return nil, err } @@ -270,7 +270,7 @@ func (e *Encoder) buildDoc(elem element.Element) (bson.D, error) { if idx < 0 || seen[idx] { continue } - val, err := e.encodeEntry(rebuild[idx]) + val, err := e.encodeEntry(elem.TypeName(), rebuild[idx]) if err != nil { return nil, err } @@ -283,7 +283,7 @@ func (e *Encoder) buildDoc(elem element.Element) (bson.D, error) { } // encodeEntry produces the BSON value for a single dirty property. -func (e *Encoder) encodeEntry(rb rebuildEntry) (any, error) { +func (e *Encoder) encodeEntry(ownerType string, rb rebuildEntry) (any, error) { wp := rb.wp // Child (Part) property. @@ -308,7 +308,7 @@ func (e *Encoder) encodeEntry(rb rebuildEntry) (any, error) { if wp.Dirty() { // Full rebuild: all children re-encoded. arr := make(bson.A, 0, 1+len(children)) - arr = append(arr, partListMarker(children)) + arr = append(arr, propertyListMarker(ownerType, rb.name, children)) for _, child := range children { childDoc, err := e.buildDoc(child) if err != nil { @@ -320,7 +320,11 @@ func (e *Encoder) encodeEntry(rb rebuildEntry) (any, error) { } // Selective rebuild: dirty children re-encoded, clean ones pass through raw bytes. arr := make(bson.A, 0, 1+len(children)) - arr = append(arr, int32(3)) + marker := int32(3) + if m, ok := lookupPropertyListMarker(ownerType, rb.name); ok { + marker = m + } + arr = append(arr, marker) for _, child := range children { if child.IsDirty() { childDoc, err := e.buildDoc(child) @@ -380,6 +384,16 @@ func zeroGUIDBinary() any { // partListMarker returns the leading typed-array marker for a PartList, derived // from the child element $Type (defaulting to 3). An empty list keeps the // default — empty mandatory lists are entity member collections, all marker 3. +// propertyListMarker is partListMarker with the owning $Type and key consulted +// first: a marker registered for the property (RegisterPropertyListMarker) wins +// over one derived from the children, and applies to an empty list too. +func propertyListMarker(ownerType, key string, children []element.Element) int32 { + if m, ok := lookupPropertyListMarker(ownerType, key); ok { + return m + } + return partListMarker(children) +} + func partListMarker(children []element.Element) int32 { if len(children) == 0 { return 3 From ee2de4fa7388abeaf8e49c5c3ec883c05b550cf7 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 06:39:25 +0000 Subject: [PATCH 24/47] fix(pages): write a nanoflow data source flat, as Studio Pro does (CE2633) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A data view with `DataSource: nanoflow Module.NF` built clean and then failed mx check with CE2633 "No nanoflow configured for the data source of this data view". mxcli nested the name in a Forms$NanoflowSettings child, by analogy with Forms$MicroflowSource; Studio Pro's Forms$NanoflowSource is flat — ForceFullObjects, Nanoflow and ParameterMappings (marker 2) directly on the source, which is gen's shape. Measured on 5 of 5 Studio Pro-authored nanoflow sources in Feedback v4.0.2 (11.13.0); the 4 microflow sources there nest Forms$MicroflowSettings as gen says, so that path is unchanged. - CREATE PAGE: nanoflowSourceToGen builds via gen, and now also carries the source's parameter mappings, which the raw builder dropped. - ALTER PAGE set DataSource: serializeDataSourceBson writes the flat shape. - Readers (ALTER PAGE flow-context lookup, describe's argument reader) accept both shapes; pages written before this fix keep the nested one. Verified with mx check 11.13.0: ShareFeedback describe -> exec, the repro script and an ALTER PAGE set all go from CE2633 to 0 errors. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-backend.jsonl | 1 + .../dataview-nanoflow-source-ce2633.mdl | 69 +++++++++++++++++ mdl/backend/modelsdk/widget_write.go | 21 +++--- .../modelsdk/widget_write_legacy_gaps.go | 41 ++++++---- .../modelsdk/widget_write_legacy_gaps_test.go | 71 ++++++++++++++---- mdl/backend/pagemutator/mutator.go | 18 +++-- .../mutator_nanoflow_source_test.go | 75 +++++++++++++++++++ mdl/executor/cmd_pages_describe_datasource.go | 9 ++- .../cmd_pages_describe_datasource_test.go | 29 ++++++- 9 files changed, 287 insertions(+), 47 deletions(-) create mode 100644 mdl-examples/bug-tests/dataview-nanoflow-source-ce2633.mdl create mode 100644 mdl/backend/pagemutator/mutator_nanoflow_source_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-backend.jsonl b/.claude/skills/fix-issue/findings/mdl-backend.jsonl index 9a677c2619..ba2d818091 100644 --- a/.claude/skills/fix-issue/findings/mdl-backend.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-backend.jsonl @@ -131,3 +131,4 @@ {"area": "mdl/backend", "date": "2026-09-22", "symptom": "`create workflow … overview page X` reports `Created workflow` and exit 0 and stores NOTHING — the written unit carries no page reference and not even the page's qualified name as a string. `mx check` passes (a workflow with no overview page is valid) and `describe workflow` omits the clause, so nothing reveals the loss. Running `alter workflow … set overview page X` afterwards DOES write it, which is what makes the split visible", "cause": "Two fields for one concept, never joined: the executor set semantic `Workflow.OverviewPage` (`cmd_workflows_write.go:170`) and `workflowToGen` only ever read `Workflow.AdminPage`, which nothing set. The READ half was wrong in the mirror direction — `workflowFromGen` took `g.OverviewPageQualifiedName()`, so even the correctly-written ALTER read back empty and the catalog's overview-page reference edge never fired", "file": "`sdk/workflows/workflow.go` (the two fields collapsed to one), `mdl/backend/modelsdk/workflow_write.go` (`workflowToGen`), `mdl/backend/modelsdk/workflow_read.go` (`workflowOverviewPageName`)", "insight": "**The Model SDK's StructureVersionInfo settles which of two rival property names is real, in one grep**: `npm pack mendixmodelsdk` then `src/gen/workflows.js` gives `overviewPage: {deleted: \"9.11.0\"}` and `adminPage: {introduced: \"9.11.0\"}` — so AdminPage (a `Workflows$PageReference` CHILD, not a by-name string) is the stored property, and `generated/metamodel` agrees by declaring AdminPage and no OverviewPage. `modelsdk/gen` declares BOTH, which is how a reader and a writer ended up on opposite sides of a 9.11 rename inside one package. **The version branch CLAUDE.md's overlay rule would demand is dead here, and that is a measurement not an assumption**: `workflowToGen` writes `WorkflowV2`, introduced in 11.1.0, unconditionally — so no reachable project wants the pre-9.11 key. Write one spelling, READ both (a read fallback invents nothing). **The differential that proves it on a real build**: same script, same project, only the write suppressed — control 0 errors, fixed `CE7410 \"The selected page 'Overview' should accept a parameter of type 'Workflow'\"` on mxbuild 11.6.6. mxbuild can only validate a page it can see, so the error IS the evidence; with a valid overview page both variants are 0 errors, which is the usual weak-signal trap. Useful side-finding: an overview page takes **System.Workflow**, while a user task's page takes **System.WorkflowUserTask** — two pages, two parameters. NOT fixed: no check rule for CE7410 yet, and `WorkflowV2` being written unconditionally is questionable for a 10.x project. Same shape as the `create … comment 'text'` bug (findings/mdl-grammar.jsonl 2026-08-25): grep for `stmt.X = …` / `wf.X = …` with no matching read. Tests `mdl/backend/modelsdk/workflow_overview_page_test.go`; repro `mdl-examples/bug-tests/workflow-586b-overview-page-dropped.mdl`", "refs": ["ako/mxcli#586"], "ce": ["CE7410"]} {"area": "mdl/backend", "date": "2026-09-23", "symptom": "`describe java action` prints `ContextObject: entity <>` for a parameter declared `entity not null`; a bare type-parameter reference (`Obj: pEntity`, `returns pEntity`) reads back nameless too. The description no longer round-trips", "cause": "The stored parameter type (`CodeActions$EntityTypeParameterType` / `ParameterizedEntityType`) holds only a BY_ID pointer to the `CodeActions$TypeParameter`. `javaActionFromGen` carried the ID into the semantic type but never resolved it to the name, and the name is all the describer prints. `javascript_read.go` had always done this resolution pass; the Java reader was ported without it", "file": "`mdl/backend/modelsdk/java_read.go` (`resolveJavaActionTypeParameterNames`)", "insight": "The executor's `entity <>` fallback in `formatJavaActionType` is the tell: an empty name at DESCRIBE means the *reader* dropped a by-ID resolution, not that the writer lost it \u2014 the write path sets both ID and name, so a create\u2192read unit test in the backend reproduces it without any project. When a JS and a Java reader cover the same `CodeActions$` shapes, diff their post-processing first; any pass one has and the other lacks is a candidate. The write was never wrong (replaying the fixed description into a fresh project describes identically), so no `mx check` run is needed. Issue mendixlabs/mxcli#1034", "refs": ["mendixlabs/mxcli#1034"]} {"area": "mdl/backend", "date": "2026-09-23", "symptom": "`CREATE OR MODIFY PERSISTENT ENTITY` was REFUSED by the #1119 storage-GUID guard on a doctype script that had been passing for months: `failed to update entity: refusing to write unit d82b0484-…: 1 element(s) kept their $ID but would be written with a different GUID — dff2ced1-… (DomainModels$Attribute): stored 4b52b36b-…, would write dff2ced1-…`. Two independent defects wore that one message.", "cause": "(1) A REAL data loss the guard caught: `mergeDeclaredOntoStoredEntity` sets `merged.Attributes = declared.Attributes` and `merged.Indexes = declared.Indexes` — the lists the STATEMENT declares, built from text by the visitor and carrying no element ID — and `carryChildIdentity` keyed entirely on that ID, reading an empty one as 'a genuinely new member, so a fresh GUID is right'. Every attribute of a re-declared entity was therefore re-minted, i.e. #1119 through a second executor path. (2) A FALSE POSITIVE in the guard itself: `canon.TransplantIDs` pairs STRUCTURALLY ($Type + shape, LCS-anchored), so on a statement that drops six differently-named attributes and adds one, it paired the NEW attribute with a REMOVED one and handed it that stored `$ID`; the codec had written `GUID = $ID` and the transplant substitutes over every 16-byte binary, so the GUID followed. The guard's premise — written into its own doc comment as 'unambiguously' — that a shared `$ID` after the transplant means the same element, is false.", "file": "`mdl/backend/modelsdk/domainmodel_child_identity.go` (`carryAttributeIdentity`, `carryIndexIdentity`), `modelsdk/canon/storageguid.go` (`sameMember`, `elementGUIDs`)", "insight": "ONE ERROR MESSAGE, TWO DEFECTS, AND FIXING EITHER ALONE LEAVES IT RED — which is why the first fix (the name fallback) changed nothing and the SAME element and GUIDs came back byte-for-byte. That repetition was the signal: an identical failure after a real fix means the reproduction is exercising a different code path than the one reasoned about. What settled it was describing the actual subject: the marketplace `PublishedBusinessEvent` has six attributes and NONE is named `EventId`, so there was no member to carry — the pairing itself was spurious. Reproduce against the real stored document before believing any theory about which elements correspond. METHOD that made this cheap: the CI failure reproduced locally in 0.5s as a backend unit test (strip the IDs off a fixture entity's attributes, call UpdateEntity) versus 26s for the integration subtest, but ONLY the integration subtest could have found the second defect, because the false pairing needs a real drop-six-add-one document. Run both. TRADE-OFF worth restating: the guard now pairs on `$ID` + `$Type` + `Name`, which loses one arm — a RENAME that re-mints a GUID is no longer refused, since the name is what changed — and that arm is covered directly by the carry tests where it is decidable. A backstop that refuses correct writes is worse than a backstop with a hole: the first makes documented statements unusable, and this one already had. Also: feeding a deliberately-approximate pairing to a guard promotes its error rate into refusals. TransplantIDs' correctness bar is low ON PURPOSE (a wrong match only makes a diff bigger); anything that reads its output as identity has to add its own test of identity.", "refs": ["mendixlabs/mxcli#1119", "mendixlabs/mxcli#1169", "ako/mxcli#643"], "ce": []} +{"area":"mdl/backend","date":"2026-09-25","symptom":"A data view with `DataSource: nanoflow Module.NF` (e.g. describe → exec of Feedback v4.0.2's FeedbackModule.ShareFeedback) passes `mxcli check` and exec, then mxbuild 11.13.0 reports CE2633 \"No nanoflow configured for the data source of this data view\". Same result through `alter page … set DataSource = nanoflow X on dv`","cause":"Both writers nested the name in a `Forms$NanoflowSettings` child (ParameterMappings marker 3) by analogy with `Forms$MicroflowSource`, which really does nest `Forms$MicroflowSettings`. Studio Pro's `Forms$NanoflowSource` is FLAT: ForceFullObjects, Nanoflow, ParameterMappings (marker 2) directly on the source — exactly gen's shape. mxbuild found no Nanoflow key. The raw nested builder also never read d.ParameterMappings, so a parameterized source nanoflow lost its arguments. On the read side, the ALTER PAGE flow-context lookup (`flowFromDataSourceDoc`) and describe's argument reader (`flowSourceArgs`) only knew the nested shape, so Studio Pro-authored nanoflow sources yielded no entity context / no arguments","file":"`mdl/backend/modelsdk/widget_write_legacy_gaps.go` (`nanoflowSourceToGen` via gen + `Forms$NanoflowSource` TypeDefaults in `widget_write.go`), `mdl/backend/pagemutator/mutator.go` (`serializeDataSourceBson`, `flowFromDataSourceDoc`), `mdl/executor/cmd_pages_describe_datasource.go` (`flowSourceArgs`)","insight":"**The code comment asserted the wrong shape as a measured fact** (\"Studio Pro nests it in a Forms$NanoflowSettings child … Legacy's shape is the one with a working project behind it\") and a unit test pinned it — both were parity-with-legacy, never measured. `Forms$NanoflowSettings` is not a type in modelsdk/gen or generated/metamodel: **when gen and a hand-rolled builder disagree about a type's shape, grep gen for the type the builder invents before trusting the builder**. What settled it in one step: a 60-line scanner that `bson.Unmarshal`s every mprcontents unit and prints the key-set (with list markers) of each `$Type` instance — 5 of 5 flat nanoflow sources, and 4 of 4 microflow sources nested as gen says, so the microflow path needed nothing. **Enumerate every writer of the type, not just the reported one**: the ALTER PAGE setter had its own copy of the same wrong literal, and the read-side lookups keyed on the wrong shape meant Studio Pro pages were the ones silently mis-read. Readers keep the nested fallback for pages written before the fix. Verified: exec + `mx check` 11.13.0 CE2633 → 0 errors for CREATE PAGE (ShareFeedback round trip, repro script) and ALTER PAGE; ShareFeedback's dataView5 DataSource ndsl now matches Studio Pro exactly. Control: implementation reverted → the 5 new tests fail with the nested key set. Repro `mdl-examples/bug-tests/dataview-nanoflow-source-ce2633.mdl`","refs":[],"ce":["CE2633"]} diff --git a/mdl-examples/bug-tests/dataview-nanoflow-source-ce2633.mdl b/mdl-examples/bug-tests/dataview-nanoflow-source-ce2633.mdl new file mode 100644 index 0000000000..f72e7fed41 --- /dev/null +++ b/mdl-examples/bug-tests/dataview-nanoflow-source-ce2633.mdl @@ -0,0 +1,69 @@ +-- ============================================================================ +-- Data view sourced from a NANOFLOW — CE2633 "No nanoflow configured" +-- ============================================================================ +-- +-- Symptom: a data view with `DataSource: nanoflow Module.NF` — for example a +-- describe → exec round trip of FeedbackModule.ShareFeedback (Feedback v4.0.2) +-- — passed `mxcli check` and exec, then mxbuild 11.13.0 reported: +-- +-- [error] [CE2633] "No nanoflow configured for the data source of this data +-- view. Select a nanoflow or change the data source." at Data view 'dataView5' +-- +-- Cause: mxcli wrote the Forms$NanoflowSource NESTED, by analogy with its +-- microflow sibling (which really does nest a Forms$MicroflowSettings): +-- +-- DataSource: Forms$NanoflowSource +-- NanoflowSettings: Forms$NanoflowSettings +-- Nanoflow: "…" +-- ParameterMappings [marker=3]: [] +-- +-- Studio Pro's shape is FLAT — measured on 5 of 5 Studio Pro-authored nanoflow +-- sources in Feedback v4.0.2 — so mxbuild found no Nanoflow key on the source: +-- +-- DataSource: Forms$NanoflowSource +-- ForceFullObjects: false +-- Nanoflow: "…" +-- ParameterMappings [marker=2]: [] +-- +-- Fix: the CREATE PAGE writer (nanoflowSourceToGen) and the ALTER PAGE setter +-- (serializeDataSourceBson) write the flat shape; describe and the ALTER PAGE +-- entity-context lookup read both shapes, since pages written before the fix +-- still carry the nested one. +-- +-- Verify (needs a project, so `make check-mdl` only syntax-checks this): +-- +-- mxcli exec dataview-nanoflow-source-ce2633.mdl -p app.mpr +-- mxcli docker check -p app.mpr +-- +-- Expect 0 errors. Before the fix: CE2633 at Data view 'dvItem'. +-- ============================================================================ + +create module BugNF2633; + +@position(100, 100) +create non-persistent entity BugNF2633.Item ( + Name: string(100) +); + +create nanoflow BugNF2633.DS_Item () +returns BugNF2633.Item as $Item +begin + $Item = create BugNF2633.Item (Name = 'from nanoflow'); + return $Item; +end; +/ + +create page BugNF2633.Item_View ( + Title: 'Item', + Layout: Atlas_Core.Atlas_Default +) { + layoutgrid lg1 { + row r1 { + column c1 (DesktopWidth: autofill) { + dataview dvItem (DataSource: nanoflow BugNF2633.DS_Item) { + textbox txtName (Label: 'Name', Attribute: Name) + } + } + } + } +}; diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index bd297391e3..0cf5a43d7a 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -162,6 +162,12 @@ func init() { MandatoryListMarkers: map[string]int32{"ParameterMappings": 2}, NullFields: []string{"ProgressMessage", "ConfirmationInfo"}, }) + // A nanoflow DATA SOURCE is flat — no settings child — and carries its + // (possibly empty) mapping list directly, marker 2. Measured: 5 of 5 Studio + // Pro-authored Forms$NanoflowSource in Feedback v4.0.2 at 11.13.0. + codec.RegisterTypeDefaults("Forms$NanoflowSource", codec.TypeDefaults{ + MandatoryListMarkers: map[string]int32{"ParameterMappings": 2}, + }) // TextBox: many null slots when unbound (attribute ref, screen-reader label, // source variable, label template, visibility/editability/native settings). codec.RegisterTypeDefaults("Forms$TextBox", codec.TypeDefaults{ @@ -1366,9 +1372,8 @@ func dataViewSourceToGen(ds pages.DataSource) (element.Element, error) { ms.SetMicroflowSettings(microflowSettingsToGen(d.Microflow, d.ParameterMappings)) return ms, nil - // A NANOFLOW data source. Its sibling above goes through gen; this one is - // built raw because gen binds the nanoflow name directly on the source while - // Studio Pro nests it in a Forms$NanoflowSettings child — see + // A NANOFLOW data source. Flat, unlike the microflow source above: the + // nanoflow binds directly on the source, no settings child (CE2633) — see // nanoflowSourceToGen. case *pages.NanoflowSource: return nanoflowSourceToGen(d), nil @@ -1481,9 +1486,8 @@ func listViewSourceToGen(ds pages.DataSource) (element.Element, error) { ms.SetMicroflowSettings(microflowSettingsToGen(d.Microflow, d.ParameterMappings)) return ms, nil - // A NANOFLOW data source. Its sibling above goes through gen; this one is - // built raw because gen binds the nanoflow name directly on the source while - // Studio Pro nests it in a Forms$NanoflowSettings child — see + // A NANOFLOW data source. Flat, unlike the microflow source above: the + // nanoflow binds directly on the source, no settings child (CE2633) — see // nanoflowSourceToGen. case *pages.NanoflowSource: return nanoflowSourceToGen(d), nil @@ -1547,9 +1551,8 @@ func customWidgetDataSourceToGen(ds pages.DataSource) (element.Element, error) { ms.SetMicroflowSettings(microflowSettingsToGen(d.Microflow, d.ParameterMappings)) return ms, nil - // A NANOFLOW data source. Its sibling above goes through gen; this one is - // built raw because gen binds the nanoflow name directly on the source while - // Studio Pro nests it in a Forms$NanoflowSettings child — see + // A NANOFLOW data source. Flat, unlike the microflow source above: the + // nanoflow binds directly on the source, no settings child (CE2633) — see // nanoflowSourceToGen. case *pages.NanoflowSource: return nanoflowSourceToGen(d), nil diff --git a/mdl/backend/modelsdk/widget_write_legacy_gaps.go b/mdl/backend/modelsdk/widget_write_legacy_gaps.go index 5043cddf26..6b34dcd6f2 100644 --- a/mdl/backend/modelsdk/widget_write_legacy_gaps.go +++ b/mdl/backend/modelsdk/widget_write_legacy_gaps.go @@ -224,23 +224,34 @@ func imageViewerSourceToGen(ds pages.DataSource) (element.Element, error) { } } -// nanoflowSourceToGen builds a Forms$NanoflowSource — a list widget's "nanoflow" -// data source. +// nanoflowSourceToGen builds a Forms$NanoflowSource — a "nanoflow" data source +// on a data view, list view or pluggable widget. // -// Built raw, and this one is a judgement rather than a limitation: gen's -// NanoflowSource offers ForceFullObjects and NanoflowQualifiedName, binding the -// nanoflow name DIRECTLY on the source, while Studio Pro nests it inside a -// Forms$NanoflowSettings child alongside ParameterMappings — which is what -// sdk/mpr writes. Writing gen's shape would put the name in a key Studio Pro -// does not read there, the same class of defect as the storage-name overrides -// (CLAUDE.md). Legacy's shape is the one with a working project behind it. +// Unlike its microflow sibling, the nanoflow source is FLAT: ForceFullObjects, +// Nanoflow and ParameterMappings sit directly on the source, which is exactly +// gen's shape. Measured: all 5 Studio Pro-authored Forms$NanoflowSource +// documents in Feedback v4.0.2 (Mendix 11.13.0) carry those three keys and no +// others. This used to nest a Forms$NanoflowSettings child by analogy with +// Forms$MicroflowSettings — a type that does not exist — and mxbuild, finding +// no Nanoflow key on the source, reported CE2633 "No nanoflow configured for +// the data source of this data view". func nanoflowSourceToGen(d *pages.NanoflowSource) element.Element { - g := newElem("Forms$NanoflowSource", string(d.ID)) - settings := newElem("Forms$NanoflowSettings", "") - addStr(settings, "Nanoflow", d.Nanoflow) - addEmptyTypedList(settings, "ParameterMappings", 3) - addPart(g, "NanoflowSettings", settings) - return g + src := genPg.NewNanoflowSource() + if d.ID != "" { + src.SetID(element.ID(d.ID)) + } + assignID(src) + src.SetForceFullObjects(false) + src.SetNanoflowQualifiedName(d.Nanoflow) + for _, pm := range d.ParameterMappings { + m := genPg.NewNanoflowParameterMapping() + assignID(m) + // Parameter is a BY_NAME reference: .. + m.SetParameterQualifiedName(d.Nanoflow + "." + pm.ParameterName) + bindParameterMappingValue(m, pm.Variable, pm.VariableKind, pm.Expression) + src.AddParameterMappings(m) + } + return src } // emptyClientTemplate is the Forms$ClientTemplate an image's AlternativeText diff --git a/mdl/backend/modelsdk/widget_write_legacy_gaps_test.go b/mdl/backend/modelsdk/widget_write_legacy_gaps_test.go index 8f16efd977..53e1190e17 100644 --- a/mdl/backend/modelsdk/widget_write_legacy_gaps_test.go +++ b/mdl/backend/modelsdk/widget_write_legacy_gaps_test.go @@ -363,24 +363,69 @@ func assertEmptyClientTemplate(t *testing.T, parent bsonv1.D, key string) { } } -// TestNanoflowSourceNestsSettings — gen binds the nanoflow name directly on the -// source; Studio Pro nests it in a Forms$NanoflowSettings child. Writing gen's -// shape would put the name in a key Studio Pro does not read there. -func TestNanoflowSourceNestsSettings(t *testing.T) { +// TestNanoflowSourceIsFlat_StudioProShape — a Forms$NanoflowSource binds its +// nanoflow DIRECTLY: ForceFullObjects, Nanoflow, ParameterMappings, and nothing +// else. Measured: all 5 Studio Pro-authored Forms$NanoflowSource documents in +// the Feedback v4.0.2 module (Mendix 11.13.0) have exactly that key set, with an +// empty ParameterMappings stored as marker 2. There is no Forms$NanoflowSettings +// type — the nested shape this writer used to mint (by analogy with +// Forms$MicroflowSource, which DOES nest a Forms$MicroflowSettings) leaves +// mxbuild seeing no nanoflow at all: CE2633 "No nanoflow configured for the +// data source of this data view". +func TestNanoflowSourceIsFlat_StudioProShape(t *testing.T) { el := nanoflowSourceToGen(&pages.NanoflowSource{Nanoflow: "MyModule.NF_GetItems"}) - doc := encodeElement(t, el) + d := encodeToD(t, el) - if got := docGet(doc, "$Type"); got != "Forms$NanoflowSource" { - t.Fatalf("$Type = %v", got) + var got []string + for _, e := range d { + got = append(got, e.Key) + } + sort.Strings(got) + want := []string{"$ID", "$Type", "ForceFullObjects", "Nanoflow", "ParameterMappings"} + if strings.Join(got, ",") != strings.Join(want, ",") { + t.Fatalf("keys = %v, want exactly %v (Studio Pro's shape)", got, want) + } + if v := docGet(d, "$Type"); v != "Forms$NanoflowSource" { + t.Errorf("$Type = %v", v) + } + if v := docGet(d, "Nanoflow"); v != "MyModule.NF_GetItems" { + t.Errorf("Nanoflow = %v, want MyModule.NF_GetItems", v) + } + if v, ok := docGet(d, "ForceFullObjects").(bool); !ok || v { + t.Errorf("ForceFullObjects = %#v, want false", docGet(d, "ForceFullObjects")) } - if docGet(doc, "Nanoflow") != nil { - t.Error("Nanoflow bound directly on the source; it belongs in NanoflowSettings") + pm, ok := docGet(d, "ParameterMappings").(bsonv1.A) + if !ok || len(pm) != 1 || pm[0] != int32(2) { + t.Errorf("ParameterMappings = %#v, want [2] (Studio Pro's empty-list marker)", docGet(d, "ParameterMappings")) } - settings, ok := docGet(doc, "NanoflowSettings").(bsonv1.D) +} + +// A parameterized source nanoflow keeps its arguments, as +// Forms$NanoflowParameterMapping items directly under the source. The raw +// nested builder never read d.ParameterMappings, so they were dropped. +func TestNanoflowSource_KeepsParameterMappings(t *testing.T) { + el := nanoflowSourceToGen(&pages.NanoflowSource{ + Nanoflow: "MyModule.NF_GetItems", + ParameterMappings: []*pages.MicroflowParameterMapping{ + {ParameterName: "Limit", Expression: "10"}, + }, + }) + d := encodeToD(t, el) + pm, ok := docGet(d, "ParameterMappings").(bsonv1.A) + if !ok || len(pm) != 2 { + t.Fatalf("ParameterMappings = %#v, want marker + 1 mapping", docGet(d, "ParameterMappings")) + } + m, ok := pm[1].(bsonv1.D) if !ok { - t.Fatalf("NanoflowSettings = %T", docGet(doc, "NanoflowSettings")) + t.Fatalf("mapping = %T", pm[1]) + } + if v := docGet(m, "$Type"); v != "Forms$NanoflowParameterMapping" { + t.Errorf("mapping $Type = %v", v) + } + if v := docGet(m, "Parameter"); v != "MyModule.NF_GetItems.Limit" { + t.Errorf("mapping Parameter = %v", v) } - if got := docGet(settings, "Nanoflow"); got != "MyModule.NF_GetItems" { - t.Errorf("NanoflowSettings.Nanoflow = %v", got) + if v := docGet(m, "Expression"); v != "10" { + t.Errorf("mapping Expression = %v", v) } } diff --git a/mdl/backend/pagemutator/mutator.go b/mdl/backend/pagemutator/mutator.go index 8e3ecdc57a..ae7c5be3fb 100644 --- a/mdl/backend/pagemutator/mutator.go +++ b/mdl/backend/pagemutator/mutator.go @@ -1123,6 +1123,12 @@ func flowFromDataSourceDoc(ds bson.D) (microflow, nanoflow string) { if s := bsonnav.DGetDoc(ds, "MicroflowSettings"); s != nil { return bsonnav.DGetString(s, "Microflow"), "" } + // A Forms$NanoflowSource names its nanoflow directly (Studio Pro's shape); + // the nested NanoflowSettings is what mxcli wrote before CE2633 was fixed, + // and such pages are still out there. + if nf := bsonnav.DGetString(ds, "Nanoflow"); nf != "" { + return "", nf + } if s := bsonnav.DGetDoc(ds, "NanoflowSettings"); s != nil { return "", bsonnav.DGetString(s, "Nanoflow") } @@ -3129,15 +3135,15 @@ func serializeDataSourceBson(ds pages.DataSource) bson.D { }}, } case *pages.NanoflowSource: + // Flat, unlike the microflow source: no settings child. Measured on 5 of + // 5 Studio Pro-authored nanoflow sources (Feedback v4.0.2, 11.13.0); the + // nested Forms$NanoflowSettings shape is CE2633 "No nanoflow configured". return bson.D{ {Key: "$ID", Value: bsonutil.NewIDBsonBinary()}, {Key: "$Type", Value: "Forms$NanoflowSource"}, - {Key: "NanoflowSettings", Value: bson.D{ - {Key: "$ID", Value: bsonutil.NewIDBsonBinary()}, - {Key: "$Type", Value: "Forms$NanoflowSettings"}, - {Key: "Nanoflow", Value: d.Nanoflow}, - {Key: "ParameterMappings", Value: bson.A{int32(3)}}, - }}, + {Key: "ForceFullObjects", Value: false}, + {Key: "Nanoflow", Value: d.Nanoflow}, + {Key: "ParameterMappings", Value: bson.A{int32(2)}}, } default: return nil diff --git a/mdl/backend/pagemutator/mutator_nanoflow_source_test.go b/mdl/backend/pagemutator/mutator_nanoflow_source_test.go new file mode 100644 index 0000000000..2a39c4d2f6 --- /dev/null +++ b/mdl/backend/pagemutator/mutator_nanoflow_source_test.go @@ -0,0 +1,75 @@ +// SPDX-License-Identifier: Apache-2.0 + +package pagemutator + +import ( + "sort" + "strings" + "testing" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/mdl/backend/bsonnav" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// A Forms$NanoflowSource is FLAT — ForceFullObjects, Nanoflow, ParameterMappings +// directly on the source. Measured: 5 of 5 Studio Pro-authored nanoflow sources +// in Feedback v4.0.2 (Mendix 11.13.0). The nested Forms$NanoflowSettings shape +// mxcli used to write makes mxbuild report CE2633 "No nanoflow configured for +// the data source of this data view". +func TestSetWidgetDataSource_NanoflowIsFlat(t *testing.T) { + dv := bson.D{ + {Key: "$Type", Value: "Forms$DataView"}, + {Key: "Name", Value: "dv1"}, + {Key: "DataSource", Value: bson.D{{Key: "$Type", Value: "Forms$MicroflowSource"}}}, + } + m := New(makeRawPage(dv), model.ID("unit-1"), nil) + if err := m.SetWidgetDataSource("dv1", &pages.NanoflowSource{Nanoflow: "Mod.DS_NF"}); err != nil { + t.Fatalf("SetWidgetDataSource: %v", err) + } + ds := bsonnav.DGetDoc(findWidgetForTest(t, m.rawData, "dv1"), "DataSource") + var keys []string + for _, e := range ds { + keys = append(keys, e.Key) + } + sort.Strings(keys) + want := "$ID,$Type,ForceFullObjects,Nanoflow,ParameterMappings" + if got := strings.Join(keys, ","); got != want { + t.Fatalf("keys = %s, want %s (Studio Pro's shape)", got, want) + } + if v := bsonnav.DGetString(ds, "Nanoflow"); v != "Mod.DS_NF" { + t.Errorf("Nanoflow = %q", v) + } + if v, ok := bsonnav.DGet(ds, "ForceFullObjects").(bool); !ok || v { + t.Errorf("ForceFullObjects = %#v, want false", bsonnav.DGet(ds, "ForceFullObjects")) + } + if pm, ok := bsonnav.DGet(ds, "ParameterMappings").(bson.A); !ok || len(pm) != 1 || pm[0] != int32(2) { + t.Errorf("ParameterMappings = %#v, want [2]", bsonnav.DGet(ds, "ParameterMappings")) + } +} + +// The flow a source names must be found in BOTH shapes: Studio Pro's flat one, +// and the nested one mxcli wrote before the fix, which stays in projects. +func TestFlowFromDataSourceDoc_NanoflowBothShapes(t *testing.T) { + flat := bson.D{ + {Key: "$Type", Value: "Forms$NanoflowSource"}, + {Key: "ForceFullObjects", Value: false}, + {Key: "Nanoflow", Value: "Mod.DS_Flat"}, + {Key: "ParameterMappings", Value: bson.A{int32(2)}}, + } + if _, nf := flowFromDataSourceDoc(flat); nf != "Mod.DS_Flat" { + t.Errorf("flat (Studio Pro) shape: nanoflow = %q, want Mod.DS_Flat", nf) + } + nested := bson.D{ + {Key: "$Type", Value: "Forms$NanoflowSource"}, + {Key: "NanoflowSettings", Value: bson.D{ + {Key: "$Type", Value: "Forms$NanoflowSettings"}, + {Key: "Nanoflow", Value: "Mod.DS_Nested"}, + }}, + } + if _, nf := flowFromDataSourceDoc(nested); nf != "Mod.DS_Nested" { + t.Errorf("nested (pre-fix mxcli) shape: nanoflow = %q, want Mod.DS_Nested", nf) + } +} diff --git a/mdl/executor/cmd_pages_describe_datasource.go b/mdl/executor/cmd_pages_describe_datasource.go index 00a5b52f8b..a844e05e44 100644 --- a/mdl/executor/cmd_pages_describe_datasource.go +++ b/mdl/executor/cmd_pages_describe_datasource.go @@ -467,10 +467,17 @@ func xpathConstraintClause(constraint string) string { // A parameterless flow yields nil, which the renderer emits without parentheses // — the grammar makes the list optional, and adding empty parens would churn // every existing description. +// +// A nanoflow source is FLAT in Studio Pro's shape — ParameterMappings directly +// on the source — so the source itself is read when it has no settings child; +// the nested NanoflowSettings form is what mxcli wrote before CE2633 was fixed. func flowSourceArgs(ds map[string]any, settingsKey, flowName string) []rawDataSourceArg { settings, ok := ds[settingsKey].(map[string]any) if !ok || settings == nil { - return nil + if _, flat := ds["ParameterMappings"]; !flat { + return nil + } + settings = ds } var out []rawDataSourceArg for _, item := range getBsonArrayElements(settings["ParameterMappings"]) { diff --git a/mdl/executor/cmd_pages_describe_datasource_test.go b/mdl/executor/cmd_pages_describe_datasource_test.go index e6ba969d14..bc0f7cdab4 100644 --- a/mdl/executor/cmd_pages_describe_datasource_test.go +++ b/mdl/executor/cmd_pages_describe_datasource_test.go @@ -328,9 +328,32 @@ func TestDataSourceArgsOmittedWhenThereAreNone(t *testing.T) { } } -// TestDataSourceArgsNanoflow pins the sibling shape: a nanoflow source stores -// its arguments under NanoflowSettings, and losing them there fails the build -// the same way. +// TestDataSourceArgsNanoflowFlat — Studio Pro stores a nanoflow source FLAT: +// Nanoflow and ParameterMappings directly on the source (5 of 5 in Feedback +// v4.0.2 at 11.13.0). Its arguments must be read from there. +func TestDataSourceArgsNanoflowFlat(t *testing.T) { + ds := map[string]any{ + "$Type": "Forms$NanoflowSource", + "ForceFullObjects": false, + "Nanoflow": "Mod.NF_Rows", + "ParameterMappings": []any{ + int32(2), + map[string]any{"Parameter": "Mod.NF_Rows.Ctx", "Expression": "$currentObject"}, + }, + } + got := parseDataSource(ds) + if got == nil { + t.Fatal("nanoflow datasource not read") + } + want := "nanoflow Mod.NF_Rows(Ctx: $currentObject)" + if expr := dataSourceExpr(got); expr != want { + t.Errorf("rendered %q, want %q", expr, want) + } +} + +// TestDataSourceArgsNanoflow pins the nested shape mxcli wrote before CE2633 +// was fixed (arguments under NanoflowSettings) — such pages still exist, so +// describe keeps reading it. func TestDataSourceArgsNanoflow(t *testing.T) { ds := map[string]any{ "$Type": "Forms$NanoflowSource", From 66f186ecb7b630969457d80d690fea8773cf74be Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 06:43:54 +0000 Subject: [PATCH 25/47] feat(pages): round-trip "Visible: based on attribute value" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit describe → exec of Administration.Account_Edit (Administration v4.3.2, Mendix 11.13.0) took its 8 Enumerations$Condition entries to 0. Every widget set up in Studio Pro to show only for certain attribute values became always visible, and check, exec and `mx check` all reported success. MDL now spells the form as Visible: IsLocalUser in (true) Visible: Status in (Running, empty) -- empty = Studio Pro's "(empty)" It lists the values that show the widget. The attribute must be a Boolean or enumeration attribute of the data container's entity, and inherited ones are qualified with their declaring entity. Studio Pro stores one condition per value, so mxcli writes every value: enumeration values in declaration order plus "(empty)", or "true" then "false". Unknown values, other attribute types, association paths and use outside a data container are refused. The settings node now uses Studio Pro's list markers, measured on all 12 settings in the project: Conditions [2] and ModuleRoles [1], empty or not. mxcli wrote [3] for both, the expression form included. Round trip: Account_Overview (4 conditions), ScheduledEvents (20) and ShareFeedback (2) come back with byte-identical ConditionalVisibilitySettings. Account_Edit round-trips 6 of its 8; the other 2 sit on a Forms$Label, which needs the `label` widget from #670. `mx check` reports no new errors. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .claude/skills/mendix/create-page/SKILL.md | 10 ++ docs-site/src/language/pages.md | 13 ++ .../visible-based-on-attribute-value.mdl | 48 +++++ mdl/ast/ast_page_v3.go | 8 + mdl/backend/modelsdk/widget_write.go | 23 ++- .../widget_write_visible_when_test.go | 91 ++++++++++ mdl/executor/cmd_pages_builder_v3.go | 3 + .../cmd_pages_builder_visible_when.go | 164 ++++++++++++++++++ mdl/executor/cmd_pages_describe.go | 8 +- mdl/executor/cmd_pages_describe_output.go | 33 ++++ mdl/executor/cmd_pages_describe_parse.go | 20 +++ mdl/executor/cmd_pages_visible_when_test.go | 152 ++++++++++++++++ mdl/executor/executor.go | 1 + mdl/executor/validate_widgets.go | 2 +- mdl/grammar/domains/MDLPage.g4 | 7 + mdl/visitor/visitor_page_v3.go | 9 + mdl/visitor/visitor_visible_when_test.go | 55 ++++++ sdk/pages/pages_widgets.go | 14 +- 19 files changed, 655 insertions(+), 7 deletions(-) create mode 100644 mdl-examples/bug-tests/visible-based-on-attribute-value.mdl create mode 100644 mdl/backend/modelsdk/widget_write_visible_when_test.go create mode 100644 mdl/executor/cmd_pages_builder_visible_when.go create mode 100644 mdl/executor/cmd_pages_visible_when_test.go create mode 100644 mdl/visitor/visitor_visible_when_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 779d5ee033..954031f8ba 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -698,3 +698,4 @@ {"area": "mdl/executor", "date": "2026-09-25", "symptom": "In the editor, on a project that never ran `mxcli widget init`, widget completions offer only the nine embedded widgets (textfilter, combobox, … barcodescanner) although e.g. Fieldset is installed in widgets/; LSP widget diagnostics read the same thin registry.", "cause": "cmd/mxcli/lsp_completion.go ensureWidgetRegistry built its registry with NewWidgetRegistry + LoadUserDefinitions, which reads only the gitignored .mxcli/widgets/*.def.json. The last reader left over after mendixlabs/mxcli#1135 and ako/mxcli#663 fixed check, DESCRIBE WIDGET and `widget list`.", "file": "cmd/mxcli/lsp_completion.go (ensureWidgetRegistry)", "insight": "**The LSP registry also feeds diagnostics** (lsp_diagnostics.go calls ensureWidgetRegistry), so it is not just thin completions: route it through executor.LoadWidgetRegistry, the SAME registry check validates against (refresh + .mpk property enrichment), rather than LoadProjectWidgetDefinitions — otherwise editor diagnostics and `check` can still disagree on a widget's properties. LoadWidgetRegistry(\"\") skips the global ~/.mxcli/widgets, so keep the old LoadUserDefinitions(\"\") path when no project is open. The registry is cached by sync.Once per server, so a test must use a fresh mdlServer{mprPath: …} over a temp dir holding one .mpk. After this, `grep -rn LoadUserDefinitions` outside widget_registry.go should show only LoadProjectWidgetDefinitions, the page builder and this no-project branch — anything new there is this bug again.", "refs": ["ako/mxcli#663", "mendixlabs/mxcli#1135"]} {"area": "mdl/executor", "symptom": "`DESCRIBE PAGE Administration.Account_New` → `exec` → mx check: **CE1613** \"The selected association 'Administration.UserRoles' no longer exists\" (also User_Language, User_TimeZone, and a DataGrid2 column `Administration.Account.UserRoles/Name`). `check --references` and `exec` both report success; describe → exec → describe is byte-identical", "cause": "A bare association name was qualified with the MODULE of an entity instead of looked up. Administration.Account extends System.User, which declares UserRoles, so the page entity's module named a nonexistent association. The describer always emitted the bare name (shortAttributeName, since 41d01f01); the regression was f0d1aea80 (issuetracker #19), which switched the combobox writer from the option list's module (right here by coincidence) to the page entity's. resolveAssociationAttributePath also qualified every hop against the path's START entity", "file": "`mdl/executor/cmd_pages_builder_input.go` (`resolveAssociationPathIn` → `declaredAssociationQN`), `cmd_pages_builder_v3.go` (`resolveAssociationAttributePath` per-hop context), `cmd_pages_describe_pluggable.go` (`associationRefForContext`)", "insight": "**Two wrong heuristics each fixed the other's case**: 'module of the option list' broke issuetracker #19, 'module of the context entity' broke #662 — both guess a module from an entity name where the model can be asked. Resolve by lookup: the association with that name having an end on the context entity or a generalization, nearest first, qualified with its DECLARING module; ambiguous or unknown keeps the old guess so the validator reports the author's spelling. To find which change regressed it, build the suspect commit and its parent side by side and diff `mx check` on a copy of the project — describe output was identical on all three builds, so the writer changed, not the describer. The bisect subject must be an INHERITED association from a DIFFERENT module whose option list lives in the declaring module; a same-module fixture passes both old rules. A full-project round trip (every page, describe → exec → mx check) found the DataGrid2 `UserRoles/Name` column the issue did not name — the resolver has four call sites, fix it once there", "refs": ["ako/mxcli#662", "issuetracker #19", "ako/mxcli#664"], "ce": ["CE1613"], "date": "2026-09-25"} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`container c1 (dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ])` - the first-class spelling mendixlabs/mxcli#750 proposes - checked clean and `exec` reported `Created page`, but the widget was stored with no DynamicClasses at all. `alter page ... set DynamicClasses = [ ... ] on w` reported `Altered page` and changed nothing; a column's `DynamicCellClass` on ALTER stored `[if$x/Ythen'a'else'b']` as its expression", "cause": "`[ ... ]` parses as propertyValueV3's array alternative, so the value reaches the AST as a []string. The create writers read only a string (GetStringProp for DynamicClasses, `v.(string)` for columnClass); the mutator's dynamicclasses case returned nil when the type check failed; setColumnPropertyMut formatted the list with %v, after the visitor's GetText() had already fused its tokens", "file": "`mdl/executor/validate_widget_expression_list.go` (MDL-WIDGET32), `mdl/backend/pagemutator/mutator.go` (`errExpressionNotAString`)", "insight": "**A proposal's example syntax is worth running through `check` before designing around it** - here the proposed spelling already parsed, and the parse was the bug. Same class as #999 / MDL-WIDGET27 (empty `[]`): a value shape no writer claims. Fix at both ends and they stay in step: a no-project check rule for CREATE, and an error (not a nil return) from the ALTER setter, which `check -p` reports for free because validateAlterSetProperties dry-runs the setter. Key the rule on the property, never on the brackets - `visible: [cond]` is valid MDL. Measured with pre-fix and fixed binaries on copies of ako/TestApp: pre-fix `describe` shows the container without DynamicClasses and the quoted control intact. Tests `validate_widget_expression_list_test.go`, `pagemutator/expression_list_value_test.go`", "refs": ["mendixlabs/mxcli#750", "#999"], "rules": ["MDL-WIDGET32"]} +{"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_Edit takes the page's 8 Enumerations$Condition entries to 0: every widget with Studio Pro's \"Visible: based on attribute value\" becomes ALWAYS visible; check, exec and mx check all report success", "cause": "MDL had no spelling for attribute-based conditional visibility. extractConditionalSettings read only Expression, conditionalVisibilityToGen wrote only Expression, and the settings' list markers were the default [3] where Studio Pro stores Conditions [2] and ModuleRoles [1]", "file": "`mdl/grammar/domains/MDLPage.g4` (`VISIBLE COLON attributePathV3 IN (…)`), `mdl/executor/cmd_pages_builder_visible_when.go` (`applyVisibleWhen`), `mdl/backend/modelsdk/widget_write.go` (`conditionalVisibilityToGen`, TypeDefaults), `mdl/executor/cmd_pages_describe_parse.go` / `_output.go` (`visibleWhenProp`)", "insight": "**A dropped visibility setting is invisible to every check**: the model stays valid, the widget just shows for everyone — count `Enumerations$Condition` in `bson dump --format ndsl` before and after a round trip, since mx check never will. Survey the corpus before designing syntax: all 12 settings in the stock project were attribute-based (booleans and one enum), none role-based or editability, which scoped the feature. Studio Pro stores EVERY value (enum values in declaration order plus \"(empty)\", or true/false) with a flag, so MDL lists only the shown values and the writer fills the rest from the domain model — and a byte-identical before/after diff of the settings block (markers included) on 3 pages is the proof. mdlIdent quotes `empty`/`true`/`false` as keywords; emit them bare in a value list. A new AST property key must be added to the known-property list (validate_widgets.go) or MDL-WIDGET07 falsely warns it is dropped", "refs": ["Administration.Account_Edit", "Administration.ScheduledEvents"], "rules": ["MDL-WIDGET07"], "date": "2026-09-25"} diff --git a/.claude/skills/mendix/create-page/SKILL.md b/.claude/skills/mendix/create-page/SKILL.md index d5bca7e7e8..d4d7d232ed 100644 --- a/.claude/skills/mendix/create-page/SKILL.md +++ b/.claude/skills/mendix/create-page/SKILL.md @@ -288,6 +288,16 @@ dynamictext tTrim (content: 'x', visible: [trim($currentObject/Slug) != '']) textbox txtSlug (label: 'Slug', attribute: Slug, editable: [length(Slug) > 0]) ``` +**Visible based on an attribute value** (Studio Pro's "Visible: based on attribute +value") — list the Boolean/enumeration values that SHOW the widget; `empty` is +"(empty)". Only an attribute of the enclosing data container's own entity: + +```sql +container cntRunning (visible: Status in (Running, empty)) { ... } +textbox txtPassword (label: 'Password', attribute: Password, visible: IsLocalUser in (true)) +``` + + > **`visible:`/`editable:` is a Mendix *expression*, not XPath** — a different > function set from a datasource `where` clause, even though both use `[ ... ]`: > diff --git a/docs-site/src/language/pages.md b/docs-site/src/language/pages.md index 82f0151310..2d49c9c2bc 100644 --- a/docs-site/src/language/pages.md +++ b/docs-site/src/language/pages.md @@ -101,6 +101,19 @@ TEXTBOX txtName (Label: 'Name', Attribute: Name, Visible: [IsActive]) Static values also work: `Visible: false` hides the widget unconditionally. +Studio Pro's **"based on attribute value"** form lists the values of a Boolean or +enumeration attribute (of the enclosing data container's entity) that show the +widget; `empty` is Studio Pro's "(empty)" choice: + +```sql +CONTAINER cntRunning (Visible: Status in (Running, empty)) { ... } +TEXTBOX txtPassword (Label: 'Password', Attribute: Password, Visible: IsLocalUser in (true)) +``` + +mxcli writes one condition per value of the attribute, as Studio Pro does, so a +value not listed hides the widget. `describe page` emits this form for widgets +set up that way in Studio Pro. + ### Conditional Editability Input widgets can be conditionally editable: diff --git a/mdl-examples/bug-tests/visible-based-on-attribute-value.mdl b/mdl-examples/bug-tests/visible-based-on-attribute-value.mdl new file mode 100644 index 0000000000..0237a16092 --- /dev/null +++ b/mdl-examples/bug-tests/visible-based-on-attribute-value.mdl @@ -0,0 +1,48 @@ +-- ============================================================================ +-- Conditional visibility "based on attribute value" was dropped on round trip +-- ============================================================================ +-- +-- Symptom: describe → exec of Administration.Account_Edit (Administration +-- v4.3.2, Mendix 11.13.0) took the page's 8 Enumerations$Condition entries to +-- 0. Every widget set to "Visible: based on attribute value" in Studio Pro +-- (e.g. show the password fields only when IsLocalUser is true) became ALWAYS +-- visible — and check, exec and mx check all reported success. +-- +-- Cause: MDL had no spelling for the form. describe read only the +-- expression form, and the writer wrote only Expression. +-- +-- Fix: `Visible: Attr in (v1, …)` lists the values that SHOW the widget +-- (`empty` = Studio Pro's "(empty)"); mxcli writes one condition per value of +-- the attribute, as Studio Pro stores it, with Studio Pro's list markers +-- (Conditions [2], ModuleRoles [1]). +-- +-- Verify: exec, then describe → exec (expect "Unchanged page"), then +-- `mxcli docker check` — 0 errors. +-- ============================================================================ + +create enumeration MyFirstModule.JobStatus ( Queued 'Queued', Running 'Running', Done 'Done' ); +/ + +create entity MyFirstModule.Job ( + Title: String(200), + Status: Enumeration(MyFirstModule.JobStatus), + IsUrgent: Boolean default false +); +/ + +create or replace page MyFirstModule.Job_View +( Title: 'Job', Layout: Atlas_Core.Atlas_Default, Params: { $Job: MyFirstModule.Job } ) +{ + dataview dv (datasource: $Job) { + textbox txtTitle (Label: 'Title', Attribute: Title, Visible: IsUrgent in (true)) + container cntActive (Visible: Status in (Queued, Running, empty)) { + dynamictext txtActive (Content: 'In progress') + } + container cntDone (Visible: Status in (Done)) { + dynamictext txtDone (Content: 'Finished') + } + } +} +/ + +describe page MyFirstModule.Job_View; diff --git a/mdl/ast/ast_page_v3.go b/mdl/ast/ast_page_v3.go index b8abe6c07d..1eec4fea4b 100644 --- a/mdl/ast/ast_page_v3.go +++ b/mdl/ast/ast_page_v3.go @@ -217,6 +217,14 @@ type OrderByItemV3 struct { } // ActionV3 represents a V3 action expression. +// VisibleWhenV3 is `Visible: Attr in (v1, v2, …)` — Studio Pro's "Visible: +// based on attribute value". Values are the ones that SHOW the widget; `empty` +// stands for Studio Pro's "(empty)". Stored in Properties["VisibleWhen"]. +type VisibleWhenV3 struct { + Attribute string + Values []string +} + type ActionV3 struct { Type string // "save", "cancel", "close", "delete", "create", "showPage", "microflow", "nanoflow", "openLink", "signOut", "completeTask" Target string // Entity, page, or flow qualified name (for create/showPage/microflow/nanoflow) diff --git a/mdl/backend/modelsdk/widget_write.go b/mdl/backend/modelsdk/widget_write.go index bd297391e3..809d9bac06 100644 --- a/mdl/backend/modelsdk/widget_write.go +++ b/mdl/backend/modelsdk/widget_write.go @@ -14,6 +14,7 @@ import ( "github.com/mendixlabs/mxcli/modelsdk/element" genCw "github.com/mendixlabs/mxcli/modelsdk/gen/customwidgets" genDm "github.com/mendixlabs/mxcli/modelsdk/gen/domainmodels" + genEnum "github.com/mendixlabs/mxcli/modelsdk/gen/enumerations" genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" genTexts "github.com/mendixlabs/mxcli/modelsdk/gen/texts" "github.com/mendixlabs/mxcli/sdk/microflows" @@ -115,11 +116,15 @@ func init() { // Attribute is "" (not null): it is a BY_NAME AttributeIdentifier and Mendix // 11.12's reader rejects a null there (StorageLoadException, "not a valid // AttributeIdentifier"). + // Markers measured on all 12 conditional-visibility settings in a stock + // Administration + Feedback project (Mendix 11.13.0): Conditions is [2] and + // ModuleRoles [1], empty or not. They were written as the default [3]. codec.RegisterTypeDefaults("Forms$ConditionalVisibilitySettings", codec.TypeDefaults{ - NullFields: []string{"SourceVariable"}, - EmptyStringFields: []string{"Attribute"}, - MandatoryLists: []string{"Conditions", "ModuleRoles"}, + NullFields: []string{"SourceVariable"}, + EmptyStringFields: []string{"Attribute"}, + MandatoryListMarkers: map[string]int32{"Conditions": 2, "ModuleRoles": 1}, }) + codec.RegisterListMarker("Enumerations$Condition", 2) codec.RegisterTypeDefaults("Forms$ConditionalEditabilitySettings", codec.TypeDefaults{ NullFields: []string{"SourceVariable"}, EmptyStringFields: []string{"Attribute"}, @@ -985,6 +990,18 @@ func conditionalVisibilityToGen(cvs *pages.ConditionalVisibilitySettings) elemen assignID(g) g.SetExpression(cvs.Expression) g.SetIgnoreSecurity(false) + // "Visible: based on attribute value": the attribute plus one condition per + // value it can hold (ako/mxcli attribute-condition visibility). + if cvs.Attribute != "" { + g.SetAttributeQualifiedName(cvs.Attribute) + for _, c := range cvs.Conditions { + cg := genEnum.NewCondition() + assignID(cg) + cg.SetAttributeValue(c.Value) + cg.SetEditableVisible(c.Visible) + g.AddConditions(cg) + } + } return g } diff --git a/mdl/backend/modelsdk/widget_write_visible_when_test.go b/mdl/backend/modelsdk/widget_write_visible_when_test.go new file mode 100644 index 0000000000..e4a93b21b7 --- /dev/null +++ b/mdl/backend/modelsdk/widget_write_visible_when_test.go @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "testing" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// Studio Pro's stored shape (Administration.Account_Edit, Mendix 11.13.0): +// +// ConditionalVisibilitySettings: Forms$ConditionalVisibilitySettings +// Attribute: "Administration.Account.IsLocalUser" +// Conditions [marker=2]: +// - Enumerations$Condition { AttributeValue: "true", EditableVisible: false } +// - Enumerations$Condition { AttributeValue: "false", EditableVisible: true } +// Expression: "" +// IgnoreSecurity: false +// ModuleRoles [marker=1]: [] +// SourceVariable: null +// +// Markers: Conditions is [2] and ModuleRoles [1] on all 12 conditional +// settings in that project, empty or not; mxcli wrote [3] for both. +func TestConditionalVisibilityToGen_AttributeConditions(t *testing.T) { + d := encodeToD(t, conditionalVisibilityToGen(&pages.ConditionalVisibilitySettings{ + Attribute: "Administration.Account.IsLocalUser", + Conditions: []pages.ValueCondition{ + {Value: "true", Visible: false}, + {Value: "false", Visible: true}, + }, + })) + get := func(k string) any { + for _, e := range d { + if e.Key == k { + return e.Value + } + } + t.Fatalf("key %q missing: %v", k, d) + return nil + } + if get("Attribute") != "Administration.Account.IsLocalUser" { + t.Errorf("Attribute = %v", get("Attribute")) + } + conds, ok := get("Conditions").(bson.A) + if !ok || len(conds) != 3 || conds[0] != int32(2) { + t.Fatalf("Conditions = %#v, want [2, cond, cond]", get("Conditions")) + } + first, _ := conds[1].(bson.D) + want := map[string]any{"$Type": "Enumerations$Condition", "AttributeValue": "true", "EditableVisible": false} + for k, v := range want { + found := false + for _, e := range first { + if e.Key == k { + found = true + if e.Value != v { + t.Errorf("condition[0].%s = %v, want %v", k, e.Value, v) + } + } + } + if !found { + t.Errorf("condition[0] lacks %s: %v", k, first) + } + } + if roles, _ := get("ModuleRoles").(bson.A); len(roles) != 1 || roles[0] != int32(1) { + t.Errorf("ModuleRoles = %#v, want [1]", get("ModuleRoles")) + } +} + +// The expression form keeps its shape, with the corrected empty markers. +func TestConditionalVisibilityToGen_ExpressionMarkers(t *testing.T) { + d := encodeToD(t, conditionalVisibilityToGen(&pages.ConditionalVisibilitySettings{Expression: "$currentObject/ImageB64 != empty"})) + for _, e := range d { + switch e.Key { + case "Conditions": + if a, _ := e.Value.(bson.A); len(a) != 1 || a[0] != int32(2) { + t.Errorf("empty Conditions = %#v, want [2]", e.Value) + } + case "ModuleRoles": + if a, _ := e.Value.(bson.A); len(a) != 1 || a[0] != int32(1) { + t.Errorf("empty ModuleRoles = %#v, want [1]", e.Value) + } + case "Attribute": + if e.Value != "" { + t.Errorf("Attribute = %v, want \"\"", e.Value) + } + } + } +} diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index 9d9d8708d3..67eefb33f5 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -496,6 +496,9 @@ func (pb *pageBuilder) buildWidgetV3(w *ast.WidgetV3) (pages.Widget, error) { // Apply conditional visibility/editability applyConditionalSettings(widget, w) + if err := pb.applyVisibleWhen(widget, w); err != nil { + return nil, err + } return widget, nil } diff --git a/mdl/executor/cmd_pages_builder_visible_when.go b/mdl/executor/cmd_pages_builder_visible_when.go new file mode 100644 index 0000000000..1cefe7fe0a --- /dev/null +++ b/mdl/executor/cmd_pages_builder_visible_when.go @@ -0,0 +1,164 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// emptyConditionValue is how Studio Pro stores an enumeration attribute's +// "(empty)" choice; MDL spells it `empty`. +const emptyConditionValue = "(empty)" + +// applyVisibleWhen writes `Visible: Attr in (v1, …)` — Studio Pro's "Visible: +// based on attribute value". +// +// Studio Pro stores ONE condition PER VALUE of the attribute, flagged visible or +// not: every enumeration value in declaration order plus "(empty)", or "true" +// then "false" for a boolean. Measured on the 12 attribute-based settings in a +// stock Administration v4.3.2 + Feedback v4.0.2 project at 11.13.0 (e.g. +// Administration.ScheduledEvents: Running true; Completed, Error, Stopped, +// (empty) false). MDL names only the values that show the widget, and the full +// list is filled from the domain model. +// +// Without this the setting had no MDL spelling, so describe → exec dropped it +// and the widget became ALWAYS visible — with mx check clean. +func (pb *pageBuilder) applyVisibleWhen(widget pages.Widget, w *ast.WidgetV3) error { + vw, ok := w.Properties["VisibleWhen"].(*ast.VisibleWhenV3) + if !ok || vw == nil { + return nil + } + type baseWidgetGetter interface { + GetBaseWidget() *pages.BaseWidget + } + bwg, ok := widget.(baseWidgetGetter) + if !ok { + return mdlerrors.NewValidationf("%s %s: `Visible: %s in (…)` is not supported on this widget", w.Type, w.Name, vw.Attribute) + } + where := fmt.Sprintf("%s %s: Visible: %s in (…)", w.Type, w.Name, vw.Attribute) + if pb.entityContext == "" { + return mdlerrors.NewValidationf("%s: the attribute is read from the enclosing data container's object — place the widget inside a data container", where) + } + if strings.ContainsAny(vw.Attribute, "/.") { + return mdlerrors.NewValidationf("%s: name an attribute of the data container's own entity (%s); association paths are not supported", where, pb.entityContext) + } + + declaring, ok := pb.declaringEntityFor(pb.entityContext, vw.Attribute) + if !ok { + return mdlerrors.NewValidationf("%s: %s has no attribute %s", where, pb.entityContext, vw.Attribute) + } + attrQN := declaring + "." + vw.Attribute + + var all []string + switch t := pb.findAttributeType(attrQN).(type) { + case *domainmodel.BooleanAttributeType: + all = []string{"true", "false"} + case *domainmodel.EnumerationAttributeType: + values, err := pb.enumerationValueNames(t.EnumerationRef) + if err != nil { + return mdlerrors.NewValidationf("%s: %v", where, err) + } + all = append(values, emptyConditionValue) + default: + return mdlerrors.NewValidationf("%s: %s is not a Boolean or enumeration attribute — use an expression instead: `Visible: [...]`", where, attrQN) + } + + visible := map[string]bool{} + for _, v := range vw.Values { + stored := v + if strings.EqualFold(v, "empty") { + stored = emptyConditionValue + } + idx := indexFold(all, stored) + if idx < 0 { + return mdlerrors.NewValidationf("%s: %q is not a value of %s (values: %s)", where, v, attrQN, + strings.Join(mdlValueNames(all), ", ")) + } + visible[all[idx]] = true + } + + conds := make([]pages.ValueCondition, 0, len(all)) + for _, v := range all { + conds = append(conds, pages.ValueCondition{Value: v, Visible: visible[v]}) + } + bwg.GetBaseWidget().ConditionalVisibility = &pages.ConditionalVisibilitySettings{ + BaseElement: model.BaseElement{ + ID: model.ID(types.GenerateID()), + TypeName: "Forms$ConditionalVisibilitySettings", + }, + Attribute: attrQN, + Conditions: conds, + } + return nil +} + +func indexFold(values []string, v string) int { + for i, x := range values { + if strings.EqualFold(x, v) { + return i + } + } + return -1 +} + +// mdlValueNames lists condition values as MDL spells them. +func mdlValueNames(values []string) []string { + out := make([]string, len(values)) + for i, v := range values { + if v == emptyConditionValue { + v = "empty" + } + out[i] = v + } + return out +} + +// enumerationValueNames returns an enumeration's value names in declaration +// order. +func (pb *pageBuilder) enumerationValueNames(enumQN string) ([]string, error) { + enums, err := pb.getEnumerations() + if err != nil { + return nil, err + } + h, err := pb.getHierarchy() + if err != nil { + return nil, err + } + for _, e := range enums { + if h.GetModuleName(h.FindModuleID(e.ContainerID))+"."+e.Name != enumQN { + continue + } + names := make([]string, 0, len(e.Values)) + for _, v := range e.Values { + names = append(names, v.Name) + } + return names, nil + } + return nil, fmt.Errorf("enumeration %s not found", enumQN) +} + +// getEnumerations returns cached enumerations or loads them. +func (pb *pageBuilder) getEnumerations() ([]*model.Enumeration, error) { + if pb.execCache != nil && pb.execCache.enumerations != nil { + return pb.execCache.enumerations, nil + } + if pb.backend == nil { + return nil, fmt.Errorf("no project loaded") + } + enums, err := pb.backend.ListEnumerations() + if err != nil { + return nil, err + } + if pb.execCache != nil { + pb.execCache.enumerations = enums + } + return enums, nil +} diff --git a/mdl/executor/cmd_pages_describe.go b/mdl/executor/cmd_pages_describe.go index b008a2222d..1495028755 100644 --- a/mdl/executor/cmd_pages_describe.go +++ b/mdl/executor/cmd_pages_describe.go @@ -715,8 +715,12 @@ type rawWidget struct { // identity is a single field DESCRIBE has to put back. Specialization string // Conditional visibility/editability - VisibleIf string // Expression from ConditionalVisibilitySettings - EditableIf string // Expression from ConditionalEditabilitySettings + VisibleIf string // Expression from ConditionalVisibilitySettings + // "Visible: based on attribute value": the attribute's short name and the + // values that SHOW the widget, as MDL spells them (`empty` for "(empty)"). + VisibleAttr string + VisibleValues []string + EditableIf string // Expression from ConditionalEditabilitySettings // Design properties from Appearance DesignProperties []rawDesignProp // Explicit widget properties (for generic PLUGGABLEWIDGET output) diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index c0c5c0b5e4..f6ddb814a0 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -121,6 +121,9 @@ func appendConditionalProps(props []string, w rawWidget) []string { if w.VisibleIf != "" { props = append(props, fmt.Sprintf("Visible: [%s]", w.VisibleIf)) } + if prop := visibleWhenProp(w); prop != "" { + props = append(props, prop) + } if w.EditableIf != "" { props = append(props, fmt.Sprintf("Editable: [%s]", w.EditableIf)) } @@ -186,6 +189,9 @@ func appendAppearanceProps(props []string, w rawWidget) []string { if w.VisibleIf != "" { props = append(props, fmt.Sprintf("Visible: [%s]", w.VisibleIf)) } + if prop := visibleWhenProp(w); prop != "" { + props = append(props, prop) + } if w.EditableIf != "" { props = append(props, fmt.Sprintf("Editable: [%s]", w.EditableIf)) } @@ -1978,3 +1984,30 @@ func describeImageWidgetProps(w rawWidget) []string { } return props } + +// visibleWhenProp renders "Visible: based on attribute value" as +// `Visible: Attr in (v1, …)`, naming the values that show the widget. +// +// A setting that shows the widget for NO value has no spelling in that form +// (the list would be empty); Studio Pro allows it, so say what it is rather +// than drop it — a dropped setting is an always-visible widget. +func visibleWhenProp(w rawWidget) string { + if w.VisibleAttr == "" { + return "" + } + if len(w.VisibleValues) == 0 { + return "-- NOT re-executable: visible for no value of " + w.VisibleAttr + + " (never shown) — MDL cannot spell an empty value list, so re-running this script would make it always visible" + } + vals := make([]string, len(w.VisibleValues)) + for i, v := range w.VisibleValues { + // `empty` (Studio Pro's "(empty)") and the boolean values are keywords + // the value list takes bare; quoting them would read as identifiers. + if v == "empty" || v == "true" || v == "false" { + vals[i] = v + continue + } + vals[i] = mdlIdent(v) + } + return fmt.Sprintf("Visible: %s in (%s)", mdlIdent(w.VisibleAttr), strings.Join(vals, ", ")) +} diff --git a/mdl/executor/cmd_pages_describe_parse.go b/mdl/executor/cmd_pages_describe_parse.go index 4b388b2471..5ddf678c8a 100644 --- a/mdl/executor/cmd_pages_describe_parse.go +++ b/mdl/executor/cmd_pages_describe_parse.go @@ -32,6 +32,26 @@ func extractConditionalSettings(widget *rawWidget, w map[string]any) { if expr, ok := cvs["Expression"].(string); ok && expr != "" { widget.VisibleIf = expr } + // Attribute-based: one Enumerations$Condition per value. Without this the + // setting was never read, so describe → exec wrote the widget always + // visible (Administration.Account_Edit: 8 conditions → 0). + if attr, ok := cvs["Attribute"].(string); ok && attr != "" { + widget.VisibleAttr = shortAttributeName(attr) + for _, c := range getBsonArrayElements(cvs["Conditions"]) { + cm, ok := c.(map[string]any) + if !ok { + continue + } + if shown, _ := cm["EditableVisible"].(bool); !shown { + continue + } + v, _ := cm["AttributeValue"].(string) + if v == emptyConditionValue { + v = "empty" + } + widget.VisibleValues = append(widget.VisibleValues, v) + } + } } if ces, ok := w["ConditionalEditabilitySettings"].(map[string]any); ok && ces != nil { if expr, ok := ces["Expression"].(string); ok && expr != "" { diff --git a/mdl/executor/cmd_pages_visible_when_test.go b/mdl/executor/cmd_pages_visible_when_test.go new file mode 100644 index 0000000000..dd2a4e1fde --- /dev/null +++ b/mdl/executor/cmd_pages_visible_when_test.go @@ -0,0 +1,152 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "context" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/domainmodel" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// A model with a boolean and an enumeration attribute, the second inherited — +// the Account_Overview shape (System.User.Active bound from a specialization). +func visibleWhenPB(entityContext string) *pageBuilder { + const ( + modID = model.ID("mod-m") + baseID = model.ID("e-base") + subID = model.ID("e-sub") + ) + return &pageBuilder{ + entityContext: entityContext, + paramEntityNames: map[string]string{}, + widgetScope: map[string]model.ID{}, + execCache: &executorCache{ + hierarchy: &ContainerHierarchy{moduleNames: map[model.ID]string{modID: "M"}}, + domainModels: []*domainmodel.DomainModel{{ + ContainerID: modID, + Entities: []*domainmodel.Entity{ + {BaseElement: model.BaseElement{ID: baseID}, Name: "Job", Attributes: []*domainmodel.Attribute{ + {Name: "Status", Type: &domainmodel.EnumerationAttributeType{EnumerationRef: "M.EventStatus"}}, + {Name: "IsLocal", Type: &domainmodel.BooleanAttributeType{}}, + {Name: "Title", Type: &domainmodel.StringAttributeType{}}, + }}, + {BaseElement: model.BaseElement{ID: subID}, Name: "SpecialJob", GeneralizationRef: "M.Job"}, + }, + }}, + enumerations: []*model.Enumeration{{ + ContainerID: modID, Name: "EventStatus", + Values: []model.EnumerationValue{{Name: "Running"}, {Name: "Completed"}, {Name: "Error"}}, + }}, + }, + } +} + +func buildVisibleWhen(t *testing.T, pb *pageBuilder, attr string, values ...string) (*pages.ConditionalVisibilitySettings, error) { + t.Helper() + w := &ast.WidgetV3{Type: "container", Name: "c1", Properties: map[string]any{ + "VisibleWhen": &ast.VisibleWhenV3{Attribute: attr, Values: values}, + }} + built, err := pb.buildWidgetV3(w) + if err != nil { + return nil, err + } + return built.(*pages.Container).ConditionalVisibility, nil +} + +// Studio Pro stores EVERY value with its flag, in enumeration order, plus +// "(empty)" for an enumeration — measured on Administration.ScheduledEvents +// (Running true; Completed, Error, Stopped, (empty) false). A boolean gets +// "true" then "false" (Administration.Account_Edit, FeedbackModule.ShareFeedback). +func TestVisibleWhen_WritesEveryValue(t *testing.T) { + cvs, err := buildVisibleWhen(t, visibleWhenPB("M.SpecialJob"), "Status", "Running", "empty") + if err != nil { + t.Fatalf("build: %v", err) + } + if cvs == nil { + t.Fatal("no ConditionalVisibilitySettings — the condition was dropped (widget always visible)") + } + if cvs.Attribute != "M.Job.Status" { + t.Errorf("Attribute = %q, want M.Job.Status (the DECLARING entity)", cvs.Attribute) + } + got := []string{} + for _, c := range cvs.Conditions { + got = append(got, c.Value+"="+map[bool]string{true: "T", false: "F"}[c.Visible]) + } + if want := "Running=T Completed=F Error=F (empty)=T"; strings.Join(got, " ") != want { + t.Errorf("Conditions = %s, want %s", strings.Join(got, " "), want) + } + if cvs.Expression != "" { + t.Errorf("Expression = %q, want empty", cvs.Expression) + } + + cvs, err = buildVisibleWhen(t, visibleWhenPB("M.Job"), "IsLocal", "false") + if err != nil { + t.Fatalf("build: %v", err) + } + got = got[:0] + for _, c := range cvs.Conditions { + got = append(got, c.Value+"="+map[bool]string{true: "T", false: "F"}[c.Visible]) + } + if want := "true=F false=T"; strings.Join(got, " ") != want { + t.Errorf("boolean Conditions = %s, want %s", strings.Join(got, " "), want) + } +} + +func TestVisibleWhen_Refusals(t *testing.T) { + for _, tc := range []struct { + name, ctx, attr string + values []string + want string + }{ + {"unknown value", "M.Job", "Status", []string{"Paused"}, "Paused"}, + {"not boolean or enum", "M.Job", "Title", []string{"x"}, "Boolean or enumeration"}, + {"unknown attribute", "M.Job", "Nope", []string{"true"}, "Nope"}, + {"no data container", "", "IsLocal", []string{"true"}, "data container"}, + {"empty on a boolean", "M.Job", "IsLocal", []string{"empty"}, "empty"}, + } { + t.Run(tc.name, func(t *testing.T) { + _, err := buildVisibleWhen(t, visibleWhenPB(tc.ctx), tc.attr, tc.values...) + if err == nil || !strings.Contains(err.Error(), tc.want) { + t.Errorf("err = %v, want it to mention %q", err, tc.want) + } + }) + } +} + +// DESCRIBE reads the stored conditions back as the visible values. +func TestDescribe_VisibleWhen(t *testing.T) { + stored := map[string]any{ + "$Type": "Forms$DivContainer", + "Name": "c1", + "ConditionalVisibilitySettings": map[string]any{ + "$Type": "Forms$ConditionalVisibilitySettings", + "Attribute": "System.ScheduledEventInformation.Status", + "Conditions": []any{int32(2), + map[string]any{"$Type": "Enumerations$Condition", "AttributeValue": "Running", "EditableVisible": true}, + map[string]any{"$Type": "Enumerations$Condition", "AttributeValue": "Completed", "EditableVisible": false}, + map[string]any{"$Type": "Enumerations$Condition", "AttributeValue": "(empty)", "EditableVisible": true}, + }, + "Expression": "", + }, + "Widgets": []any{int32(2)}, + } + var buf bytes.Buffer + ctx := (&Executor{}).newExecContext(context.Background()) + ctx.Output = &buf + raw := parseRawWidget(ctx, stored) + outputWidgetMDLV3(ctx, raw[0], 1) + got := buf.String() + if !strings.Contains(got, `Visible: "Status" in (Running, empty)`) { // Status is a keyword, so quoted + t.Errorf("describe output lacks the attribute condition:\n%s", got) + } + if _, errs := visitor.Build("create page M.P (Title: 'x', Layout: A.L) {\n" + got + "}\n"); len(errs) > 0 { + t.Errorf("describe output does not parse: %v\n%s", errs, got) + } +} diff --git a/mdl/executor/executor.go b/mdl/executor/executor.go index adcb6d59b0..0bd056918b 100644 --- a/mdl/executor/executor.go +++ b/mdl/executor/executor.go @@ -30,6 +30,7 @@ type executorCache struct { units []*types.UnitInfo folders []*types.FolderInfo domainModels []*domainmodel.DomainModel + enumerations []*model.Enumeration hierarchy *ContainerHierarchy // pages, layouts, microflows are cached separately as they may change during execution diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index 4a7b61462c..65fed81533 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -679,7 +679,7 @@ var staticWidgetKnownProps = func() map[string]bool { // keys the builders/visitor consume and the conditional-binding metadata "CaptionAttribute", "Collapsible", "DatabaseHost", "DefaultLanguage", "Footer", "FormOrientation", "HeaderMode", "LabelWidth", "Prefix", "ShowContentAs", "Title", - "Widget", "WidgetType", "ShowLabel", "VisibleIf", "EditableIf", "DynamicClasses", + "Widget", "WidgetType", "ShowLabel", "VisibleIf", "VisibleWhen", "EditableIf", "DynamicClasses", // vocabulary describe page emits (native widgets + datagrid columns) "Alignment", "AlternativeText", "ColumnClass", "ColumnWidth", "DesktopColumns", "DisplayAs", "Draggable", "DynamicCellClass", "HeightUnit", "Hidable", "ImageType", diff --git a/mdl/grammar/domains/MDLPage.g4 b/mdl/grammar/domains/MDLPage.g4 index d2d6190e00..090c160af9 100644 --- a/mdl/grammar/domains/MDLPage.g4 +++ b/mdl/grammar/domains/MDLPage.g4 @@ -512,6 +512,7 @@ widgetPropertyV3 | WIDTH COLON NUMBER_LITERAL // Width: 200 | HEIGHT COLON NUMBER_LITERAL // Height: 100 | VISIBLE COLON xpathConstraint // Visible: [IsActive = true] + | VISIBLE COLON attributePathV3 IN LPAREN visibleValueV3 (COMMA visibleValueV3)* RPAREN // Visible: Status in (Running, empty) | VISIBLE COLON propertyValueV3 // Visible: false | EDITABLE COLON xpathConstraint // Editable: [Status != 'Closed'] | EDITABLE COLON propertyValueV3 // Editable: Never | Always @@ -658,6 +659,12 @@ microflowArgV3 | VARIABLE EQUALS expression // $Param = $value (microflow-style, also accepted) ; +// A value in `Visible: Attr in (…)`: an enumeration value name, true/false, +// or `empty` for Studio Pro's "(empty)". +visibleValueV3 + : IDENTIFIER | QUOTED_IDENTIFIER | keyword + ; + // V3 Attribute path: Name, Product/Category, "Order" (quoted to escape reserved words) attributePathV3 : (IDENTIFIER | QUOTED_IDENTIFIER | keyword) (SLASH (IDENTIFIER | QUOTED_IDENTIFIER | keyword))* diff --git a/mdl/visitor/visitor_page_v3.go b/mdl/visitor/visitor_page_v3.go index faac425e84..9063997279 100644 --- a/mdl/visitor/visitor_page_v3.go +++ b/mdl/visitor/visitor_page_v3.go @@ -831,6 +831,15 @@ func parseWidgetPropertyV3(ctx parser.IWidgetPropertyV3Context, widget *ast.Widg // Visible: [expression] (conditional visibility) or Visible: false (static) if propCtx.VISIBLE() != nil { + // `Visible: Attr in (v1, …)` — Studio Pro's "based on attribute value". + if propCtx.IN() != nil { + vw := &ast.VisibleWhenV3{Attribute: buildAttributePathV3(propCtx.AttributePathV3())} + for _, v := range propCtx.AllVisibleValueV3() { + vw.Values = append(vw.Values, unquoteIdentifier(v.GetText())) + } + widget.Properties["VisibleWhen"] = vw + return + } if xc := propCtx.XpathConstraint(); xc != nil { widget.Properties["VisibleIf"] = buildConditionalExpression(xc) } else if valCtx := propCtx.PropertyValueV3(); valCtx != nil { diff --git a/mdl/visitor/visitor_visible_when_test.go b/mdl/visitor/visitor_visible_when_test.go new file mode 100644 index 0000000000..41a266ebc3 --- /dev/null +++ b/mdl/visitor/visitor_visible_when_test.go @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "reflect" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// Studio Pro's "Visible: based on attribute value" — stored as a +// ConditionalVisibilitySettings with an Attribute and one Enumerations$Condition +// per value — had no MDL spelling, so describe → exec dropped it and the widget +// became always visible (Administration.Account_Edit: 8 conditions → 0, with +// mx check clean). `Visible: Attr in (values)` names the values that show it. +func TestVisibleWhenAttributeIn(t *testing.T) { + for _, tc := range []struct { + src string + attr string + values []string + }{ + {"textbox tb (Attribute: Name, Visible: IsLocalUser in (false))", "IsLocalUser", []string{"false"}}, + {"container c (Visible: Status in (Running, empty)) { dynamictext t (Content: 'x') }", "Status", []string{"Running", "empty"}}, + {"container c (Visible: \"Type\" in (Completed)) { dynamictext t (Content: 'x') }", "Type", []string{"Completed"}}, + } { + prog, errs := Build("create page M.P (Title: 'x', Layout: A.L) { " + tc.src + " };") + if len(errs) > 0 { + t.Fatalf("%s: parse: %v", tc.src, errs) + } + w := prog.Statements[0].(*ast.CreatePageStmtV3).Widgets[0] + vw, ok := w.Properties["VisibleWhen"].(*ast.VisibleWhenV3) + if !ok { + t.Fatalf("%s: VisibleWhen = %T (%v)", tc.src, w.Properties["VisibleWhen"], w.Properties) + } + if vw.Attribute != tc.attr || !reflect.DeepEqual(vw.Values, tc.values) { + t.Errorf("%s: got %s in %v, want %s in %v", tc.src, vw.Attribute, vw.Values, tc.attr, tc.values) + } + } +} + +// The existing forms are unchanged. +func TestVisibleExpressionFormsUnchanged(t *testing.T) { + prog, errs := Build("create page M.P (Title: 'x', Layout: A.L) { textbox tb (Attribute: Name, Visible: [IsActive = true]) };") + if len(errs) > 0 { + t.Fatalf("parse: %v", errs) + } + w := prog.Statements[0].(*ast.CreatePageStmtV3).Widgets[0] + if _, ok := w.Properties["VisibleIf"]; !ok { + t.Errorf("bracketed Visible no longer lands in VisibleIf: %v", w.Properties) + } + if _, ok := w.Properties["VisibleWhen"]; ok { + t.Errorf("bracketed Visible parsed as an attribute condition") + } +} diff --git a/sdk/pages/pages_widgets.go b/sdk/pages/pages_widgets.go index bcef3fa28e..67b0d87f65 100644 --- a/sdk/pages/pages_widgets.go +++ b/sdk/pages/pages_widgets.go @@ -96,7 +96,19 @@ type ConditionalVisibilitySettings struct { Expression string `json:"expression,omitempty"` ModuleRoles []model.ID `json:"moduleRoles,omitempty"` SourceVariable *PageVariable `json:"sourceVariable,omitempty"` - Attribute model.ID `json:"attribute,omitempty"` + // Attribute is Studio Pro's "based on attribute value": the qualified + // Module.Entity.Attr (a BY_NAME reference), with one Conditions entry per + // value of it. Empty for the expression and module-role forms. + Attribute string `json:"attribute,omitempty"` + Conditions []ValueCondition `json:"conditions,omitempty"` +} + +// ValueCondition is one Enumerations$Condition: whether the widget is +// visible (or editable) when the attribute holds Value. Studio Pro stores one +// per value — every enumeration value plus "(empty)", or "true"/"false". +type ValueCondition struct { + Value string `json:"value"` + Visible bool `json:"visible"` } // ConditionalEditabilitySettings represents editability conditions. From 61c2b33f1b022b7f39a1a6981cf47be2deba73d3 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 08:29:05 +0000 Subject: [PATCH 26/47] feat(odata): ServiceUrl names a constant, like ProxyHost Studio Pro picks an OData client's service URL as a constant (CE6825: "'Service url' must be a constant") and stores the reference in HttpConfiguration.CustomLocation as `@Module.Name` (ako/TestApp Odata.Bug1073: "@Odata.Bug1073_Location"). So ServiceUrl is a reference, not an expression, and MDL now writes it the way it writes ProxyHost: - `ServiceUrl: Module.Location` is accepted alongside `@Module.Location` and `'@Module.Location'`; before, the bare name was refused as "not a constant reference". serviceURLConstant normalizes all three to the stored `@Module.Location` on create, create-or-modify and alter, and still refuses a literal URL. - describe prints the bare name (`ServiceUrl: Module.Location`), as it prints `ProxyHost: Odata.Bug1073_ProxyHost`; a stored value that is not `@Module.Name` keeps the quoted form. Tests failed first with the pre-change rule: the bare spelling was refused on create and alter and describe printed the quoted form; the `@` and quoted-`@` spellings passed as controls. Measured on a copy of ako/TestApp: `ServiceUrl: SU.Loc` stores CustomLocation "@SU.Loc", and describe -> exec stores the identical value. The OData skill, walkthrough, quick reference, case study and syntax help show the bare name; the bug-test examples keep the other spellings, which remain accepted. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../skills/mendix/odata-data-sharing/SKILL.md | 6 +- .../reference/walkthroughs.md | 8 +- CHANGELOG.md | 3 +- cmd/mxcli/syntax/features_integration.go | 4 +- .../CASE_STUDY_MxGraphStudioDemo.md | 2 +- docs/01-project/MDL_QUICK_REFERENCE.md | 2 +- .../PROPOSAL_first_class_expressions.md | 2 +- .../750-odata-serviceurl-constant-name.mdl | 39 +++++ mdl/executor/cmd_odata.go | 56 ++++--- .../cmd_odata_client_describe_quoting_test.go | 8 +- .../cmd_odata_service_url_constant_test.go | 140 ++++++++++++++++++ 11 files changed, 238 insertions(+), 32 deletions(-) create mode 100644 mdl-examples/bug-tests/750-odata-serviceurl-constant-name.mdl create mode 100644 mdl/executor/cmd_odata_service_url_constant_test.go diff --git a/.claude/skills/mendix/odata-data-sharing/SKILL.md b/.claude/skills/mendix/odata-data-sharing/SKILL.md index c1b5a7a882..2dfa92d948 100644 --- a/.claude/skills/mendix/odata-data-sharing/SKILL.md +++ b/.claude/skills/mendix/odata-data-sharing/SKILL.md @@ -65,7 +65,7 @@ CREATE OR MODIFY ODATA CLIENT F1Now.NowApi ( ODataVersion: OData4, MetadataUrl: './contracts/live-now-metadata.xml', Timeout: 300, - ServiceUrl: '@F1Now.ApiLocation' + ServiceUrl: F1Now.ApiLocation ); ``` @@ -104,7 +104,7 @@ CREATE CONSTANT ProductClient.ProductDataApiLocation CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( ODataVersion: OData4, MetadataUrl: 'https://api.example.com/$metadata', - ServiceUrl: '@ProductClient.ProductDataApiLocation' -- ✅ Constant reference + ServiceUrl: ProductClient.ProductDataApiLocation -- ✅ Constant reference ); ``` @@ -270,7 +270,7 @@ Before consuming: - HTTP(S) URL: `https://api.example.com/$metadata` - Local file (absolute): `file:///path/to/metadata.xml` - Local file (relative): `./metadata/service.xml` (resolved against `.mpr` directory) -- [ ] OData client uses `ServiceUrl: '@Module.Constant'` for runtime endpoint +- [ ] OData client uses `ServiceUrl: Module.Constant` for runtime endpoint - [ ] External entities match the published exposed names and types - [ ] Module role created and granted on external entities (READ, optionally CREATE/WRITE/DELETE) diff --git a/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md b/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md index 29d18996b1..4245647149 100644 --- a/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md +++ b/.claude/skills/mendix/odata-data-sharing/reference/walkthroughs.md @@ -184,7 +184,7 @@ create odata client ProductClient.ProductDataApiClient ( ODataVersion: OData4, MetadataUrl: 'http://localhost:8080/odata/productdataapi/v1/$metadata', timeout: 300, - ServiceUrl: '@ProductClient.ProductDataApiLocation', + ServiceUrl: ProductClient.ProductDataApiLocation, UseAuthentication: Yes, -- HttpUsername/HttpPassword hold a Mendix expression, written as-is: -- 'MxAdmin' is the string, @ProductClient.ApiPassword (no quotes) reads a @@ -199,7 +199,7 @@ CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( ODataVersion: OData4, MetadataUrl: './metadata/productdataapi.xml', Timeout: 300, - ServiceUrl: '@ProductClient.ProductDataApiLocation', + ServiceUrl: ProductClient.ProductDataApiLocation, UseAuthentication: Yes, HttpUsername: 'MxAdmin', HttpPassword: '1' @@ -210,7 +210,7 @@ CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( ODataVersion: OData4, MetadataUrl: 'metadata/productdataapi.xml', Timeout: 300, - ServiceUrl: '@ProductClient.ProductDataApiLocation', + ServiceUrl: ProductClient.ProductDataApiLocation, UseAuthentication: Yes, HttpUsername: 'MxAdmin', HttpPassword: '1' @@ -221,7 +221,7 @@ CREATE ODATA CLIENT ProductClient.ProductDataApiClient ( ODataVersion: OData4, MetadataUrl: 'file:///Users/team/contracts/productdataapi.xml', Timeout: 300, - ServiceUrl: '@ProductClient.ProductDataApiLocation', + ServiceUrl: ProductClient.ProductDataApiLocation, UseAuthentication: Yes, HttpUsername: 'MxAdmin', HttpPassword: '1' diff --git a/CHANGELOG.md b/CHANGELOG.md index a487522972..c2e278d325 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Changed -- **An OData client's credentials and header values are written as Mendix expressions** (mendixlabs/mxcli#750) — `HttpUsername`, `HttpPassword`, `ClientCertificate` and every `headers (…)` value hold an expression, and MDL now writes it as-is: `HttpUsername: 'admin'` is the string `'admin'`, `@Module.Const` reads a constant, and `'Bearer ' + @Module.Token` concatenates. Before, a quoted value was the expression's *text*, so `'admin'` stored the identifier `admin` and a string needed `'''admin'''`. `describe` prints the stored expression as-is, so Studio Pro's `'abc'` now reads `HttpUsername: 'abc'`; measured against a Studio Pro-authored client, and a describe → exec round trip stores identical values. **Migrating a script:** `'''admin'''` becomes `'admin'`, and a quoted constant `'@Module.Const'` becomes `@Module.Const` — both old forms still parse but would now store something else, so `check` and `exec` refuse them as **MDL-ODATA07**. A compound expression in any other OData property (`Path: 'a' + 'b'`) is an error rather than an empty value. `ServiceUrl` is unchanged. +- **An OData client's credentials and header values are written as Mendix expressions** (mendixlabs/mxcli#750) — `HttpUsername`, `HttpPassword`, `ClientCertificate` and every `headers (…)` value hold an expression, and MDL now writes it as-is: `HttpUsername: 'admin'` is the string `'admin'`, `@Module.Const` reads a constant, and `'Bearer ' + @Module.Token` concatenates. Before, a quoted value was the expression's *text*, so `'admin'` stored the identifier `admin` and a string needed `'''admin'''`. `describe` prints the stored expression as-is, so Studio Pro's `'abc'` now reads `HttpUsername: 'abc'`; measured against a Studio Pro-authored client, and a describe → exec round trip stores identical values. **Migrating a script:** `'''admin'''` becomes `'admin'`, and a quoted constant `'@Module.Const'` becomes `@Module.Const` — both old forms still parse but would now store something else, so `check` and `exec` refuse them as **MDL-ODATA07**. A compound expression in any other OData property (`Path: 'a' + 'b'`) is an error rather than an empty value. `ServiceUrl` is a constant reference, not an expression — see the next entry. +- **An OData client's `ServiceUrl` names a constant, like `ProxyHost`** (mendixlabs/mxcli#750) — Studio Pro picks the service URL as a constant and stores it as `@Module.Name`. `ServiceUrl: Module.Location` is now accepted alongside `@Module.Location` and `'@Module.Location'` (the bare name used to be refused as "not a constant reference"); all three store the same value, and `describe` prints the bare name, as it does for the proxy references. A literal URL is still refused (CE6825). ### Fixed diff --git a/cmd/mxcli/syntax/features_integration.go b/cmd/mxcli/syntax/features_integration.go index 622e29efd1..cb98e9b715 100644 --- a/cmd/mxcli/syntax/features_integration.go +++ b/cmd/mxcli/syntax/features_integration.go @@ -44,7 +44,7 @@ func init() { " ODataVersion: OData4,\n" + " MetadataUrl: 'https://.../$metadata',\n" + " Timeout: 300,\n" + - " ServiceUrl: @Module.ServiceUrlConstant, -- must be a constant ref\n" + + " ServiceUrl: Module.ServiceUrlConstant, -- must be a constant ref\n" + " -- Configuration source dropdown — pick ONE: constants only (omit\n" + " -- both microflows), Configuration microflow, OR Headers microflow.\n" + " -- Both MDL keywords map to the same BSON field; Studio Pro picks\n" + @@ -61,7 +61,7 @@ func init() { " (Attr: Type, ...);\n\n" + "CREATE EXTERNAL ENTITIES FROM Module.Client\n" + " [INTO Module] [ENTITIES (Name1, Name2)];", - Example: "CREATE CONSTANT MyModule.SvcUrl TYPE String DEFAULT 'https://api.example.com/odata/v4/';\n\nCREATE ODATA CLIENT MyModule.SalesforceAPI (\n Version: '1.0',\n ODataVersion: OData4,\n MetadataUrl: 'https://api.example.com/odata/$metadata',\n Timeout: 300,\n ServiceUrl: @MyModule.SvcUrl\n);\n\nCREATE EXTERNAL ENTITIES FROM MyModule.SalesforceAPI INTO Integration;", + Example: "CREATE CONSTANT MyModule.SvcUrl TYPE String DEFAULT 'https://api.example.com/odata/v4/';\n\nCREATE ODATA CLIENT MyModule.SalesforceAPI (\n Version: '1.0',\n ODataVersion: OData4,\n MetadataUrl: 'https://api.example.com/odata/$metadata',\n Timeout: 300,\n ServiceUrl: MyModule.SvcUrl\n);\n\nCREATE EXTERNAL ENTITIES FROM MyModule.SalesforceAPI INTO Integration;", SeeAlso: []string{"odata", "odata.publish", "odata.show"}, }) diff --git a/docs/01-project/CASE_STUDY_MxGraphStudioDemo.md b/docs/01-project/CASE_STUDY_MxGraphStudioDemo.md index 1b75abc8af..2256c36078 100644 --- a/docs/01-project/CASE_STUDY_MxGraphStudioDemo.md +++ b/docs/01-project/CASE_STUDY_MxGraphStudioDemo.md @@ -110,7 +110,7 @@ create odata client OdataPlm.MxPlmOdataApiClient ( ODataVersion: OData4, MetadataUrl: 'https://graphstudio.mendixdemo.com/dataondemand/Mx-PLM-example/MxPlmExample/$metadata', timeout: 300, - ServiceUrl: '@OdataPlm.MxPlmOdataApiClient_Location', + ServiceUrl: OdataPlm.MxPlmOdataApiClient_Location, UseAuthentication: Yes, HttpUsername: @Main.MxPlmGraphClient_username, HttpPassword: @Main.MxPlmGraphClient_password diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index 67ca255ca7..3ecf990031 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -377,7 +377,7 @@ CREATE ODATA CLIENT MyModule.LocalService2 ( ODataVersion: OData4, MetadataUrl: './metadata/service.xml', Timeout: 300, - ServiceUrl: '@MyModule.ServiceLocation' -- Must be a constant reference + ServiceUrl: MyModule.ServiceLocation -- Must be a constant reference ); ``` diff --git a/docs/11-proposals/PROPOSAL_first_class_expressions.md b/docs/11-proposals/PROPOSAL_first_class_expressions.md index dc9a87ec68..09f4e4b0a4 100644 --- a/docs/11-proposals/PROPOSAL_first_class_expressions.md +++ b/docs/11-proposals/PROPOSAL_first_class_expressions.md @@ -533,7 +533,7 @@ the target type to resolve against, which a bare qualified name does not. | Slot | Metamodel | Kind | Today | Status | |---|---|---|---|---| -| OData client `ServiceUrl` | `ConsumedODataService.serviceUrl` `Primitive[string]` | expression | `@Mod.C`, also `'@Mod.C'` | correct; stays `@` | +| OData client `ServiceUrl` | `HttpConfiguration.CustomLocation` `Primitive[string]`, always `@Module.Name` | reference (decided 2026-09-25: Studio Pro picks it as a constant, CE6825) | bare, `@Mod.C`, `'@Mod.C'`; stored `@Mod.C` | **done**: written and described like `ProxyHost` | | OData client `ProxyHost` / `ProxyPort` / `ProxyUsername` / `ProxyPassword` | `ByNameRef` (`rest/types.go`) | reference | `@Mod.C` → stored `"@Mod.C"` verbatim by `addStrIf` (`odata_write.go`) | **broken** — `@` kept in the name | | database connection `connection string` / `username` / `password` | `ByNameRef` → `Constants$Constant` (`databaseconnector/types.go`) | reference | `@Mod.C`; visitor strips `@`, sets `*IsRef` | works; gains `constant` spelling | | REST client `Username:` / `Password:` (and other constant-capable properties) | `Rest$ConstantValue.value` `ByNameRef` | reference | `@Mod.C`, legacy `$Mod.C`; visitor rewrites both to `$Mod.C` | works; two spellings already, `constant` becomes the canonical one | diff --git a/mdl-examples/bug-tests/750-odata-serviceurl-constant-name.mdl b/mdl-examples/bug-tests/750-odata-serviceurl-constant-name.mdl new file mode 100644 index 0000000000..66742d43ce --- /dev/null +++ b/mdl-examples/bug-tests/750-odata-serviceurl-constant-name.mdl @@ -0,0 +1,39 @@ +-- mendixlabs/mxcli#750: an OData client's ServiceUrl names a constant, the same +-- way ProxyHost does. Studio Pro picks the service URL as a constant (CE6825: +-- "'Service url' must be a constant") and stores the reference in +-- HttpConfiguration.CustomLocation as `@Module.Name` (ako/TestApp Odata.Bug1073: +-- "@Odata.Bug1073_Location"). +-- +-- All three spellings below store CustomLocation "@OdSvc.Location"; describe +-- prints the bare name, `ServiceUrl: OdSvc.Location`, like `ProxyHost: +-- Odata.Bug1073_ProxyHost`. The bare name used to be refused as "not a constant +-- reference". A literal URL is still refused. + +create constant OdSvc.Location +type string +default 'https://api.example.com/odata/v4/'; +/ + +create odata client OdSvc.Bare ( + ODataVersion: OData4, + MetadataUrl: 'https://api.example.com/odata/v4/$metadata', + ServiceUrl: OdSvc.Location +); +/ + +create odata client OdSvc.At ( + ODataVersion: OData4, + MetadataUrl: 'https://api.example.com/odata/v4/$metadata', + ServiceUrl: @OdSvc.Location +); +/ + +create odata client OdSvc.QuotedAt ( + ODataVersion: OData4, + MetadataUrl: 'https://api.example.com/odata/v4/$metadata', + ServiceUrl: '@OdSvc.Location' +); +/ + +alter odata client OdSvc.QuotedAt set ServiceUrl = OdSvc.Location; +/ diff --git a/mdl/executor/cmd_odata.go b/mdl/executor/cmd_odata.go index 08cada3516..917feadcf3 100644 --- a/mdl/executor/cmd_odata.go +++ b/mdl/executor/cmd_odata.go @@ -9,6 +9,7 @@ import ( "net/http" "os" "path/filepath" + "regexp" "sort" "strings" "time" @@ -158,7 +159,13 @@ func outputConsumedODataServiceMDL(ctx *ExecContext, svc *model.ConsumedODataSer // HTTP configuration if cfg := svc.HttpConfiguration; cfg != nil { if cfg.OverrideLocation && cfg.CustomLocation != "" { - props = append(props, fmt.Sprintf(" ServiceUrl: %s", formatExprValue(cfg.CustomLocation))) + // The constant's bare name, as ProxyHost prints (serviceURLConstant). + // A stored value that is not `@Module.Name` keeps the quoted form. + if ref := strings.TrimPrefix(cfg.CustomLocation, "@"); strings.HasPrefix(cfg.CustomLocation, "@") && qualifiedConstantName.MatchString(ref) { + props = append(props, fmt.Sprintf(" ServiceUrl: %s", ref)) + } else { + props = append(props, fmt.Sprintf(" ServiceUrl: %s", formatExprValue(cfg.CustomLocation))) + } } // HttpUsername / HttpPassword / ClientCertificate and header values are // Mendix expressions, and MDL writes an expression as-is: the stored @@ -1035,11 +1042,12 @@ func createODataClient(ctx *ExecContext, stmt *ast.CreateODataClientStmt) error svc.HttpConfiguration = &model.HttpConfiguration{} } if stmt.ServiceUrl != "" { - if err := validateServiceURL(stmt.ServiceUrl); err != nil { + location, err := serviceURLConstant(stmt.ServiceUrl) + if err != nil { return err } svc.HttpConfiguration.OverrideLocation = true - svc.HttpConfiguration.CustomLocation = stmt.ServiceUrl + svc.HttpConfiguration.CustomLocation = location } svc.HttpConfiguration.UseAuthentication = stmt.UseAuthentication if stmt.HttpUsername != "" { @@ -1141,18 +1149,17 @@ func createODataClient(ctx *ExecContext, stmt *ast.CreateODataClientStmt) error ClientCertificate: stmt.ClientCertificate, } if stmt.ServiceUrl != "" { - // ServiceUrl must be a constant reference (e.g., @Module.ConstantName) - if !strings.HasPrefix(stmt.ServiceUrl, "@") { - return fmt.Errorf(`ServiceUrl must now be a constant reference (e.g., '@Module.ApiLocation'). -Previously literal URLs were allowed; this enforces the Mendix best practice of externalizing configuration. + location, err := serviceURLConstant(stmt.ServiceUrl) + if err != nil { + return fmt.Errorf(`ServiceUrl must name a constant (e.g., Module.ApiLocation) — Studio Pro CE6825. Create a constant first: CREATE CONSTANT Module.ApiLocation TYPE String DEFAULT 'https://api.example.com/'; Then reference it: - ServiceUrl: '@Module.ApiLocation' + ServiceUrl: Module.ApiLocation Got: %s`, stmt.ServiceUrl) } cfg.OverrideLocation = true - cfg.CustomLocation = stmt.ServiceUrl + cfg.CustomLocation = location } for _, h := range stmt.Headers { cfg.HeaderEntries = append(cfg.HeaderEntries, &model.HttpHeaderEntry{ @@ -1299,14 +1306,15 @@ func alterODataClient(ctx *ExecContext, stmt *ast.AlterODataClientStmt) error { case "description": svc.Description = strVal case "serviceurl": - if err := validateServiceURL(strVal); err != nil { + location, err := serviceURLConstant(strVal) + if err != nil { return err } if svc.HttpConfiguration == nil { svc.HttpConfiguration = &model.HttpConfiguration{} } svc.HttpConfiguration.OverrideLocation = true - svc.HttpConfiguration.CustomLocation = strVal + svc.HttpConfiguration.CustomLocation = location case "useauthentication": if svc.HttpConfiguration == nil { svc.HttpConfiguration = &model.HttpConfiguration{} @@ -1737,15 +1745,27 @@ func dropODataService(ctx *ExecContext, stmt *ast.DropODataServiceStmt) error { return mdlerrors.NewNotFoundMsg("OData service", fmt.Sprint(stmt.Name), fmt.Sprintf("OData service not found: %s", stmt.Name)) } -// validateServiceURL returns an error if url is not a constant reference (@Module.Name). -// CE6825: Studio Pro requires the Service URL to be a constant, not a string literal. -func validateServiceURL(url string) error { - if !strings.HasPrefix(url, "@") { - return mdlerrors.NewValidation("ServiceUrl must be a constant reference (e.g., @Module.ServiceUrlConstant) — Studio Pro CE6825: 'Service url' must be a constant") - } - return nil +// serviceURLConstant turns a ServiceUrl value into what Studio Pro stores in +// HttpConfiguration.CustomLocation: `@Module.Name`. +// +// ServiceUrl names a constant, like ProxyHost — Studio Pro picks the service URL +// as a constant (CE6825: "'Service url' must be a constant") and stores the +// reference as `@Module.Name` (ako/TestApp Odata.Bug1073: +// "@Odata.Bug1073_Location"). So it takes the proxy references' spellings: the +// bare name, `@Module.Name` and the quoted `'@Module.Name'`. Anything that is +// not a constant's name — a literal URL — is refused. +func serviceURLConstant(value string) (string, error) { + ref := extractConstantRef(value) + if !qualifiedConstantName.MatchString(ref) { + return "", mdlerrors.NewValidation(fmt.Sprintf( + "ServiceUrl must name a constant (e.g., ServiceUrl: Module.ApiLocation) — Studio Pro CE6825: 'Service url' must be a constant; got %s", value)) + } + return "@" + ref, nil } +// qualifiedConstantName matches `Module.Name` (or deeper), without an @. +var qualifiedConstantName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)+$`) + // validateMetadataURL returns an error if the MetadataUrl is obviously malformed. // A valid value must be an http/https URL, a file:// URL, or a path that contains // at least one path separator or dot (indicating an extension or subdirectory). diff --git a/mdl/executor/cmd_odata_client_describe_quoting_test.go b/mdl/executor/cmd_odata_client_describe_quoting_test.go index d784de10e4..abc0c1e7e6 100644 --- a/mdl/executor/cmd_odata_client_describe_quoting_test.go +++ b/mdl/executor/cmd_odata_client_describe_quoting_test.go @@ -89,11 +89,17 @@ func TestDescribeODataClient_StoredValuesSurviveReExec(t *testing.T) { got, out := describeAndReparse(t, stored, "Api/O'Clients") cfg := stored.HttpConfiguration + // ServiceUrl names a constant: describe prints the bare name and exec adds + // the @ back (serviceURLConstant), so compare what exec would store. + location, err := serviceURLConstant(got.ServiceUrl) + if err != nil { + t.Fatalf("describe printed a ServiceUrl exec refuses: %v\n%s", err, out) + } for _, c := range []struct{ field, want, got string }{ {"Version", stored.Version, got.Version}, {"MetadataUrl", stored.MetadataUrl, got.MetadataUrl}, {"Folder", "Api/O'Clients", got.Folder}, - {"ServiceUrl", cfg.CustomLocation, got.ServiceUrl}, + {"ServiceUrl", cfg.CustomLocation, location}, {"ClientCertificate", cfg.ClientCertificate, got.ClientCertificate}, } { if c.got != c.want { diff --git a/mdl/executor/cmd_odata_service_url_constant_test.go b/mdl/executor/cmd_odata_service_url_constant_test.go new file mode 100644 index 0000000000..965a698167 --- /dev/null +++ b/mdl/executor/cmd_odata_service_url_constant_test.go @@ -0,0 +1,140 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" +) + +// ServiceUrl names a constant, the same way ProxyHost does: Studio Pro picks the +// service URL as a constant, and stores it in HttpConfiguration.CustomLocation +// as `@Module.Name` (ako/TestApp Odata.Bug1073: "@Odata.Bug1073_Location"). +// So MDL takes the same spellings as the proxy references — the bare name, `@` +// and quoted `@` — and describe prints the bare name, like `ProxyHost: +// Odata.Bug1073_ProxyHost`. Before, only the two `@` spellings were accepted and +// the bare name was refused as "not a constant reference". + +var serviceURLSpellings = []struct{ name, mdl string }{ + {"bare", "MyModule.Location"}, + // The two spellings that already worked: the controls. + {"at", "@MyModule.Location"}, + {"quoted at", "'@MyModule.Location'"}, +} + +func TestCreateODataClient_ServiceUrlNamesAConstant(t *testing.T) { + for _, sp := range serviceURLSpellings { + t.Run(sp.name, func(t *testing.T) { + mod := mkModule("MyModule") + var captured *model.ConsumedODataService + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListConstantsFunc: func() ([]*model.Constant, error) { return nil, nil }, + ListConsumedODataServicesFunc: func() ([]*model.ConsumedODataService, error) { + return nil, nil + }, + CreateConsumedODataServiceFunc: func(svc *model.ConsumedODataService) error { + captured = svc + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(mkHierarchy(mod))) + prog := parseMDL(t, `create odata client MyModule.Api ( + ODataVersion: OData4, + MetadataUrl: 'https://example.com/odata/$metadata', + ServiceUrl: `+sp.mdl+` +);`) + _ = createODataClient(ctx, prog.Statements[0].(*ast.CreateODataClientStmt)) + if captured == nil || captured.HttpConfiguration == nil { + t.Fatalf("ServiceUrl: %s was not written", sp.mdl) + } + if got := captured.HttpConfiguration.CustomLocation; got != "@MyModule.Location" { + t.Errorf("ServiceUrl: %s stored CustomLocation %q, want %q", sp.mdl, got, "@MyModule.Location") + } + if !captured.HttpConfiguration.OverrideLocation { + t.Errorf("ServiceUrl: %s did not set OverrideLocation", sp.mdl) + } + }) + } +} + +func TestAlterODataClient_ServiceUrlNamesAConstant(t *testing.T) { + for _, sp := range serviceURLSpellings { + t.Run(sp.name, func(t *testing.T) { + mod := mkModule("MyModule") + svc := &model.ConsumedODataService{ + BaseElement: model.BaseElement{ID: nextID("cos")}, + ContainerID: mod.ID, + Name: "Api", + } + h := mkHierarchy(mod) + withContainer(h, svc.ContainerID, mod.ID) + var updated *model.ConsumedODataService + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListConsumedODataServicesFunc: func() ([]*model.ConsumedODataService, error) { + return []*model.ConsumedODataService{svc}, nil + }, + UpdateConsumedODataServiceFunc: func(s *model.ConsumedODataService) error { + updated = s + return nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + prog := parseMDL(t, `alter odata client MyModule.Api set ServiceUrl = `+sp.mdl+`;`) + assertNoError(t, alterODataClient(ctx, prog.Statements[0].(*ast.AlterODataClientStmt))) + if updated == nil || updated.HttpConfiguration == nil { + t.Fatal("UpdateConsumedODataService was not called with an HTTP configuration") + } + if got := updated.HttpConfiguration.CustomLocation; got != "@MyModule.Location" { + t.Errorf("set ServiceUrl = %s stored CustomLocation %q, want %q", sp.mdl, got, "@MyModule.Location") + } + }) + } +} + +// A literal URL is still refused: Studio Pro requires a constant (CE6825). +func TestServiceURLConstant_RefusesAString(t *testing.T) { + for _, v := range []string{"https://api.example.com/odata", "", "not a name"} { + if _, err := serviceURLConstant(v); err == nil { + t.Errorf("serviceURLConstant(%q) accepted a value that is not a constant name", v) + } + } + for _, v := range []string{"M.Loc", "@M.Loc", "M.Sub.Loc"} { + if got, err := serviceURLConstant(v); err != nil || got != "@"+strings.TrimPrefix(v, "@") { + t.Errorf("serviceURLConstant(%q) = %q, %v", v, got, err) + } + } +} + +// describe prints the constant's bare name, as it does for ProxyHost, and that +// output re-stores the same CustomLocation. +func TestDescribeODataClient_ServiceUrlPrintsTheConstantName(t *testing.T) { + stored := &model.ConsumedODataService{ + Name: "Bug1073", + ODataVersion: "OData4", + HttpConfiguration: &model.HttpConfiguration{ + OverrideLocation: true, + CustomLocation: "@Odata.Bug1073_Location", + }, + } + var out bytes.Buffer + if err := outputConsumedODataServiceMDL(&ExecContext{Output: &out}, stored, "Odata", ""); err != nil { + t.Fatal(err) + } + if !strings.Contains(out.String(), "ServiceUrl: Odata.Bug1073_Location,") && + !strings.Contains(out.String(), "ServiceUrl: Odata.Bug1073_Location\n") { + t.Errorf("describe should print the bare constant name, got:\n%s", out.String()) + } + stmt := parseMDL(t, out.String()).Statements[0].(*ast.CreateODataClientStmt) + if got, err := serviceURLConstant(stmt.ServiceUrl); err != nil || got != stored.HttpConfiguration.CustomLocation { + t.Errorf("re-exec would store %q (%v), want %q", got, err, stored.HttpConfiguration.CustomLocation) + } +} From ccbfa26829cc9ddb5f8ef3f93b5ac48f6d048f46 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 08:33:46 +0000 Subject: [PATCH 27/47] fix(describe): keep a ComboBox's expression caption (#664) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit describe → exec of Administration.Account_New failed the build: [CE0642] "Property 'Caption' is required." at Combo box 'comboBox2' comboBox2 stores its caption as an expression (optionsSourceAssociationCaptionType = expression, optionsSourceAssociationCaptionExpression = '$currentObject/Description'). Describe read only the attribute caption, so the widget came back with none. The write side already worked. The explicit-property pass persists both storage keys, and the page builds clean. Describe now emits those two keys for an expression caption, the same way it spells a generic pluggable widget's properties. MDL-WIDGET06 ("recognized but not yet persisted") claimed both keys would be dropped, which is false. It now fires only for keys whose declared type the explicit pass cannot write (it writes Expression, TextTemplate, Attribute and scalar types). The #643 test pinned the false claim for CaptionType and now asserts the opposite; a widgets-typed slot still warns. Full round trip of the 17-page stock project: `mx check` goes from 2 errors (CE0642 on Account_New comboBox2 and Account_Edit comboBox4) to 0. Fixes #664 Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../664-combobox-expression-caption.mdl | 44 ++++++++++ .../cmd_pages_combobox_caption_expr_test.go | 87 +++++++++++++++++++ mdl/executor/cmd_pages_describe.go | 4 + mdl/executor/cmd_pages_describe_output.go | 8 ++ mdl/executor/cmd_pages_describe_parse.go | 8 ++ mdl/executor/cmd_pages_describe_pluggable.go | 21 +++++ .../validate_widget_explicit_writable.go | 57 ++++++++++++ mdl/executor/validate_widgets.go | 3 + mdl/executor/validate_widgets_643_test.go | 44 +++++++++- 10 files changed, 273 insertions(+), 4 deletions(-) create mode 100644 mdl-examples/bug-tests/664-combobox-expression-caption.mdl create mode 100644 mdl/executor/cmd_pages_combobox_caption_expr_test.go create mode 100644 mdl/executor/validate_widget_explicit_writable.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index ef9d0f0481..e0818493ed 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -701,3 +701,4 @@ {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_Edit / FeedbackModule.ShareFeedback(_Logo) fails `Parse error: line 14:21 extraneous input '(' expecting the start of a statement`; describe emitted `statictext (Content: '…')` with no widget name", "cause": "The stored widget is Studio Pro's Label (Forms$Label), and it is NAMED (`label4`). The describe emitter hard-coded `statictext (Content: %s)`, dropping name and appearance. MDL had no widget that writes Forms$Label; `statictext` writes Forms$Text, which Mendix 11 cannot load (MDL-WIDGET29), so neither 'make the name optional' nor 'synthesize a name' could have produced a correct round trip", "file": "`mdl/executor/cmd_pages_describe_output.go` (Forms$Label case), `mdl/grammar/domains/MDLPage.g4` (`LABEL` in widgetTypeV3), `mdl/executor/cmd_pages_builder_v3_widgets.go` (`buildLabelV3`), `mdl/backend/modelsdk/widget_write.go` (`*pages.Label` case + Forms$Label TypeDefaults)", "insight": "**Dump the stored widget before accepting the report's diagnosis**: 'the stored Name is empty' was an inference from the output, and one `bson dump --format ndsl` showed a named Forms$Label — which also made both proposed fixes wrong, since the keyword itself wrote an unloadable type. A generic (IDENTIFIER) widget type parses but `check -p` requires it to resolve to a pluggable definition (MDL-WIDGET25); a built-in widget needs its token in widgetTypeV3, and a visitor test must assert `!TypeIsGeneric` or it passes against the unfixed grammar. gen's Label declares top-level Class/Style/AccessibilitySettings that Studio Pro 11 does not store — assert the encoded key set with encodeToD, and register NullFields for ConditionalVisibilitySettings or the key is omitted. Round-tripping a page that previously failed to parse EXPOSES older write gaps on the same page: here attribute-condition visibility (8 Enumerations$Condition → 0, all widgets), a nanoflow data-view source shape (CE2633), and compound design-property list markers (2 → 3) — diff the stored BSON before/after, not just mx check", "refs": ["Administration.Account_Edit", "FeedbackModule.ShareFeedback"], "rules": ["MDL-WIDGET25", "MDL-WIDGET29"], "date": "2026-09-25"} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`container c1 (dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ])` - the first-class spelling mendixlabs/mxcli#750 proposes - checked clean and `exec` reported `Created page`, but the widget was stored with no DynamicClasses at all. `alter page ... set DynamicClasses = [ ... ] on w` reported `Altered page` and changed nothing; a column's `DynamicCellClass` on ALTER stored `[if$x/Ythen'a'else'b']` as its expression", "cause": "`[ ... ]` parses as propertyValueV3's array alternative, so the value reaches the AST as a []string. The create writers read only a string (GetStringProp for DynamicClasses, `v.(string)` for columnClass); the mutator's dynamicclasses case returned nil when the type check failed; setColumnPropertyMut formatted the list with %v, after the visitor's GetText() had already fused its tokens", "file": "`mdl/executor/validate_widget_expression_list.go` (MDL-WIDGET32), `mdl/backend/pagemutator/mutator.go` (`errExpressionNotAString`)", "insight": "**A proposal's example syntax is worth running through `check` before designing around it** - here the proposed spelling already parsed, and the parse was the bug. Same class as #999 / MDL-WIDGET27 (empty `[]`): a value shape no writer claims. Fix at both ends and they stay in step: a no-project check rule for CREATE, and an error (not a nil return) from the ALTER setter, which `check -p` reports for free because validateAlterSetProperties dry-runs the setter. Key the rule on the property, never on the brackets - `visible: [cond]` is valid MDL. Measured with pre-fix and fixed binaries on copies of ako/TestApp: pre-fix `describe` shows the container without DynamicClasses and the quoted control intact. Tests `validate_widget_expression_list_test.go`, `pagemutator/expression_list_value_test.go`", "refs": ["mendixlabs/mxcli#750", "#999"], "rules": ["MDL-WIDGET32"]} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_Edit takes the page's 8 Enumerations$Condition entries to 0: every widget with Studio Pro's \"Visible: based on attribute value\" becomes ALWAYS visible; check, exec and mx check all report success", "cause": "MDL had no spelling for attribute-based conditional visibility. extractConditionalSettings read only Expression, conditionalVisibilityToGen wrote only Expression, and the settings' list markers were the default [3] where Studio Pro stores Conditions [2] and ModuleRoles [1]", "file": "`mdl/grammar/domains/MDLPage.g4` (`VISIBLE COLON attributePathV3 IN (…)`), `mdl/executor/cmd_pages_builder_visible_when.go` (`applyVisibleWhen`), `mdl/backend/modelsdk/widget_write.go` (`conditionalVisibilityToGen`, TypeDefaults), `mdl/executor/cmd_pages_describe_parse.go` / `_output.go` (`visibleWhenProp`)", "insight": "**A dropped visibility setting is invisible to every check**: the model stays valid, the widget just shows for everyone — count `Enumerations$Condition` in `bson dump --format ndsl` before and after a round trip, since mx check never will. Survey the corpus before designing syntax: all 12 settings in the stock project were attribute-based (booleans and one enum), none role-based or editability, which scoped the feature. Studio Pro stores EVERY value (enum values in declaration order plus \"(empty)\", or true/false) with a flag, so MDL lists only the shown values and the writer fills the rest from the domain model — and a byte-identical before/after diff of the settings block (markers included) on 3 pages is the proof. mdlIdent quotes `empty`/`true`/`false` as keywords; emit them bare in a value list. A new AST property key must be added to the known-property list (validate_widgets.go) or MDL-WIDGET07 falsely warns it is dropped", "refs": ["Administration.Account_Edit", "Administration.ScheduledEvents"], "rules": ["MDL-WIDGET07"], "date": "2026-09-25"} +{"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_New fails `mx check` with [CE0642] \"Property 'Caption' is required.\" at Combo box 'comboBox2' (Account_Edit comboBox4 too); check and exec report success", "cause": "The ComboBox caption is an EXPRESSION (optionsSourceAssociationCaptionType=expression, optionsSourceAssociationCaptionExpression='$currentObject/Description'); describe read only the attribute caption, so the widget was rewritten with none. The write side already worked via the explicit-property pass, but MDL-WIDGET06 claimed both keys 'will be dropped'", "file": "`mdl/executor/cmd_pages_describe_parse.go` (combobox branch), `cmd_pages_describe_output.go`, `validate_widget_explicit_writable.go` (`persistedByExplicitPass`), `validate_widgets.go` (MDL-WIDGET06)", "insight": "**Resolve pluggable-widget properties by key before theorising**: a 20-line script mapping each Property's TypePointer to its PropertyKey through the widget's own Type.ObjectType settled the cause in one run, where the ndsl dump shows only anonymous values. Then TEST THE WRITE SIDE before building one: adding the two storage keys to the describe output by hand persisted both and built clean, which shrank the fix to describe + a false warning — no new syntax. A validator rule that asserts 'not persisted' must be tied to what the write path handles (here: the explicit pass writes Expression/TextTemplate/Attribute and scalar types), or it goes stale when the writer grows; the #643 test pinned the stale claim. A dedicated extractor for a 'known' pluggable widget silently drops every property it does not map — generic widgets emit them, known ones do not", "refs": ["ako/mxcli#664", "ako/mxcli#643"], "ce": ["CE0642"], "rules": ["MDL-WIDGET06"], "date": "2026-09-25"} diff --git a/mdl-examples/bug-tests/664-combobox-expression-caption.mdl b/mdl-examples/bug-tests/664-combobox-expression-caption.mdl new file mode 100644 index 0000000000..986dd9dcdf --- /dev/null +++ b/mdl-examples/bug-tests/664-combobox-expression-caption.mdl @@ -0,0 +1,44 @@ +-- ============================================================================ +-- ako/mxcli#664: an association ComboBox's EXPRESSION caption was dropped +-- ============================================================================ +-- +-- Symptom: describe → exec of Administration.Account_New (Administration +-- v4.3.2, Mendix 11.13.0): +-- [error] [CE0642] "Property 'Caption' is required." at Combo box 'comboBox2' +-- +-- Cause: comboBox2 stores its caption as an expression — +-- optionsSourceAssociationCaptionType = expression, +-- optionsSourceAssociationCaptionExpression = '$currentObject/Description' — +-- and DESCRIBE read only the attribute caption (`CaptionAttribute:`), so the +-- widget came back with none. The WRITE side already worked: the explicit +-- property pass persists both storage keys. `check` nonetheless warned +-- (MDL-WIDGET06) that each "will be dropped", which was false. +-- +-- Fix: DESCRIBE emits the two storage keys for an expression caption, and +-- MDL-WIDGET06 fires only for keys the explicit pass cannot write. +-- +-- Verify: `mxcli check` shows no MDL-WIDGET06; exec, then describe → exec +-- (expect "Unchanged page"), then `mxcli docker check` — 0 errors. +-- ============================================================================ + +create entity MyFirstModule.Region ( Code: String(10), Description: String(200) ); +create entity MyFirstModule.Office ( Name: String(200) ); +create association MyFirstModule.Office_Region from MyFirstModule.Office to MyFirstModule.Region; +/ + +create or replace page MyFirstModule.Office_Edit +( Title: 'Office', Layout: Atlas_Core.Atlas_Default, Params: { $Office: MyFirstModule.Office } ) +{ + dataview dv (datasource: $Office) { + combobox cmbRegion ( + Label: 'Region', + Attribute: Office_Region, + DataSource: database from MyFirstModule.Region, + optionsSourceAssociationCaptionType: expression, + optionsSourceAssociationCaptionExpression: '$currentObject/Code + '' — '' + $currentObject/Description' + ) + } +} +/ + +describe page MyFirstModule.Office_Edit; diff --git a/mdl/executor/cmd_pages_combobox_caption_expr_test.go b/mdl/executor/cmd_pages_combobox_caption_expr_test.go new file mode 100644 index 0000000000..f049a8a24e --- /dev/null +++ b/mdl/executor/cmd_pages_combobox_caption_expr_test.go @@ -0,0 +1,87 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "context" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// ako/mxcli#664: a ComboBox in association mode whose caption is an +// EXPRESSION — Administration.Account_New's comboBox2 (Administration v4.3.2, +// Mendix 11.13.0) stores +// +// optionsSourceAssociationCaptionType = "expression" +// optionsSourceAssociationCaptionExpression = "$currentObject/Description" +// +// DESCRIBE read only the attribute caption, so the widget came back with no +// caption at all and exec produced +// +// [CE0642] "Property 'Caption' is required." at Combo box 'comboBox2' +func comboBoxWithExpressionCaption() map[string]any { + w := buildComboBoxAssocWidget("System.User_TimeZone", "") + pts := w["Type"].(map[string]any)["ObjectType"].(map[string]any) + pts["PropertyTypes"] = append(pts["PropertyTypes"].([]any), + map[string]any{"$ID": "type-id-005", "PropertyKey": "optionsSourceAssociationCaptionType"}, + map[string]any{"$ID": "type-id-006", "PropertyKey": "optionsSourceAssociationCaptionExpression"}, + ) + obj := w["Object"].(map[string]any) + props := obj["Properties"].([]any) + // Drop the attribute caption: an expression-caption widget stores none. + kept := props[:0] + for _, p := range props { + if p.(map[string]any)["TypePointer"] != "type-id-004" { + kept = append(kept, p) + } + } + obj["Properties"] = append(kept, + map[string]any{"TypePointer": "type-id-005", "Value": map[string]any{"PrimitiveValue": "expression"}}, + map[string]any{"TypePointer": "type-id-006", "Value": map[string]any{"Expression": "$currentObject/Description"}}, + ) + w["$Type"] = "CustomWidgets$CustomWidget" + w["Name"] = "comboBox2" + return w +} + +func TestDescribeComboBox_ExpressionCaption(t *testing.T) { + var buf bytes.Buffer + ctx := (&Executor{}).newExecContext(context.Background()) + ctx.Output = &buf + raw := parseRawWidget(ctx, comboBoxWithExpressionCaption(), "Administration.Account") + outputWidgetMDLV3(ctx, raw[0], 1) + got := buf.String() + for _, want := range []string{ + "optionsSourceAssociationCaptionType: expression", + "optionsSourceAssociationCaptionExpression: '$currentObject/Description'", + } { + if !strings.Contains(got, want) { + t.Errorf("describe output lacks %q — the caption is dropped and exec fails CE0642:\n%s", want, got) + } + } + if strings.Contains(got, "CaptionAttribute:") { + t.Errorf("an expression caption must not also emit CaptionAttribute:\n%s", got) + } + if _, errs := visitor.Build("create page M.P (Title: 'x', Layout: A.L) {\n" + got + "}\n"); len(errs) > 0 { + t.Fatalf("describe output does not parse: %v\n%s", errs, got) + } +} + +// The attribute caption is unchanged. +func TestDescribeComboBox_AttributeCaptionUnchanged(t *testing.T) { + var buf bytes.Buffer + ctx := (&Executor{}).newExecContext(context.Background()) + ctx.Output = &buf + w := buildComboBoxAssocWidget("MyFirstModule.Task_Category", "MyFirstModule.Category.Name") + w["$Type"] = "CustomWidgets$CustomWidget" + w["Name"] = "cb" + raw := parseRawWidget(ctx, w, "MyFirstModule.Task") + outputWidgetMDLV3(ctx, raw[0], 1) + got := buf.String() + if !strings.Contains(got, "CaptionAttribute: Name") || strings.Contains(got, "CaptionExpression") { + t.Errorf("attribute caption changed:\n%s", got) + } +} diff --git a/mdl/executor/cmd_pages_describe.go b/mdl/executor/cmd_pages_describe.go index 1495028755..e60d326897 100644 --- a/mdl/executor/cmd_pages_describe.go +++ b/mdl/executor/cmd_pages_describe.go @@ -689,6 +689,10 @@ type rawWidget struct { PhoneColumns string // e.g. "2", "1" // ComboBox association mode properties CaptionAttribute string // Display attribute for association-mode ComboBox + // CaptionExpression is an association-mode ComboBox caption of type + // Expression; emitted under its storage keys, which the explicit-property + // pass writes back (#664). + CaptionExpression string // GroupBox properties Collapsible string // "No", "YesInitiallyExpanded", "YesInitiallyCollapsed" HeaderMode string // "Div", "H1"-"H6" diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index 676ecd0958..8ccd8cc98f 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -866,6 +866,14 @@ func outputWidgetMDLV3(ctx *ExecContext, w rawWidget, indent int) { if w.CaptionAttribute != "" { props = append(props, fmt.Sprintf("CaptionAttribute: %s", w.CaptionAttribute)) } + // An expression caption has no MDL alias; its storage keys are the + // spelling the explicit-property pass writes, as for any + // pluggable-widget property (#664). + if w.CaptionExpression != "" { + props = append(props, + "optionsSourceAssociationCaptionType: expression", + fmt.Sprintf("optionsSourceAssociationCaptionExpression: %s", mdlQuote(w.CaptionExpression))) + } } // A pluggable widget's on-change action (ComboBox `onChangeEvent`). // Emitted for the same reason as the built-in inputs above: without diff --git a/mdl/executor/cmd_pages_describe_parse.go b/mdl/executor/cmd_pages_describe_parse.go index 5ddf678c8a..9ecda63105 100644 --- a/mdl/executor/cmd_pages_describe_parse.go +++ b/mdl/executor/cmd_pages_describe_parse.go @@ -408,6 +408,14 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s if widget.DataSource != nil { widget.Content = associationRefForContext(extractCustomWidgetPropertyAssociationQN(ctx, w, "attributeAssociation"), inheritedCtx) widget.CaptionAttribute = extractCustomWidgetPropertyAttributeRef(ctx, w, "optionsSourceAssociationCaptionAttribute") + // The caption can be an EXPRESSION instead (Studio Pro's "Caption + // type: Expression"). Reading only the attribute form dropped it, + // and exec wrote a combobox with no caption — CE0642 "Property + // 'Caption' is required" on Administration.Account_New (#664). + if extractCustomWidgetPropertyString(ctx, w, "optionsSourceAssociationCaptionType") == "expression" { + widget.CaptionExpression = extractCustomWidgetPropertyExpression(w, "optionsSourceAssociationCaptionExpression") + widget.CaptionAttribute = "" + } } // The on-change action, in BOTH modes — the def maps `onChangeEvent` // in each, and modes are exclusive. Read outside the DataSource diff --git a/mdl/executor/cmd_pages_describe_pluggable.go b/mdl/executor/cmd_pages_describe_pluggable.go index ecbded2054..5daa553c91 100644 --- a/mdl/executor/cmd_pages_describe_pluggable.go +++ b/mdl/executor/cmd_pages_describe_pluggable.go @@ -1011,6 +1011,27 @@ func extractCustomWidgetPropertyImage(ctx *ExecContext, w map[string]any, proper return "" } +// extractCustomWidgetPropertyExpression reads an expression-typed property's +// stored Expression, or "" when it is unset. +func extractCustomWidgetPropertyExpression(w map[string]any, propertyKey string) string { + obj, ok := w["Object"].(map[string]any) + if !ok { + return "" + } + propTypeKeyMap := buildPropertyTypeKeyMap(w, false) + for _, prop := range getBsonArrayElements(obj["Properties"]) { + propMap, ok := prop.(map[string]any) + if !ok || propTypeKeyMap[extractBinaryID(propMap["TypePointer"])] != propertyKey { + continue + } + if value, ok := propMap["Value"].(map[string]any); ok { + expr, _ := value["Expression"].(string) + return expr + } + } + return "" +} + func extractCustomWidgetPropertyString(ctx *ExecContext, w map[string]any, propertyKey string) string { obj, ok := w["Object"].(map[string]any) if !ok { diff --git a/mdl/executor/validate_widget_explicit_writable.go b/mdl/executor/validate_widget_explicit_writable.go new file mode 100644 index 0000000000..054297f780 --- /dev/null +++ b/mdl/executor/validate_widget_explicit_writable.go @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "sync" + + mwidgets "github.com/mendixlabs/mxcli/modelsdk/widgets" +) + +// explicitPassWritableTypes are the property value types the widget engine's +// explicit-property pass (PluggableWidgetEngine.Build, step 4.6) writes by +// storage key: Expression, TextTemplate and Attribute through their own +// setters, and the scalar types as a primitive. A value of any other type +// (objects, widgets, icons, images, actions, datasources…) has no correct +// route there, so MDL-WIDGET06 stays true for it. +var explicitPassWritableTypes = map[string]bool{ + "expression": true, "texttemplate": true, "attribute": true, + "string": true, "boolean": true, "integer": true, "decimal": true, "enumeration": true, +} + +var widgetPropertyTypesCache sync.Map // widgetID → map[lowercased key]lowercased type + +// widgetPropertyTypes returns each property key's declared value type, from +// the widget's embedded template. Empty when there is no template. +func widgetPropertyTypes(widgetID string) map[string]string { + if v, ok := widgetPropertyTypesCache.Load(widgetID); ok { + return v.(map[string]string) + } + out := map[string]string{} + if tmpl, err := mwidgets.GetTemplate(widgetID); err == nil && tmpl != nil { + for _, p := range propsFromTemplate(tmpl.Type) { + if p.Key != "" && p.Type != "" { + out[strings.ToLower(p.Key)] = strings.ToLower(p.Type) + } + } + } + widgetPropertyTypesCache.Store(widgetID, out) + return out +} + +// persistedByExplicitPass reports whether a key MDL-WIDGET06 would call "not +// yet persisted" is in fact written by the explicit-property pass. +// +// It was not: `optionsSourceAssociationCaptionType: expression` and +// `optionsSourceAssociationCaptionExpression: '…'` on a ComboBox are both +// written, and the page builds clean (measured on Mendix 11.13.0, #664) — yet +// check warned that each "will be dropped". DESCRIBE now emits exactly those +// keys for an expression caption, so the false warning would land on every +// round trip of Administration.Account_New. +// +// An unknown type keeps the warning: saying "not persisted" for something that +// is, is the lesser error than the reverse. +func persistedByExplicitPass(widgetID, lowerKey string) bool { + return explicitPassWritableTypes[widgetPropertyTypes(widgetID)[lowerKey]] +} diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index 65fed81533..aac5dfa73c 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -1264,6 +1264,9 @@ func validatePluggableWidgetProperties(w *ast.WidgetV3, registry *WidgetRegistry // Recognized real property the .def.json doesn't map to a write path: // don't reject it as unknown, but be honest that a non-default value // won't persist through mxcli yet (issue #643). + if knownUnmapped[lower] && persistedByExplicitPass(def.WidgetID, lower) { + continue + } if knownUnmapped[lower] { out = append(out, linter.Violation{ RuleID: "MDL-WIDGET06", diff --git a/mdl/executor/validate_widgets_643_test.go b/mdl/executor/validate_widgets_643_test.go index ff216d6192..a34cd64254 100644 --- a/mdl/executor/validate_widgets_643_test.go +++ b/mdl/executor/validate_widgets_643_test.go @@ -47,10 +47,11 @@ func TestIssue643_DatasourceByName_Rejected(t *testing.T) { if _, ok := got["MDL-WIDGET05"]; !ok { t.Errorf("expected MDL-WIDGET05 for datasource-by-name, got rules: %v", keysOf(got)) } - if msg, ok := got["MDL-WIDGET06"]; !ok { - t.Errorf("expected MDL-WIDGET06 warning for CaptionType, got rules: %v", keysOf(got)) - } else if !strings.Contains(msg, "optionsSourceAssociationCaptionType") { - t.Errorf("WIDGET04 message should name the property, got: %q", msg) + // CaptionType is an enumeration the explicit-property pass WRITES — measured + // (#664): `optionsSourceAssociationCaptionType: expression` persists and the + // page builds clean — so a "not persisted" warning would be false. + if msg, ok := got["MDL-WIDGET06"]; ok { + t.Errorf("CaptionType is persisted by the explicit-property pass; MDL-WIDGET06 is false here: %q", msg) } if _, ok := got["MDL-WIDGET01"]; ok { t.Errorf("CaptionType must NOT be a false 'unknown property' (MDL-WIDGET01)") @@ -82,3 +83,38 @@ func keysOf(m map[string]string) []string { } return ks } + +// ako/mxcli#664: MDL-WIDGET06 ("recognized but not yet persisted") must fire only +// for keys the explicit-property pass cannot write. The expression caption's two +// keys ARE written (measured: exec persists both and mx check is clean), and +// DESCRIBE now emits them, so a warning would tell the reader their own +// round-tripped caption is dropped. A widgets-typed slot is still not written. +func TestWidget06_OnlyForKeysTheExplicitPassCannotWrite(t *testing.T) { + reg := LoadWidgetRegistry("") + if reg == nil { + t.Fatal("built-in widget registry not available") + } + w := combo(map[string]any{ + "optionsSourceAssociationCaptionType": "expression", + "optionsSourceAssociationCaptionExpression": "$currentObject/Description", + "optionsSourceAssociationCustomContent": "x", + }) + var w06 []string + for _, v := range validatePluggableWidgetProperties(w, reg, "page P") { + if v.RuleID == "MDL-WIDGET06" { + w06 = append(w06, v.Message) + } + if v.RuleID == "MDL-WIDGET01" { + t.Errorf("unexpected MDL-WIDGET01: %s", v.Message) + } + } + joined := strings.Join(w06, "\n") + for _, k := range []string{"optionsSourceAssociationCaptionType", "optionsSourceAssociationCaptionExpression"} { + if strings.Contains(joined, "`"+k+"`") { + t.Errorf("%s is written by the explicit pass; MDL-WIDGET06 is false for it", k) + } + } + if !strings.Contains(joined, "optionsSourceAssociationCustomContent") { + t.Errorf("a widgets-typed slot is not written; MDL-WIDGET06 must still fire for it (got %q)", joined) + } +} From 864440dbba70ac6b65575cb62bf1c1aeaa9afe87 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 08:46:03 +0000 Subject: [PATCH 28/47] fix: excluded page's dangling action references no longer block exec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Mendix does not validate excluded documents: Feedback v4.0.2 ships FeedbackModule.ShareFeedback_Logo as an excluded page bound to nanoflows the module does not contain, and the untouched project checks at 0 errors. mxcli refused it twice — the page reference check ("has reference errors") and, under --no-check, the page builder ("failed to resolve nanoflow"). For a page or snippet that exec will write EXCLUDED (@excluded, or the #914 carry from a stored namesake), a dangling action target or snippet call is now reported as a "Reference warning" by check and exec and kept by name by the builder (the writer only ever stores the qualified name). Excluded microflows/nanoflows/rules, previously skipped silently, now warn too. A dangling DATA SOURCE (or entity) still blocks, with the reason. The source's flow puts the entity in scope; without it the nested bindings are written as bare names, and on 11.13.0 that made mx unable to load the project (ArgumentNullException setting 'Attribute') — measured by forcing it. Verified on 11.13.0: an identical page with three dangling action targets checks at 0 errors excluded and fails 3x CE1613 live. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .claude/skills/mendix/check-syntax/SKILL.md | 9 + cmd/mxcli/cmd_check.go | 25 ++- cmd/mxcli/cmd_exec.go | 6 +- docs-site/src/tutorial/validation.md | 7 + .../excluded-page-dangling-references.mdl | 57 ++++++ mdl/executor/cmd_pages_builder.go | 24 +++ mdl/executor/cmd_pages_builder_v3.go | 8 +- mdl/executor/cmd_pages_builder_v3_widgets.go | 2 +- mdl/executor/cmd_pages_create_v3.go | 7 + mdl/executor/helpers.go | 15 ++ mdl/executor/validate.go | 186 +++++++++++++++--- mdl/executor/validate_excluded_page_test.go | 179 +++++++++++++++++ .../validate_script_javaactions_test.go | 4 +- 14 files changed, 496 insertions(+), 34 deletions(-) create mode 100644 mdl-examples/bug-tests/excluded-page-dangling-references.mdl create mode 100644 mdl/executor/validate_excluded_page_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index ef9d0f0481..8d6bc88da1 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -701,3 +701,4 @@ {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_Edit / FeedbackModule.ShareFeedback(_Logo) fails `Parse error: line 14:21 extraneous input '(' expecting the start of a statement`; describe emitted `statictext (Content: '…')` with no widget name", "cause": "The stored widget is Studio Pro's Label (Forms$Label), and it is NAMED (`label4`). The describe emitter hard-coded `statictext (Content: %s)`, dropping name and appearance. MDL had no widget that writes Forms$Label; `statictext` writes Forms$Text, which Mendix 11 cannot load (MDL-WIDGET29), so neither 'make the name optional' nor 'synthesize a name' could have produced a correct round trip", "file": "`mdl/executor/cmd_pages_describe_output.go` (Forms$Label case), `mdl/grammar/domains/MDLPage.g4` (`LABEL` in widgetTypeV3), `mdl/executor/cmd_pages_builder_v3_widgets.go` (`buildLabelV3`), `mdl/backend/modelsdk/widget_write.go` (`*pages.Label` case + Forms$Label TypeDefaults)", "insight": "**Dump the stored widget before accepting the report's diagnosis**: 'the stored Name is empty' was an inference from the output, and one `bson dump --format ndsl` showed a named Forms$Label — which also made both proposed fixes wrong, since the keyword itself wrote an unloadable type. A generic (IDENTIFIER) widget type parses but `check -p` requires it to resolve to a pluggable definition (MDL-WIDGET25); a built-in widget needs its token in widgetTypeV3, and a visitor test must assert `!TypeIsGeneric` or it passes against the unfixed grammar. gen's Label declares top-level Class/Style/AccessibilitySettings that Studio Pro 11 does not store — assert the encoded key set with encodeToD, and register NullFields for ConditionalVisibilitySettings or the key is omitted. Round-tripping a page that previously failed to parse EXPOSES older write gaps on the same page: here attribute-condition visibility (8 Enumerations$Condition → 0, all widgets), a nanoflow data-view source shape (CE2633), and compound design-property list markers (2 → 3) — diff the stored BSON before/after, not just mx check", "refs": ["Administration.Account_Edit", "FeedbackModule.ShareFeedback"], "rules": ["MDL-WIDGET25", "MDL-WIDGET29"], "date": "2026-09-25"} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`container c1 (dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ])` - the first-class spelling mendixlabs/mxcli#750 proposes - checked clean and `exec` reported `Created page`, but the widget was stored with no DynamicClasses at all. `alter page ... set DynamicClasses = [ ... ] on w` reported `Altered page` and changed nothing; a column's `DynamicCellClass` on ALTER stored `[if$x/Ythen'a'else'b']` as its expression", "cause": "`[ ... ]` parses as propertyValueV3's array alternative, so the value reaches the AST as a []string. The create writers read only a string (GetStringProp for DynamicClasses, `v.(string)` for columnClass); the mutator's dynamicclasses case returned nil when the type check failed; setColumnPropertyMut formatted the list with %v, after the visitor's GetText() had already fused its tokens", "file": "`mdl/executor/validate_widget_expression_list.go` (MDL-WIDGET32), `mdl/backend/pagemutator/mutator.go` (`errExpressionNotAString`)", "insight": "**A proposal's example syntax is worth running through `check` before designing around it** - here the proposed spelling already parsed, and the parse was the bug. Same class as #999 / MDL-WIDGET27 (empty `[]`): a value shape no writer claims. Fix at both ends and they stay in step: a no-project check rule for CREATE, and an error (not a nil return) from the ALTER setter, which `check -p` reports for free because validateAlterSetProperties dry-runs the setter. Key the rule on the property, never on the brackets - `visible: [cond]` is valid MDL. Measured with pre-fix and fixed binaries on copies of ako/TestApp: pre-fix `describe` shows the container without DynamicClasses and the quoted control intact. Tests `validate_widget_expression_list_test.go`, `pagemutator/expression_list_value_test.go`", "refs": ["mendixlabs/mxcli#750", "#999"], "rules": ["MDL-WIDGET32"]} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_Edit takes the page's 8 Enumerations$Condition entries to 0: every widget with Studio Pro's \"Visible: based on attribute value\" becomes ALWAYS visible; check, exec and mx check all report success", "cause": "MDL had no spelling for attribute-based conditional visibility. extractConditionalSettings read only Expression, conditionalVisibilityToGen wrote only Expression, and the settings' list markers were the default [3] where Studio Pro stores Conditions [2] and ModuleRoles [1]", "file": "`mdl/grammar/domains/MDLPage.g4` (`VISIBLE COLON attributePathV3 IN (…)`), `mdl/executor/cmd_pages_builder_visible_when.go` (`applyVisibleWhen`), `mdl/backend/modelsdk/widget_write.go` (`conditionalVisibilityToGen`, TypeDefaults), `mdl/executor/cmd_pages_describe_parse.go` / `_output.go` (`visibleWhenProp`)", "insight": "**A dropped visibility setting is invisible to every check**: the model stays valid, the widget just shows for everyone — count `Enumerations$Condition` in `bson dump --format ndsl` before and after a round trip, since mx check never will. Survey the corpus before designing syntax: all 12 settings in the stock project were attribute-based (booleans and one enum), none role-based or editability, which scoped the feature. Studio Pro stores EVERY value (enum values in declaration order plus \"(empty)\", or true/false) with a flag, so MDL lists only the shown values and the writer fills the rest from the domain model — and a byte-identical before/after diff of the settings block (markers included) on 3 pages is the proof. mdlIdent quotes `empty`/`true`/`false` as keywords; emit them bare in a value list. A new AST property key must be added to the known-property list (validate_widgets.go) or MDL-WIDGET07 falsely warns it is dropped", "refs": ["Administration.Account_Edit", "Administration.ScheduledEvents"], "rules": ["MDL-WIDGET07"], "date": "2026-09-25"} +{"area":"mdl/executor","date":"2026-09-25","symptom":"describe → exec of an EXCLUDED page (Feedback v4.0.2 FeedbackModule.ShareFeedback_Logo, 11.13.0) refused: `page '…' has reference errors: - nanoflow not found: FeedbackModule.DS_FeedbackForm …`, though the untouched project passes mx check at 0 errors (Mendix does not validate excluded documents). With --no-check the page builder refused the same names again (`failed to resolve nanoflow`).","cause":"Two refusals, not one: validate.go's CreatePageStmtV3/CreateSnippetStmtV3 cases ignored exclusion (microflow/nanoflow/rule had been exempt since #312, silently), and pageBuilder.resolveMicroflow/resolveNanoflowByName/resolvePageRef/resolveSnippetRef fail on a missing name though the writer only ever stores the qualified NAME (IDs are never serialized).","file":"mdl/executor/validate.go (relaxExcludedWidgetRefs, carriedExclusion, warnExcluded), mdl/executor/cmd_pages_builder.go (tolerateDanglingRefs/danglingRefOK), cmd/mxcli/cmd_exec.go + cmd_check.go (ValidateProgramWithWarnings)","insight":"Relaxing the check is NOT safe for a DATA SOURCE, and only a real run shows it: the source flow's return type is the entity in scope, describe prints the nested bindings as bare names (`Attribute: Subject`, `ImageUrlParams: [{1} = ImageB64]`), and writing them without the entity left a bare `ImageB64` AttributeRef that made mx unable to LOAD the project (ArgumentNullException setting 'Attribute') — excluded page or not, where the pre-fix refusal had been protecting it by accident. So dangling action targets/snippet calls are warnings, dangling data sources and entities still block with the reason. 'Excluded' must mean what exec WRITES — @excluded OR the #914 carry (every stored namesake excluded) — or check and exec disagree. A/B on 11.13.0: identical page with 3 dangling action targets, excluded → 0 errors; live → 3x CE1613. Follow-up not fixed: describe loses attribute qualification inside a container whose flow is unresolvable, so ShareFeedback_Logo itself still cannot round-trip.","refs":["mdl-examples/bug-tests/excluded-page-dangling-references.mdl","#312","#914"]} diff --git a/.claude/skills/mendix/check-syntax/SKILL.md b/.claude/skills/mendix/check-syntax/SKILL.md index 661a2be135..be4b1cee6f 100644 --- a/.claude/skills/mendix/check-syntax/SKILL.md +++ b/.claude/skills/mendix/check-syntax/SKILL.md @@ -58,6 +58,15 @@ icon or entity sailed through a command that had been handed the project. A run without a project now says what it did not check, so a pass is never read as more than it is. +**An excluded document's dangling references are warnings, not errors.** Mendix +does not validate excluded documents (Feedback v4.0.2 ships an excluded page bound +to nanoflows it lacks, and the project checks at 0 errors), so `check` and `exec` +print them as `Reference warning` lines for excluded microflows, nanoflows, rules, +and pages/snippets exec will write excluded (`@excluded`, or a stored namesake that +is). **A page's or snippet's missing data source (or entity) still blocks:** the +widgets inside bind against it, and written without it their bindings are bare +names — on 11.13.0 that left a project `mx` could not load. + ### It also reports a name the PROJECT already has A plain `create` of a document the project already carries is a `check` error, diff --git a/cmd/mxcli/cmd_check.go b/cmd/mxcli/cmd_check.go index bcfe553fd7..59cea5afce 100644 --- a/cmd/mxcli/cmd_check.go +++ b/cmd/mxcli/cmd_check.go @@ -235,14 +235,35 @@ Examples: } // Validate the program (considers objects defined within the script) - validationErrors := exec.ValidateProgram(prog) + validationErrors, refWarnings := exec.ValidateProgramWithWarnings(prog) // Check for project conflicts: plain CREATE where the document already exists validationErrors = append(validationErrors, exec.CheckProjectConflicts(prog)...) + // Unresolved references in EXCLUDED documents: reported, never + // failing the run — Mendix does not validate excluded documents. + // In structured mode they join the error list (one document, not two) + // or are emitted on their own when there is nothing else. + var warnViolations []linter.Violation + for _, w := range refWarnings { + warnViolations = append(warnViolations, linter.Violation{ + RuleID: "MDL-REF", + Severity: linter.SeverityWarning, + Message: w, + }) + } + if len(refWarnings) > 0 && !isStructured { + fmt.Fprintf(os.Stderr, "Reference warnings:\n") + for _, w := range refWarnings { + fmt.Fprintf(os.Stderr, " %s\n", w) + } + } else if len(warnViolations) > 0 && len(validationErrors) == 0 { + formatter.Format(warnViolations, os.Stderr) + } + if len(validationErrors) > 0 { if isStructured { - var refViolations []linter.Violation + refViolations := warnViolations for _, err := range validationErrors { refViolations = append(refViolations, linter.Violation{ RuleID: "MDL-REF", diff --git a/cmd/mxcli/cmd_exec.go b/cmd/mxcli/cmd_exec.go index 5ceef28620..317a57ba7a 100644 --- a/cmd/mxcli/cmd_exec.go +++ b/cmd/mxcli/cmd_exec.go @@ -157,7 +157,11 @@ Example: // is worth reporting when validating a script, but it is ordinary for a // re-run, and refusing it would break scripts that work today. if !skipCheck && projectPath != "" { - if refErrs := exec.ValidateProgram(prog); len(refErrs) > 0 { + refErrs, refWarnings := exec.ValidateProgramWithWarnings(prog) + for _, w := range refWarnings { + fmt.Fprintf(os.Stderr, "Reference warning: %s\n", w) + } + if len(refErrs) > 0 { for _, refErr := range refErrs { fmt.Fprintf(os.Stderr, "Reference error: %v\n", refErr) } diff --git a/docs-site/src/tutorial/validation.md b/docs-site/src/tutorial/validation.md index 3003709d22..bdb7e8a4ab 100644 --- a/docs-site/src/tutorial/validation.md +++ b/docs-site/src/tutorial/validation.md @@ -47,6 +47,13 @@ This catches everything Level 1 catches, plus: This is the check you should run before executing a script. It's fast (reads the project but doesn't modify it) and catches most mistakes. +References inside an **excluded** document (`@excluded`, or a page or snippet +that stays excluded because its stored namesake is) are reported as +`Reference warnings` rather than errors, because Mendix does not validate +excluded documents. A page's or snippet's missing *data source* still fails the +check: the widgets inside it bind against that source's entity, and cannot be +written without it. + ### Name conflicts with the project A plain `create` of something the project already has is reported here rather diff --git a/mdl-examples/bug-tests/excluded-page-dangling-references.mdl b/mdl-examples/bug-tests/excluded-page-dangling-references.mdl new file mode 100644 index 0000000000..4e8a6ce73b --- /dev/null +++ b/mdl-examples/bug-tests/excluded-page-dangling-references.mdl @@ -0,0 +1,57 @@ +-- ============================================================================ +-- Bug: an EXCLUDED page with dangling references was refused by exec +-- ============================================================================ +-- +-- Symptom (before fix): +-- Feedback v4.0.2 (Mendix 11.13.0) ships FeedbackModule.ShareFeedback_Logo +-- as an excluded example page bound to five nanoflows the module does not +-- contain. The untouched project passes `mx check` with 0 errors — Mendix +-- does not validate excluded documents — but describe → exec was refused: +-- +-- Reference error: statement 1: page 'FeedbackModule.ShareFeedback_Logo' has reference errors: +-- - nanoflow not found: FeedbackModule.DS_FeedbackForm +-- - nanoflow not found: FeedbackModule.ACT_TriggerScreenshotMode +-- ... +-- +-- and with --no-check the page builder refused the same names again. +-- +-- Root cause: +-- The page reference check (validate.go) and the page builder's flow / page +-- / snippet resolution ignored exclusion. Excluded microflows, nanoflows and +-- rules had been exempt since #312; pages and snippets never were. +-- +-- After fix: +-- For a page that exec will write EXCLUDED (the statement's @excluded, or +-- the exclusion carried from a stored page whose namesakes are all +-- excluded, #914) a dangling ACTION target or snippet call is a warning — +-- "Reference warning: page '…' is excluded, so its unresolved references do +-- not block" — and the builder keeps it by name. A dangling DATA SOURCE +-- (or entity) still blocks, with the reason: its flow's return type is what +-- the widgets inside bind against, and writing the container without it left +-- an unqualified attribute binding that made mx unable to LOAD the project +-- (ArgumentNullException setting 'Attribute'), measured on 11.13.0. +-- +-- Verify: +-- mxcli exec mdl-examples/bug-tests/excluded-page-dangling-references.mdl -p app.mpr +-- -> three reference warnings, "Created page BugTestExclRefs.Dangling_Excluded" +-- mxcli docker check -p app.mpr +-- -> The app contains: 0 errors. (the page stayed excluded) +-- Control: delete the `@excluded` line and re-run — exec refuses with +-- "has reference errors". The same page written live (targets created, then +-- dropped) is 3x CE1613 "The selected nanoflow/microflow/page … no longer +-- exists." in mx check. +-- ============================================================================ + +create module BugTestExclRefs; + +@excluded +create page BugTestExclRefs.Dangling_Excluded ( + Title: 'Dangling', + Layout: Atlas_Core.PopupLayout +) { + container c1 { + actionbutton b1 (Caption: 'NF', Action: nanoflow BugTestExclRefs.No_Such_Nanoflow) + actionbutton b2 (Caption: 'MF', Action: microflow BugTestExclRefs.No_Such_Microflow) + actionbutton b3 (Caption: 'PG', Action: show_page BugTestExclRefs.No_Such_Page) + } +} diff --git a/mdl/executor/cmd_pages_builder.go b/mdl/executor/cmd_pages_builder.go index 7d5b13d726..d468f89b16 100644 --- a/mdl/executor/cmd_pages_builder.go +++ b/mdl/executor/cmd_pages_builder.go @@ -4,6 +4,7 @@ package executor import ( "context" + "errors" "fmt" "log" "strings" @@ -22,6 +23,14 @@ import ( // Page Builder // ============================================================================ +// danglingRefOK reports whether a failed reference resolution may be kept by +// name: only for an excluded document, and only when the name resolved to +// nothing (a backend failure is never swallowed). See tolerateDanglingRefs. +func (pb *pageBuilder) danglingRefOK(err error) bool { + var nf *mdlerrors.NotFoundError + return pb.tolerateDanglingRefs && errors.As(err, &nf) +} + // pageBuilder constructs pages from AST. type pageBuilder struct { ctx *ExecContext // execution context (for building-block expansion, etc.) @@ -65,6 +74,21 @@ type pageBuilder struct { // would otherwise say only "this widget". currentWidget string + // tolerateDanglingRefs is set when the document being built is EXCLUDED + // (by @excluded, or carried from the stored document). Mendix does not + // validate an excluded document, and one may name flows, pages or snippets + // the project does not contain — Feedback v4.0.2 ships such an example + // page. The writer stores an ACTION's or snippet call's target BY NAME, so + // an unresolved one is kept as written instead of failing the build. + // + // A DATA SOURCE is deliberately not tolerated: its flow's return type is + // what puts an entity in scope, and without it every attribute binding + // inside the container is written unqualified. Measured on Mendix 11.13.0: + // a bare `ImageB64` in an image's URL parameter made `mx check` fail to + // LOAD the project (ArgumentNullException setting 'Attribute'), even + // though the page was excluded. See validateExcludedWidgetRefs. + tolerateDanglingRefs bool + // Local page/snippet variables (Variables: { $name: Type = 'default' }). // Used to distinguish a $localVar reference from a page parameter when // resolving TextTemplate parameters — local variables must be stored as diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index 81911b4d2b..12b57cda58 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -1500,7 +1500,7 @@ func (pb *pageBuilder) buildClientActionV3(action *ast.ActionV3) (pages.ClientAc // Handle THEN action (show page) if action.ThenAction != nil && action.ThenAction.Type == "showPage" { pageID, err := pb.resolvePageRef(action.ThenAction.Target) - if err != nil { + if err != nil && !pb.danglingRefOK(err) { return nil, mdlerrors.NewBackend("resolve page", err) } createAct.PageID = pageID @@ -1511,7 +1511,7 @@ func (pb *pageBuilder) buildClientActionV3(action *ast.ActionV3) (pages.ClientAc case "showPage": _, err := pb.resolvePageRef(action.Target) - if err != nil { + if err != nil && !pb.danglingRefOK(err) { return nil, mdlerrors.NewBackend("resolve page", err) } @@ -1565,7 +1565,7 @@ func (pb *pageBuilder) buildClientActionV3(action *ast.ActionV3) (pages.ClientAc case "microflow": mfID, err := pb.resolveMicroflow(action.Target) - if err != nil { + if err != nil && !pb.danglingRefOK(err) { return nil, mdlerrors.NewBackend("resolve microflow", err) } @@ -1609,7 +1609,7 @@ func (pb *pageBuilder) buildClientActionV3(action *ast.ActionV3) (pages.ClientAc case "nanoflow": nfID, err := pb.resolveNanoflowByName(action.Target) - if err != nil { + if err != nil && !pb.danglingRefOK(err) { return nil, mdlerrors.NewBackend("resolve nanoflow", err) } diff --git a/mdl/executor/cmd_pages_builder_v3_widgets.go b/mdl/executor/cmd_pages_builder_v3_widgets.go index 9d0d0a3c12..3a02960f8b 100644 --- a/mdl/executor/cmd_pages_builder_v3_widgets.go +++ b/mdl/executor/cmd_pages_builder_v3_widgets.go @@ -1133,7 +1133,7 @@ func (pb *pageBuilder) buildSnippetCallV3(w *ast.WidgetV3) (*pages.SnippetCallWi snippetName := w.GetSnippet() if snippetName != "" { snippetID, err := pb.resolveSnippetRef(snippetName) - if err != nil { + if err != nil && !pb.danglingRefOK(err) { return nil, mdlerrors.NewBackend(fmt.Sprintf("resolve snippet %s", snippetName), err) } sc.SnippetID = snippetID diff --git a/mdl/executor/cmd_pages_create_v3.go b/mdl/executor/cmd_pages_create_v3.go index 1226a3d185..7179a945b4 100644 --- a/mdl/executor/cmd_pages_create_v3.go +++ b/mdl/executor/cmd_pages_create_v3.go @@ -113,6 +113,10 @@ func execCreatePageV3(ctx *ExecContext, s *ast.CreatePageStmtV3) error { // The root of a document that this pass walks in full: there is no // enclosing data widget, so there is no context object. #1029. argCtx: atDocumentRoot(), + // An excluded page may name documents that do not exist; Mendix does + // not validate it, and the check reports them as warnings. The page + // stays excluded whether the statement says so or the carry does. + tolerateDanglingRefs: s.Excluded || existingExcluded, } page, err := pb.buildPageV3(s) @@ -252,6 +256,9 @@ func execCreateSnippetV3(ctx *ExecContext, s *ast.CreateSnippetStmtV3) error { // The root of a document that this pass walks in full: there is no // enclosing data widget, so there is no context object. #1029. argCtx: atDocumentRoot(), + // A snippet has no @excluded of its own; it stays excluded through the + // carry, and an excluded one may name documents that do not exist. + tolerateDanglingRefs: existingExcluded, } snippet, err := pb.buildSnippetV3(s) diff --git a/mdl/executor/helpers.go b/mdl/executor/helpers.go index 3265cfbd31..f27bb0c48c 100644 --- a/mdl/executor/helpers.go +++ b/mdl/executor/helpers.go @@ -290,6 +290,18 @@ type widgetRefCollector struct { entities []string images []string menus []string + // dataSources holds the subset of the references above that a widget's + // DATA SOURCE names (flow or entity). An excluded document may keep a + // dangling action target, but not a dangling data source — see + // splitExcludedWidgetRefs. + dataSources map[string]bool +} + +func (c *widgetRefCollector) addDataSource(ref string) { + if c.dataSources == nil { + c.dataSources = map[string]bool{} + } + c.dataSources[ref] = true } // dedupe collapses repeated references within each category, preserving first @@ -343,14 +355,17 @@ func (c *widgetRefCollector) collectFromWidget(w *ast.WidgetV3) { case "microflow": if ds.Reference != "" { c.microflows = append(c.microflows, ds.Reference) + c.addDataSource(ds.Reference) } case "nanoflow": if ds.Reference != "" { c.nanoflows = append(c.nanoflows, ds.Reference) + c.addDataSource(ds.Reference) } case "database": if ds.Reference != "" { c.entities = append(c.entities, ds.Reference) + c.addDataSource(ds.Reference) } } } diff --git a/mdl/executor/validate.go b/mdl/executor/validate.go index eb98fd31b0..abf45b7515 100644 --- a/mdl/executor/validate.go +++ b/mdl/executor/validate.go @@ -71,6 +71,11 @@ type scriptContext struct { associations map[string]string // Association (unqualified) -> Module.Association entityAttrs map[string]map[string]bool // Module.Entity -> attribute names ambiguousAssc map[string]bool // names defined in more than one module + + // warnings are findings that do not block: dangling references in an + // EXCLUDED document, which Mendix itself does not validate. Reported so + // that relaxing the check hides nothing. + warnings []string } // newScriptContext creates a new script context. @@ -305,8 +310,15 @@ func (sc *scriptContext) has(name string) bool { // validateProgram validates all statements in a program, skipping references // to objects that are defined within the script itself. func validateProgram(ctx *ExecContext, prog *ast.Program) []error { + errs, _ := validateProgramWithWarnings(ctx, prog) + return errs +} + +// validateProgramWithWarnings is validateProgram that also returns the findings +// that do not block — the dangling references of excluded documents. +func validateProgramWithWarnings(ctx *ExecContext, prog *ast.Program) ([]error, []string) { if !ctx.Connected() { - return []error{mdlerrors.NewNotConnected()} + return []error{mdlerrors.NewNotConnected()}, nil } // Collect all objects defined in the script @@ -355,7 +367,7 @@ func validateProgram(ctx *ExecContext, prog *ast.Program) []error { // widget that is already stored, so its property can only be resolved // against the document — which is why it passed check and failed exec. errors = append(errors, validateAlterSetProperties(ctx, prog, sc)...) - return errors + return errors, sc.warnings } // validateForwardPageRefs catches widget `show_page` actions whose target page @@ -429,6 +441,13 @@ func (e *Executor) ValidateProgram(prog *ast.Program) []error { return validateProgram(e.newExecContext(context.Background()), prog) } +// ValidateProgramWithWarnings is ValidateProgram plus the findings that do not +// block: unresolved references inside EXCLUDED documents, which Mendix does +// not validate. Callers print them so that nothing the check relaxed is hidden. +func (e *Executor) ValidateProgramWithWarnings(prog *ast.Program) ([]error, []string) { + return validateProgramWithWarnings(e.newExecContext(context.Background()), prog) +} + // CheckProjectConflicts walks prog in statement order and returns errors for // any plain CREATE (non-OR-MODIFY) that targets a document name that already // exists in the connected project. Names created earlier in the same script are @@ -572,9 +591,13 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext s.Name.String(), strings.Join(validationErrors, "\n - ")) } // Validate references inside microflow body (pages, microflows, java actions, entities) - if refErrors := validateMicroflowReferences(ctx, s, sc); len(refErrors) > 0 { - return mdlerrors.NewValidationf("microflow '%s' has reference errors:\n - %s", - s.Name.String(), strings.Join(refErrors, "\n - ")) + if refErrors := validateFlowBodyReferences(ctx, s.Body, sc); len(refErrors) > 0 { + if s.Excluded { + sc.warnExcluded("microflow", s.Name.String(), refErrors) + } else { + return mdlerrors.NewValidationf("microflow '%s' has reference errors:\n - %s", + s.Name.String(), strings.Join(refErrors, "\n - ")) + } } case *ast.CreateRuleStmt: if s.Name.Module != "" && !sc.modules[s.Name.Module] { @@ -591,8 +614,10 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext return mdlerrors.NewValidationf("rule '%s' has validation errors:\n - %s", s.Name.String(), strings.Join(validationErrors, "\n - ")) } - if !s.Excluded { - if refErrors := validateFlowBodyReferences(ctx, s.Body, sc); len(refErrors) > 0 { + if refErrors := validateFlowBodyReferences(ctx, s.Body, sc); len(refErrors) > 0 { + if s.Excluded { + sc.warnExcluded("rule", s.Name.String(), refErrors) + } else { return mdlerrors.NewValidationf("rule '%s' has reference errors:\n - %s", s.Name.String(), strings.Join(refErrors, "\n - ")) } @@ -608,9 +633,11 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext return mdlerrors.NewValidationf("nanoflow '%s' has validation errors:\n - %s", s.Name.String(), strings.Join(validationErrors, "\n - ")) } - // Validate references inside nanoflow body (skip excluded nanoflows) - if !s.Excluded { - if refErrors := validateFlowBodyReferences(ctx, s.Body, sc); len(refErrors) > 0 { + // Validate references inside nanoflow body (an excluded nanoflow's are warnings) + if refErrors := validateFlowBodyReferences(ctx, s.Body, sc); len(refErrors) > 0 { + if s.Excluded { + sc.warnExcluded("nanoflow", s.Name.String(), refErrors) + } else { return mdlerrors.NewValidationf("nanoflow '%s' has reference errors:\n - %s", s.Name.String(), strings.Join(refErrors, "\n - ")) } @@ -623,10 +650,21 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext } // Every widget-bearing field, not just the bare body — see pageWidgets. pageWidgets := allPageWidgets(s) - // Validate widget references (DataSource, Action, Snippet) + // Validate widget references (DataSource, Action, Snippet). An EXCLUDED + // page may name documents that do not exist — Mendix does not validate + // excluded documents, and a marketplace module can ship one as an + // example (Feedback v4.0.2's ShareFeedback_Logo) — so for it they are + // warnings. "Excluded" is what exec will WRITE: the statement's + // @excluded, or the exclusion carried from the stored page (#914). if refErrors := validateWidgetReferences(ctx, pageWidgets, sc); len(refErrors) > 0 { - return mdlerrors.NewValidationf("page '%s' has reference errors:\n - %s", - s.Name.String(), strings.Join(refErrors, "\n - ")) + if s.Excluded || carriedExclusion(ctx, "page", s.Name, s.IsReplace || s.IsModify) { + if err := sc.relaxExcludedWidgetRefs("page", s.Name.String(), pageWidgets, refErrors); err != nil { + return err + } + } else { + return mdlerrors.NewValidationf("page '%s' has reference errors:\n - %s", + s.Name.String(), strings.Join(refErrors, "\n - ")) + } } // Validate page context tree (parameter/selection/attribute bindings) if ctxErrors := validatePageContextTree(ctx, s.Parameters, pageWidgets); len(ctxErrors) > 0 { @@ -655,10 +693,18 @@ func validateWithContext(ctx *ExecContext, stmt ast.Statement, sc *scriptContext return mdlerrors.NewNotFound("module", s.Name.Module) } } - // Validate widget references (DataSource, Action, Snippet) + // Validate widget references (DataSource, Action, Snippet). A snippet + // has no @excluded of its own; one exec keeps excluded (the carry) is + // treated as an excluded page is. if refErrors := validateWidgetReferences(ctx, s.Widgets, sc); len(refErrors) > 0 { - return mdlerrors.NewValidationf("snippet '%s' has reference errors:\n - %s", - s.Name.String(), strings.Join(refErrors, "\n - ")) + if carriedExclusion(ctx, "snippet", s.Name, s.IsReplace || s.IsModify) { + if err := sc.relaxExcludedWidgetRefs("snippet", s.Name.String(), s.Widgets, refErrors); err != nil { + return err + } + } else { + return mdlerrors.NewValidationf("snippet '%s' has reference errors:\n - %s", + s.Name.String(), strings.Join(refErrors, "\n - ")) + } } // A snippet takes the same data sources and actions a page does, and // CE1571 does not care which document the widget lives in. @@ -822,16 +868,108 @@ func (e *Executor) Validate(stmt ast.Statement) error { // Microflow Body Reference Validation // ---------------------------------------------------------------------------- -// validateMicroflowReferences validates that all qualified name references in a -// microflow body (pages, microflows, java actions, entities) point to existing objects. -func validateMicroflowReferences(ctx *ExecContext, s *ast.CreateMicroflowStmt, sc *scriptContext) []string { - if s.Excluded { - // Studio Pro allows excluded documents to keep stale references. Reference - // checks should not fail a roundtrip audit for microflows that are not part - // of the runnable app. +// warnExcluded records the dangling references of an EXCLUDED document as +// warnings. Mendix does not validate excluded documents (an untouched project +// holding one passes `mx check` with 0 errors), so they must not block exec or +// `check --references` — but they are reported, so relaxing hides nothing. +func (sc *scriptContext) warnExcluded(kind, name string, refErrors []string) { + sc.warnings = append(sc.warnings, fmt.Sprintf( + "%s '%s' is excluded, so its unresolved references do not block (Mendix does not validate excluded documents):\n - %s", + kind, name, strings.Join(refErrors, "\n - "))) +} + +// relaxExcludedWidgetRefs handles the unresolved widget references of an +// EXCLUDED page or snippet: a dangling action target or snippet call becomes a +// warning (the writer stores it by name, and Mendix does not validate the +// document), while a dangling DATA SOURCE still blocks. +// +// The data source is the exception because it is not just a name: its flow's +// return type, or its entity, is what the widgets inside the container bind +// against. Without it the builder cannot qualify those bindings, and DESCRIBE +// has already printed them bare. Measured on Mendix 11.13.0 with Feedback +// v4.0.2's ShareFeedback_Logo: writing it anyway left an image URL parameter +// bound to a bare `ImageB64`, and `mx check` could no longer LOAD the project +// (ArgumentNullException setting 'Attribute') — excluded or not. +func (sc *scriptContext) relaxExcludedWidgetRefs(kind, name string, widgets []*ast.WidgetV3, refErrors []string) error { + refs := &widgetRefCollector{} + refs.collectFromWidgets(widgets) + var blocking, warnings []string + for _, e := range refErrors { + ref := e[strings.LastIndex(e, ": ")+2:] + switch { + case refs.dataSources[ref]: + blocking = append(blocking, e+" (data source)") + case strings.HasPrefix(e, "entity not found"): + // The builder resolves an entity to write it (create_object and the + // like), so exec would refuse it anyway; say so here instead of + // letting check pass what exec then fails. + blocking = append(blocking, e) + default: + warnings = append(warnings, e) + } + } + if len(warnings) > 0 { + sc.warnExcluded(kind, name, warnings) + } + if len(blocking) == 0 { return nil } - return validateFlowBodyReferences(ctx, s.Body, sc) + return mdlerrors.NewValidationf("%s '%s' is excluded, but a data source or entity it names does not exist:\n - %s\n"+ + " An excluded document may keep a dangling action target, but not a dangling data source: the\n"+ + " flow or entity decides what the widgets inside it bind to, and without it those bindings are\n"+ + " written unqualified — which leaves a project Mendix cannot load. Create the missing document,\n"+ + " or leave this %s as it is stored.", + kind, name, strings.Join(blocking, "\n - "), kind) +} + +// carriedExclusion reports whether exec will write the page or snippet named +// qn as EXCLUDED without the statement saying so: a CREATE OR REPLACE/MODIFY +// whose every stored namesake is excluded rewrites the first of them in place +// and keeps it excluded (#914, execCreatePageV3 / execCreateSnippetV3). A live +// namesake is the one rewritten, and it stays live. +func carriedExclusion(ctx *ExecContext, kind string, qn ast.QualifiedName, rewrite bool) bool { + if !rewrite || !ctx.Connected() { + return false + } + h, err := getHierarchy(ctx) + if err != nil { + return false + } + type doc struct { + container model.ID + name string + excluded bool + } + var docs []doc + switch kind { + case "page": + pgs, err := ctx.Backend.ListPages() + if err != nil { + return false + } + for _, p := range pgs { + docs = append(docs, doc{p.ContainerID, p.Name, p.Excluded}) + } + case "snippet": + snips, err := ctx.Backend.ListSnippets() + if err != nil { + return false + } + for _, sn := range snips { + docs = append(docs, doc{sn.ContainerID, sn.Name, sn.Excluded}) + } + } + excluded := false + for _, d := range docs { + if d.name != qn.Name || h.GetModuleName(h.FindModuleID(d.container)) != qn.Module { + continue + } + if !d.excluded { + return false + } + excluded = true + } + return excluded } // validateFlowBodyReferences validates references in any flow body (microflow or nanoflow). diff --git a/mdl/executor/validate_excluded_page_test.go b/mdl/executor/validate_excluded_page_test.go new file mode 100644 index 0000000000..2b772141c8 --- /dev/null +++ b/mdl/executor/validate_excluded_page_test.go @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/microflows" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// An EXCLUDED page may reference documents that do not exist. Mendix does not +// validate excluded documents: Feedback v4.0.2 ships +// FeedbackModule.ShareFeedback_Logo as an excluded example page bound to five +// nanoflows the module does not contain, and the untouched project passes +// `mx check` with 0 errors. describe → exec of that page was refused: +// +// Reference error: statement 1: page 'FeedbackModule.ShareFeedback_Logo' has reference errors: +// - nanoflow not found: FeedbackModule.DS_FeedbackForm +// - nanoflow not found: FeedbackModule.ACT_TriggerScreenshotMode +// ... +// +// and with --no-check the page builder refused the same names a second time +// ("failed to resolve nanoflow"). The references are still reported — as +// warnings — so nothing is hidden. +// +// Except a DATA SOURCE: its flow is what puts an entity in scope, and writing a +// container without one left unqualified attribute bindings that made mx +// unable to LOAD the project (measured, 11.13.0). That one still blocks. + +func excludedPageStmt(excluded bool) *ast.CreatePageStmtV3 { + return &ast.CreatePageStmtV3{ + Name: ast.QualifiedName{Module: "Feedback", Name: "ShareFeedback_Logo"}, + IsModify: true, + Excluded: excluded, + Widgets: actionWidget("nanoflow", "Feedback.ACT_ClearForm"), + } +} + +func TestValidateExcludedPage_DanglingRefsAreWarnings(t *testing.T) { + ctx, _ := newMockCtx(t) + sc := newScriptContext() + sc.modules["Feedback"] = true + + if err := validateWithContext(ctx, excludedPageStmt(true), sc); err != nil { + t.Fatalf("an excluded page's dangling reference must not block; got:\n%v", err) + } + if len(sc.warnings) != 1 || !strings.Contains(sc.warnings[0], "nanoflow not found: Feedback.ACT_ClearForm") || + !strings.Contains(sc.warnings[0], "excluded") { + t.Errorf("the dangling reference must still be reported as a warning; got %q", sc.warnings) + } +} + +// CONTROL: the same page, not excluded, is still refused. +func TestValidateLivePage_DanglingRefsAreErrors(t *testing.T) { + ctx, _ := newMockCtx(t) + sc := newScriptContext() + sc.modules["Feedback"] = true + + err := validateWithContext(ctx, excludedPageStmt(false), sc) + if err == nil || !strings.Contains(err.Error(), "has reference errors") || + !strings.Contains(err.Error(), "nanoflow not found: Feedback.ACT_ClearForm") { + t.Fatalf("a live page's dangling reference must be refused; got %v", err) + } + if len(sc.warnings) != 0 { + t.Errorf("no warnings expected for a refused statement; got %q", sc.warnings) + } +} + +// exec keeps a page excluded when every stored page of that name is excluded, +// even if the statement does not say @excluded (#914). The check has to agree +// with what exec will write, not only with what the statement says. +func TestValidateCarriedExclusion_DanglingRefsAreWarnings(t *testing.T) { + mod := &model.Module{BaseElement: model.BaseElement{ID: "mod1"}, Name: "Feedback"} + stored := &pages.Page{BaseElement: model.BaseElement{ID: "pg1"}, ContainerID: "mod1", + Name: "ShareFeedback_Logo", Excluded: true} + ctx, _ := newMockCtx(t, withBackend(&mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListPagesFunc: func() ([]*pages.Page, error) { return []*pages.Page{stored}, nil }, + })) + sc := newScriptContext() + + if err := validateWithContext(ctx, excludedPageStmt(false), sc); err != nil { + t.Fatalf("a page exec keeps excluded must not be refused; got:\n%v", err) + } + if len(sc.warnings) != 1 { + t.Errorf("want one warning, got %q", sc.warnings) + } + + // CONTROL: a plain CREATE does not carry anything, and a live twin wins. + stored.Excluded = false + if err := validateWithContext(ctx, excludedPageStmt(false), newScriptContext()); err == nil { + t.Error("with a live page of the same name the rewrite is live, and must be refused") + } +} + +// The builder is the second refusal: resolveNanoflowByName fails on the +// missing name. For an excluded page the name is kept (the writer stores it +// BY_NAME; the ID is never serialized) and the build continues. +func TestBuildExcludedPage_DanglingFlowIsKeptByName(t *testing.T) { + ctx, _ := newMockCtx(t, withBackend(&mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListNanoflowsFunc: func() ([]*microflows.Nanoflow, error) { return nil, nil }, + })) + newPB := func(tolerate bool) *pageBuilder { + pb := newPopupPageBuilder() + pb.ctx, pb.backend = ctx, ctx.Backend + pb.tolerateDanglingRefs = tolerate + return pb + } + cases := []*ast.ActionV3{ + {Type: "nanoflow", Target: "Feedback.ACT_ClearForm"}, + {Type: "microflow", Target: "Feedback.ACT_Missing"}, + {Type: "showPage", Target: "Feedback.Missing_Page"}, + } + for _, act := range cases { + t.Run(act.Type, func(t *testing.T) { + got, err := newPB(true).buildClientActionV3(act) + if err != nil { + t.Fatalf("excluded page: %v", err) + } + var name string + switch a := got.(type) { + case *pages.NanoflowClientAction: + name = a.NanoflowName + case *pages.MicroflowClientAction: + name = a.MicroflowName + case *pages.PageClientAction: + name = a.PageName + } + if name != act.Target { + t.Errorf("reference must be kept by name; got %q", name) + } + // CONTROL: a live page still refuses. + if _, err := newPB(false).buildClientActionV3(act); err == nil { + t.Error("a live page must still refuse a dangling reference") + } + }) + } + + // A data source is never tolerated, excluded or not. + ds := &ast.DataSourceV3{Type: "nanoflow", Reference: "Feedback.DS_FeedbackForm"} + if _, _, err := newPB(true).buildDataSourceV3(ds); err == nil { + t.Error("a dangling data source must be refused even on an excluded page") + } +} + +// A dangling data source on an excluded page still blocks, with the reason; +// the action targets beside it are still only warnings. +func TestValidateExcludedPage_DanglingDataSourceBlocks(t *testing.T) { + ctx, _ := newMockCtx(t) + sc := newScriptContext() + sc.modules["Feedback"] = true + s := excludedPageStmt(true) + s.Widgets = []*ast.WidgetV3{{ + Name: "dv", Type: "dataview", + Properties: map[string]any{ + "DataSource": &ast.DataSourceV3{Type: "nanoflow", Reference: "Feedback.DS_FeedbackForm"}, + }, + Children: actionWidget("nanoflow", "Feedback.ACT_ClearForm"), + }} + + err := validateWithContext(ctx, s, sc) + if err == nil || !strings.Contains(err.Error(), "nanoflow not found: Feedback.DS_FeedbackForm (data source)") || + !strings.Contains(err.Error(), "is excluded, but a data source") { + t.Fatalf("want the data source refused with its reason; got %v", err) + } + if strings.Contains(err.Error(), "ACT_ClearForm") { + t.Errorf("the action target must not be in the refusal: %v", err) + } + if len(sc.warnings) != 1 || !strings.Contains(sc.warnings[0], "nanoflow not found: Feedback.ACT_ClearForm") { + t.Errorf("the action target must still be a warning; got %q", sc.warnings) + } +} diff --git a/mdl/executor/validate_script_javaactions_test.go b/mdl/executor/validate_script_javaactions_test.go index 75296efee4..2016e74ec4 100644 --- a/mdl/executor/validate_script_javaactions_test.go +++ b/mdl/executor/validate_script_javaactions_test.go @@ -45,7 +45,7 @@ func TestValidate_JavaActionCreatedInScriptIsNotReportedMissing(t *testing.T) { sc.collectDefinitions(prog) mf := prog.Statements[1].(*ast.CreateMicroflowStmt) - if errs := validateMicroflowReferences(ctx, mf, sc); len(errs) != 0 { + if errs := validateFlowBodyReferences(ctx, mf.Body, sc); len(errs) != 0 { t.Fatalf("reference errors for an action created in the same script: %v", errs) } } @@ -61,7 +61,7 @@ func TestValidate_JavaActionCreatedInScriptStillChecksParameterNames(t *testing. sc.collectDefinitions(prog) mf := prog.Statements[1].(*ast.CreateMicroflowStmt) - errs := validateMicroflowReferences(ctx, mf, sc) + errs := validateFlowBodyReferences(ctx, mf.Body, sc) if len(errs) != 1 || !strings.Contains(errs[0], `has no parameter "Inputt"`) { t.Fatalf("errors = %v, want one complaint about the misspelled parameter", errs) } From 872e42daefa5721e3b6b31acc3a36f2ff5166376 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 09:05:02 +0000 Subject: [PATCH 29/47] fix(check): name the current property for a design-property key the theme renamed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit MDL-WIDGET11 reported Atlas's old design-property keys ("Spacing bottom", "Align content", "Font Weight", …) as "not defined for this widget type" and listed every current key. They are old names, recorded in the theme's design-properties.json as `oldNames` — on a property, on a multi-select option, and per side of a Spacing step as "::". They are not valid either: mxbuild reports CE6087 "Design properties have been renamed in your theme" on any live page carrying them. The Feedback v4.0.2 *_Logo pages only checked clean because they are Excluded; including PopupFailure_Logo reproduces CE6087. The theme reader now decodes `oldNames`, and the rule keeps the warning but names the current property and its spelling with the value mapped ('Spacing': ['margin-bottom': 'M']). A key that is neither current nor an old name still gets "not defined". Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../renamed-design-property-legacy-names.mdl | 54 +++++ mdl/executor/design_property_renames.go | 145 +++++++++++++ mdl/executor/theme_reader.go | 32 +++ mdl/executor/validate_design_properties.go | 17 ++ .../validate_renamed_design_property_test.go | 198 ++++++++++++++++++ 6 files changed, 447 insertions(+) create mode 100644 mdl-examples/bug-tests/renamed-design-property-legacy-names.mdl create mode 100644 mdl/executor/design_property_renames.go create mode 100644 mdl/executor/validate_renamed_design_property_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index ef9d0f0481..da4d01f7f2 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -701,3 +701,4 @@ {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_Edit / FeedbackModule.ShareFeedback(_Logo) fails `Parse error: line 14:21 extraneous input '(' expecting the start of a statement`; describe emitted `statictext (Content: '…')` with no widget name", "cause": "The stored widget is Studio Pro's Label (Forms$Label), and it is NAMED (`label4`). The describe emitter hard-coded `statictext (Content: %s)`, dropping name and appearance. MDL had no widget that writes Forms$Label; `statictext` writes Forms$Text, which Mendix 11 cannot load (MDL-WIDGET29), so neither 'make the name optional' nor 'synthesize a name' could have produced a correct round trip", "file": "`mdl/executor/cmd_pages_describe_output.go` (Forms$Label case), `mdl/grammar/domains/MDLPage.g4` (`LABEL` in widgetTypeV3), `mdl/executor/cmd_pages_builder_v3_widgets.go` (`buildLabelV3`), `mdl/backend/modelsdk/widget_write.go` (`*pages.Label` case + Forms$Label TypeDefaults)", "insight": "**Dump the stored widget before accepting the report's diagnosis**: 'the stored Name is empty' was an inference from the output, and one `bson dump --format ndsl` showed a named Forms$Label — which also made both proposed fixes wrong, since the keyword itself wrote an unloadable type. A generic (IDENTIFIER) widget type parses but `check -p` requires it to resolve to a pluggable definition (MDL-WIDGET25); a built-in widget needs its token in widgetTypeV3, and a visitor test must assert `!TypeIsGeneric` or it passes against the unfixed grammar. gen's Label declares top-level Class/Style/AccessibilitySettings that Studio Pro 11 does not store — assert the encoded key set with encodeToD, and register NullFields for ConditionalVisibilitySettings or the key is omitted. Round-tripping a page that previously failed to parse EXPOSES older write gaps on the same page: here attribute-condition visibility (8 Enumerations$Condition → 0, all widgets), a nanoflow data-view source shape (CE2633), and compound design-property list markers (2 → 3) — diff the stored BSON before/after, not just mx check", "refs": ["Administration.Account_Edit", "FeedbackModule.ShareFeedback"], "rules": ["MDL-WIDGET25", "MDL-WIDGET29"], "date": "2026-09-25"} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`container c1 (dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ])` - the first-class spelling mendixlabs/mxcli#750 proposes - checked clean and `exec` reported `Created page`, but the widget was stored with no DynamicClasses at all. `alter page ... set DynamicClasses = [ ... ] on w` reported `Altered page` and changed nothing; a column's `DynamicCellClass` on ALTER stored `[if$x/Ythen'a'else'b']` as its expression", "cause": "`[ ... ]` parses as propertyValueV3's array alternative, so the value reaches the AST as a []string. The create writers read only a string (GetStringProp for DynamicClasses, `v.(string)` for columnClass); the mutator's dynamicclasses case returned nil when the type check failed; setColumnPropertyMut formatted the list with %v, after the visitor's GetText() had already fused its tokens", "file": "`mdl/executor/validate_widget_expression_list.go` (MDL-WIDGET32), `mdl/backend/pagemutator/mutator.go` (`errExpressionNotAString`)", "insight": "**A proposal's example syntax is worth running through `check` before designing around it** - here the proposed spelling already parsed, and the parse was the bug. Same class as #999 / MDL-WIDGET27 (empty `[]`): a value shape no writer claims. Fix at both ends and they stay in step: a no-project check rule for CREATE, and an error (not a nil return) from the ALTER setter, which `check -p` reports for free because validateAlterSetProperties dry-runs the setter. Key the rule on the property, never on the brackets - `visible: [cond]` is valid MDL. Measured with pre-fix and fixed binaries on copies of ako/TestApp: pre-fix `describe` shows the container without DynamicClasses and the quoted control intact. Tests `validate_widget_expression_list_test.go`, `pagemutator/expression_list_value_test.go`", "refs": ["mendixlabs/mxcli#750", "#999"], "rules": ["MDL-WIDGET32"]} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_Edit takes the page's 8 Enumerations$Condition entries to 0: every widget with Studio Pro's \"Visible: based on attribute value\" becomes ALWAYS visible; check, exec and mx check all report success", "cause": "MDL had no spelling for attribute-based conditional visibility. extractConditionalSettings read only Expression, conditionalVisibilityToGen wrote only Expression, and the settings' list markers were the default [3] where Studio Pro stores Conditions [2] and ModuleRoles [1]", "file": "`mdl/grammar/domains/MDLPage.g4` (`VISIBLE COLON attributePathV3 IN (…)`), `mdl/executor/cmd_pages_builder_visible_when.go` (`applyVisibleWhen`), `mdl/backend/modelsdk/widget_write.go` (`conditionalVisibilityToGen`, TypeDefaults), `mdl/executor/cmd_pages_describe_parse.go` / `_output.go` (`visibleWhenProp`)", "insight": "**A dropped visibility setting is invisible to every check**: the model stays valid, the widget just shows for everyone — count `Enumerations$Condition` in `bson dump --format ndsl` before and after a round trip, since mx check never will. Survey the corpus before designing syntax: all 12 settings in the stock project were attribute-based (booleans and one enum), none role-based or editability, which scoped the feature. Studio Pro stores EVERY value (enum values in declaration order plus \"(empty)\", or true/false) with a flag, so MDL lists only the shown values and the writer fills the rest from the domain model — and a byte-identical before/after diff of the settings block (markers included) on 3 pages is the proof. mdlIdent quotes `empty`/`true`/`false` as keywords; emit them bare in a value list. A new AST property key must be added to the known-property list (validate_widgets.go) or MDL-WIDGET07 falsely warns it is dropped", "refs": ["Administration.Account_Edit", "Administration.ScheduledEvents"], "rules": ["MDL-WIDGET07"], "date": "2026-09-25"} +{"area":"mdl/executor","date":"2026-09-25","symptom":"describe → `check --references` of FeedbackModule.ShareFeedback_Logo (Feedback v4.0.2, Mendix 11.13.0) warns MDL-WIDGET11 \"sets design property \\\"Spacing bottom\\\", which is not defined for this widget type\" (×5) and \"Align content\" (×1), suggesting a list of unrelated keys; 12 such warnings across the three *_Logo pages. mx check on the untouched project reports 0 errors, so the Studio Pro-authored values looked valid and the warning like a false positive","cause":"Not a false positive — a misdiagnosis. The keys are Atlas Core's OLD names, kept in themesource/atlas_core/web/design-properties.json as `oldNames`: \"Align content\" on DivContainer's \"Align content (deprecated)\", and \"Spacing bottom::Outer medium\" on the bottom side of the M margin step of Widget's \"Spacing\". The *_Logo pages are Excluded, which is the only reason mx check was quiet: flipping PopupFailure_Logo's Excluded byte makes mxbuild report CE6087 \"Design properties have been renamed in your theme and need to be updated\", as does any live page written with these keys. The theme reader never decoded `oldNames`, so the rule could only say \"not defined\"","file":"`mdl/executor/design_property_renames.go` (new: `findRenamedThemeProp`), `theme_reader.go` (`OldNames` on property/option, `Margin`/`Padding` spacing steps), `validate_design_properties.go`","insight":"\"mx check is clean on the untouched project\" says nothing about an Excluded document — check the Excluded column (`show pages`) before calling a Studio Pro value valid, and include the page (the Excluded bool is one BSON byte, `\\x08Excluded\\x00\\x01`) to get mxbuild's real answer. The warning stays (CE6087 is an error on a live page) but now names the current property and its spelling, value mapped through the old names: 'Spacing': ['margin-bottom': 'M']. Cross-check: Studio Pro's own migrated sibling FeedbackModule.ShareFeedback stores exactly the suggested forms for the same widgets. Typo control kept: a key that is neither current nor old (\"Spacing bottomx\", measured CE6083) still gets \"not defined\"; an option oldName on a non-multi-select property is an old VALUE, not a key. Seen but not fixed: MDL `label` has no design-props key mapping, so its legacy 'Spacing bottom' goes unvalidated; ALTER STYLING still words an old name as CE6083","refs":["FeedbackModule.ShareFeedback_Logo","mdl-examples/bug-tests/renamed-design-property-legacy-names.mdl"],"rules":["MDL-WIDGET11"],"ce":["CE6087","CE6083"]} diff --git a/mdl-examples/bug-tests/renamed-design-property-legacy-names.mdl b/mdl-examples/bug-tests/renamed-design-property-legacy-names.mdl new file mode 100644 index 0000000000..d65c87e36c --- /dev/null +++ b/mdl-examples/bug-tests/renamed-design-property-legacy-names.mdl @@ -0,0 +1,54 @@ +-- @version: 10.0+ +-- ============================================================================ +-- Design-property keys the theme has RENAMED, reported as "not defined". +-- +-- Symptom: describe → check --references of FeedbackModule.ShareFeedback_Logo +-- (Feedback v4.0.2, Mendix 11.13.0, Atlas Core 4.1.3) printed six MDL-WIDGET11 +-- "sets design property "Spacing bottom", which is not defined for this widget +-- type", plus one for "Align content", and suggested a list of unrelated keys. +-- mx check on the untouched project reports 0 errors, so the values looked +-- valid. +-- +-- Cause: they are not valid — they are Atlas's OLD names. Atlas Core's +-- design-properties.json keeps them as `oldNames`: "Align content" on the +-- DivContainer property "Align content (deprecated)", and "Spacing +-- bottom::Outer medium" on the bottom side of the "M" margin of the Widget +-- property "Spacing". mx check was silent only because the three pages that +-- carry them are Excluded. Including the Studio Pro-authored +-- PopupFailure_Logo makes mxbuild report CE6087 "Design properties have been +-- renamed in your theme and need to be updated"; so does any live page +-- written with these keys. The theme reader did not decode `oldNames`, so the +-- rule could only say "not defined" and could not say what the key became. +-- +-- Fix: the theme reader decodes `oldNames` (property, option and Spacing side +-- level). MDL-WIDGET11 still fires for a renamed key — mxbuild refuses it on +-- a live page — but names the current property, cites CE6087, and suggests +-- the current spelling (`'Spacing': ['margin-bottom': 'M']`). A key that is +-- neither current nor an old name keeps the "not defined" message. +-- +-- Verify: `mxcli check` this with -p on an Atlas Core 4.x project: the excluded +-- page gets MDL-WIDGET11 "renamed … to "Spacing"" / "… "Align content +-- (deprecated)"" warnings with the replacement; the live page is clean. +-- exec it, then `mxcli docker check`: 0 errors (the legacy page is excluded). +-- ============================================================================ + +create module BugRenamedDesignProp; + +@excluded +create page BugRenamedDesignProp.LegacyNames ( + title: 'Legacy Names', + layout: Atlas_Core.Atlas_Default +) { + container container1 (designproperties: ['Align content': 'Right align as a row', 'Spacing bottom': 'Outer medium']) { + dynamictext text1 (content: 'Legacy', designproperties: ['Spacing top': 'Outer small', 'Font Weight': 'Light']) + } +} + +create page BugRenamedDesignProp.CurrentNames ( + title: 'Current Names', + layout: Atlas_Core.Atlas_Default +) { + container container1 (designproperties: ['Align content (deprecated)': 'Right align as a row', 'Spacing': ['margin-bottom': 'M']]) { + dynamictext text1 (content: 'Current', designproperties: ['Spacing': ['margin-top': 'S'], 'Weight': 'Light']) + } +} diff --git a/mdl/executor/design_property_renames.go b/mdl/executor/design_property_renames.go new file mode 100644 index 0000000000..d9eaaf5116 --- /dev/null +++ b/mdl/executor/design_property_renames.go @@ -0,0 +1,145 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" +) + +// designPropRename is what a stored design-property key that the theme has +// RENAMED became. Themes keep a key's earlier spellings as `oldNames` so Studio +// Pro can offer to update pages authored against an older version; until that +// happens the page keeps the old key, and mxbuild reports CE6087 "Design +// properties have been renamed in your theme and need to be updated" on it — +// unless the page is excluded, which is why a project full of them can still +// check clean (Feedback v4.0.2's *_Logo example pages on Atlas Core 4.1.3). +type designPropRename struct { + // NewKey is the current property name. + NewKey string + // Replacement is the current MDL spelling of the stored key AND value, or + // "" when the value has no current equivalent the theme declares. + Replacement string + // OldValues lists the values the old key took, for when Replacement is "". + OldValues []string +} + +// findRenamedThemeProp reports whether key is an old name of one of props, and +// what to write instead. It reads the three places a theme records one: +// +// - a property's own oldNames ("Align content" → "Align content (deprecated)"), +// with the value mapped through the options' oldNames as well; +// - a Spacing property's per-side oldNames, "::" +// ("Spacing bottom::Outer medium" → Spacing, margin-bottom, M); +// - a multi-select option's oldNames, which are the separate toggles the +// option replaced ("Hide on phone" → Hide on: Phone). +// +// An option's oldNames on an ordinary property are old VALUES, not keys, so they +// are only consulted to map a value. +func findRenamedThemeProp(props []ThemeProperty, key, value string) *designPropRename { + for i := range props { + p := &props[i] + if containsString(p.OldNames, key) { + r := &designPropRename{NewKey: p.Name} + switch { + case strings.EqualFold(value, "on") || strings.EqualFold(value, "off"): + r.Replacement = fmt.Sprintf("'%s': %s", p.Name, strings.ToLower(value)) + case len(p.Options) == 0: + r.Replacement = fmt.Sprintf("'%s': '%s'", p.Name, value) + default: + if opt := currentOptionName(p.Options, value); opt != "" { + r.Replacement = fmt.Sprintf("'%s': '%s'", p.Name, opt) + } + for _, o := range p.Options { + r.OldValues = append(r.OldValues, o.Name) + } + } + return r + } + if p.Type == "Spacing" { + if r := spacingRename(p, key, value); r != nil { + return r + } + } + if p.MultiSelect { + for _, o := range p.Options { + if containsString(o.OldNames, key) { + return &designPropRename{ + NewKey: p.Name, + Replacement: fmt.Sprintf("'%s': ['%s': on]", p.Name, o.Name), + } + } + } + } + } + return nil +} + +// spacingRename maps an old per-side spacing key ("Spacing bottom") to the +// Spacing compound. The side and the step both come from the matching +// "::" entry, so the old value decides the step: "Outer +// medium" is margin M, "Inner large" padding L. +func spacingRename(p *ThemeProperty, key, value string) *designPropRename { + var r *designPropRename + seen := map[string]bool{} + for _, group := range []struct { + kind string + steps []ThemeSpacingStep + }{{"margin", p.Margin}, {"padding", p.Padding}} { + for _, step := range group.steps { + for _, side := range []struct { + name string + s *ThemeSpacingSide + }{{"top", step.Top}, {"right", step.Right}, {"bottom", step.Bottom}, {"left", step.Left}} { + if side.s == nil { + continue + } + for _, old := range side.s.OldNames { + oldKey, oldValue, ok := strings.Cut(old, "::") + if !ok || oldKey != key { + continue + } + if r == nil { + r = &designPropRename{NewKey: p.Name} + } + if oldValue == value && r.Replacement == "" { + r.Replacement = fmt.Sprintf("'%s': ['%s-%s': '%s']", p.Name, group.kind, side.name, step.Name) + } + if !seen[oldValue] { + seen[oldValue] = true + r.OldValues = append(r.OldValues, oldValue) + } + } + } + } + } + return r +} + +// currentOptionName returns the current name of value — itself when it is a +// declared option, the option it was renamed to when it is an old one, "" when +// neither. +func currentOptionName(options []ThemeOption, value string) string { + for _, o := range options { + if o.Name == value { + return o.Name + } + } + for _, o := range options { + if containsString(o.OldNames, value) { + return o.Name + } + } + return "" +} + +// renamedDesignPropSuggestion is the fix for a renamed key: its current +// spelling, or — when the stored value has none — the old values the theme +// maps, so the author can see which one was meant. +func renamedDesignPropSuggestion(r *designPropRename, value string) string { + if r.Replacement != "" { + return fmt.Sprintf("Write it as %s.", r.Replacement) + } + return fmt.Sprintf("Write it under %q. %q has no current equivalent in the theme; the values it maps are: %s", + r.NewKey, value, strings.Join(r.OldValues, ", ")) +} diff --git a/mdl/executor/theme_reader.go b/mdl/executor/theme_reader.go index 95de9c811d..55bc93d466 100644 --- a/mdl/executor/theme_reader.go +++ b/mdl/executor/theme_reader.go @@ -31,12 +31,44 @@ type ThemeProperty struct { // mxbuild refuses with CE6084 "Expected design property Hide on to be of type // Toggle button group, but found Option" (ako/mxcli#511). MultiSelect bool `json:"multiSelect"` + // OldNames are the keys this property had in earlier theme versions. A + // page authored against one still stores the old key, and mxbuild reports + // CE6087 "Design properties have been renamed in your theme and need to be + // updated" on it unless the page is excluded (measured on 11.13.0 with Atlas + // Core 4.1.3, where "Align content" became "Align content (deprecated)"). + OldNames []string `json:"oldNames"` + // Margin and Padding are the steps of a `"type": "Spacing"` property. Its + // old names live on each side of each step, spelled "::": Atlas's one Spacing property replaced the per-side dropdowns + // "Spacing top" … "Spacing left", so an old key maps to one side. + Margin []ThemeSpacingStep `json:"margin"` + Padding []ThemeSpacingStep `json:"padding"` } // ThemeOption represents a single option within a dropdown/picker design property. type ThemeOption struct { Name string `json:"name"` Class string `json:"class"` + // OldNames are earlier names of this option. On an ordinary property they + // are old VALUES ("Left align as row"); on a multi-select property they are + // the separate toggles the option replaced ("Hide on phone" became Hide on: + // Phone), i.e. old KEYS. + OldNames []string `json:"oldNames"` +} + +// ThemeSpacingStep is one step ("None", "S", "M", …) of a Spacing property. +type ThemeSpacingStep struct { + Name string `json:"name"` + Top *ThemeSpacingSide `json:"top"` + Right *ThemeSpacingSide `json:"right"` + Bottom *ThemeSpacingSide `json:"bottom"` + Left *ThemeSpacingSide `json:"left"` +} + +// ThemeSpacingSide is one side of a Spacing step. +type ThemeSpacingSide struct { + Class string `json:"class"` + OldNames []string `json:"oldNames"` } // ThemeRegistry holds all design property definitions loaded from the project's themesource. diff --git a/mdl/executor/validate_design_properties.go b/mdl/executor/validate_design_properties.go index 4ce407a5f0..526dc330d9 100644 --- a/mdl/executor/validate_design_properties.go +++ b/mdl/executor/validate_design_properties.go @@ -133,6 +133,23 @@ func validateWidgetDesignProps(w *ast.WidgetV3, reg *ThemeRegistry, locationPref continue } if tp == nil { + // An OLD name of a current property is not a typo: it is what a page + // authored against an earlier theme version stores. It is still + // flagged — mxbuild refuses it on a live page with CE6087 — but as a + // rename with its current spelling, which "not defined" plus a list + // of every key could not give. + if r := findRenamedThemeProp(props, p.Key, p.Value); r != nil { + out = append(out, linter.Violation{ + RuleID: "MDL-WIDGET11", + Severity: linter.SeverityWarning, + Message: fmt.Sprintf("%s: widget %q (%s) sets design property %q, which the theme has renamed to %q"+ + " — mxbuild reports CE6087 \"Design properties have been renamed in your theme\" unless the page is excluded", + locationPrefix, w.Name, w.Type, p.Key, r.NewKey), + Location: linter.Location{DocumentType: "page", DocumentName: locationPrefix}, + Suggestion: renamedDesignPropSuggestion(r, p.Value), + }) + continue + } out = append(out, linter.Violation{ RuleID: "MDL-WIDGET11", Severity: linter.SeverityWarning, diff --git a/mdl/executor/validate_renamed_design_property_test.go b/mdl/executor/validate_renamed_design_property_test.go new file mode 100644 index 0000000000..c2996de4a7 --- /dev/null +++ b/mdl/executor/validate_renamed_design_property_test.go @@ -0,0 +1,198 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/linter" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// renamedThemeJSON mirrors the shape of Atlas Core 4.1.3's +// themesource/atlas_core/web/design-properties.json: old names live in +// `oldNames` at three levels — on a property (DivContainer "Align content +// (deprecated)"), on an option (DynamicText "Weight"), and on one side of a +// Spacing margin/padding step, spelled "::". +const renamedThemeJSON = `{ + "Widget": [ + { + "name": "Spacing", "type": "Spacing", + "margin": [ + {"name": "None", + "top": {"class": "spacing-outer-top-none", "oldNames": ["Spacing top::None", "Spacing top::Outer none"]}, + "bottom": {"class": "spacing-outer-bottom-none", "oldNames": ["Spacing bottom::None", "Spacing bottom::Outer none"]}}, + {"name": "S", + "top": {"class": "spacing-outer-top", "oldNames": ["Spacing top::Small", "Spacing top::Outer small"]}, + "bottom": {"class": "spacing-outer-bottom", "oldNames": ["Spacing bottom::Small", "Spacing bottom::Outer small"]}}, + {"name": "M", + "top": {"class": "spacing-outer-top-medium", "oldNames": ["Spacing top::Medium", "Spacing top::Outer medium"]}, + "bottom": {"class": "spacing-outer-bottom-medium", "oldNames": ["Spacing bottom::Medium", "Spacing bottom::Outer medium"]}} + ], + "padding": [ + {"name": "L", + "bottom": {"class": "spacing-inner-bottom-large", "oldNames": ["Spacing bottom::Inner large"]}} + ] + }, + {"name": "Align self", "type": "Dropdown", "oldNames": ["Align Self"], + "options": [{"name": "Left", "class": "pull-left"}, {"name": "Right", "class": "pull-right"}]}, + {"name": "Hide on", "type": "ToggleButtonGroup", "multiSelect": true, + "options": [{"name": "Phone", "class": "hide-phone", "oldNames": ["Hide on phone", "Hide On Phone"]}]} + ], + "DivContainer": [ + {"name": "Align content (deprecated)", "type": "Dropdown", "oldNames": ["Align content"], + "options": [ + {"name": "Right align as a row", "oldNames": ["Right align as row"], "class": "row-right"}, + {"name": "Left align as a column", "oldNames": ["Left align as column"], "class": "col-left"} + ]}, + {"name": "Background color", "type": "Dropdown", + "options": [{"name": "Brand Primary", "oldNames": ["Primary"], "class": "background-primary"}]} + ], + "DynamicText": [ + {"name": "Weight", "type": "ToggleButtonGroup", "oldNames": ["Font Weight"], + "options": [{"name": "Light", "class": "text-light"}]} + ] +}` + +func renamedThemeRegistry(t *testing.T) *ThemeRegistry { + t.Helper() + props, err := parseDesignPropertiesJSON([]byte(renamedThemeJSON)) + if err != nil { + t.Fatalf("parse fixture: %v", err) + } + return &ThemeRegistry{WidgetProperties: props} +} + +func allDesignPropViolations(t *testing.T, src string, reg *ThemeRegistry) []linter.Violation { + t.Helper() + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + var out []linter.Violation + for _, stmt := range prog.Statements { + out = append(out, ValidateDesignPropertiesForStatement(stmt, reg)...) + } + return out +} + +func violationFor(vs []linter.Violation, key string) *linter.Violation { + for i := range vs { + if strings.Contains(vs[i].Message, `"`+key+`"`) { + return &vs[i] + } + } + return nil +} + +// TestValidateDesignProperties_RenamedKeyNamesItsReplacement is the +// ShareFeedback_Logo case (Feedback v4.0.2 on Atlas Core 4.1.3): stored keys +// Atlas has RENAMED. mxbuild refuses them on a live page with CE6087 "Design +// properties have been renamed in your theme", so they must still be flagged — +// but as a rename, naming the current property and its current spelling, not as +// "not defined for this widget type" with a list of unrelated keys. +func TestValidateDesignProperties_RenamedKeyNamesItsReplacement(t *testing.T) { + reg := renamedThemeRegistry(t) + vs := allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + container c1 (designproperties: ['Align content': 'Right align as a row', 'Spacing bottom': 'Outer medium']) { + dynamictext t1 (content: 'x', designproperties: ['Spacing top': 'Outer small', 'Font Weight': 'Light']) + } + container c2 (designproperties: ['Spacing bottom': 'Inner large', 'Hide on phone': on, 'Align content': 'Left align as column']) {} +}`, reg) + + cases := []struct { + key, renamedTo, suggestion string + }{ + {"Align content", "Align content (deprecated)", `'Align content (deprecated)': 'Right align as a row'`}, + {"Font Weight", "Weight", `'Weight': 'Light'`}, + } + for _, c := range cases { + v := violationFor(vs, c.key) + if v == nil { + t.Errorf("%q: expected a violation, got none (%d total)", c.key, len(vs)) + continue + } + if v.RuleID != "MDL-WIDGET11" { + t.Errorf("%q: rule %s, want MDL-WIDGET11", c.key, v.RuleID) + } + if strings.Contains(v.Message, "not defined") { + t.Errorf("%q: reported as undefined, want a rename: %s", c.key, v.Message) + } + if !strings.Contains(v.Message, `renamed to "`+c.renamedTo+`"`) || !strings.Contains(v.Message, "CE6087") { + t.Errorf("%q: message should name %q and CE6087: %s", c.key, c.renamedTo, v.Message) + } + if !strings.Contains(v.Suggestion, c.suggestion) { + t.Errorf("%q: suggestion should contain %s, got: %s", c.key, c.suggestion, v.Suggestion) + } + } + + // Every legacy key on the page is flagged exactly once, each as a rename, + // with the value mapped to the current spelling. + wantSuggestions := []string{ + `'Spacing': ['margin-bottom': 'M']`, // Spacing bottom: Outer medium + `'Spacing': ['margin-top': 'S']`, // Spacing top: Outer small + `'Spacing': ['padding-bottom': 'L']`, // Spacing bottom: Inner large + `'Hide on': ['Phone': on]`, // multi-select option's old toggle + `'Align content (deprecated)': 'Left align as a column'`, // old key AND old value + } + for _, want := range wantSuggestions { + found := false + for _, v := range vs { + if strings.Contains(v.Suggestion, want) { + found = true + } + } + if !found { + t.Errorf("no violation suggests %s", want) + } + } + if len(vs) != 7 { + t.Errorf("expected 7 violations (one per legacy key), got %d", len(vs)) + } + for _, v := range vs { + if strings.Contains(v.Message, "not defined") { + t.Errorf("legacy key reported as undefined: %s", v.Message) + } + } +} + +// TestValidateDesignProperties_UnknownKeyStillUndefined is the control: a key +// that is neither current nor an old name keeps the "not defined" warning, so +// recognising old names does not weaken the typo check. mxbuild reports these +// as CE6083 "not supported by your theme" (measured on 11.13.0). +func TestValidateDesignProperties_UnknownKeyStillUndefined(t *testing.T) { + reg := renamedThemeRegistry(t) + vs := allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + container c1 (designproperties: ['Spacing bottomx': 'Outer medium', 'Primary': on]) {} +}`, reg) + for _, key := range []string{"Spacing bottomx", "Primary"} { + v := violationFor(vs, key) + if v == nil || v.RuleID != "MDL-WIDGET11" || !strings.Contains(v.Message, "not defined") { + t.Errorf("%q: expected MDL-WIDGET11 \"not defined\", got %+v", key, v) + } + } + + // A legacy Spacing key with a value that no old name covers is still a + // rename, but its value has no current spelling to offer. + vs = allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + container c1 (designproperties: ['Spacing bottom': 'Bogus']) {} +}`, reg) + v := violationFor(vs, "Spacing bottom") + if v == nil || !strings.Contains(v.Message, `renamed to "Spacing"`) { + t.Fatalf("expected a rename for Spacing bottom, got %+v", v) + } + if !strings.Contains(v.Suggestion, "Outer medium") { + t.Errorf("unmapped value should list the old values it accepted, got: %s", v.Suggestion) + } + + // Current names are clean. + vs = allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + container c1 (designproperties: ['Align content (deprecated)': 'Right align as a row', 'Spacing': ['margin-bottom': 'M']]) { + dynamictext t1 (content: 'x', designproperties: ['Weight': 'Light']) + } +}`, reg) + if len(vs) != 0 { + t.Errorf("current names should be clean, got %v", vs) + } +} From 1984c9c168e21029480e1dbf71645e2251d7b381 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 09:05:59 +0000 Subject: [PATCH 30/47] fix(pages): round-trip a page over a flow the project lacks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit describe → exec of Feedback v4.0.2's EXCLUDED FeedbackModule.ShareFeedback_Logo was refused: "nanoflow not found: FeedbackModule.DS_FeedbackForm (data source)". Forcing it through left a project `mx check` could not LOAD (ArgumentNullException setting 'Attribute'). With the flow missing, DESCRIBE had no context entity and printed every binding inside the data view bare (`Attribute: Subject`, `{1} = ImageB64`, `Visible: _showEmail in (…)`). Exec had nothing to qualify them against. The stored model always has the full names. - DESCRIBE keeps the stored Module.Entity.Attr for bindings inside a data view, list view or gallery whose flow cannot be resolved. This covers Attribute:, template parameters, pluggable attribute properties, object-list items and the attribute visibility condition. - `Visible: Module.Entity.Attr in (…)` is accepted and needs no entity in scope. - On an excluded page a missing data-source flow is kept by name, like the action targets in the previous commit. - The check reports it as a warning when every binding inside is qualified. Otherwise it refuses and names the bare binding and its widget. - As a last line, the page and snippet writer refuses any DomainModels$AttributeRef that is not Module.Entity.Attr. Studio Pro qualifies every one (72 of 72 across the project), and a bare one takes the loader down. ShareFeedback_Logo now round-trips: warnings only, all 6 attribute references identical to the stored ones, the page still excluded, and `mx check` 0 errors. A forced-fault variant with one binding hand-edited back to bare is refused, naming the widget. All 17 pages of the project now exec. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .claude/skills/mendix/check-syntax/SKILL.md | 10 +- docs-site/src/tutorial/validation.md | 8 +- ...age-unresolved-flow-qualified-bindings.mdl | 46 ++++++++ .../modelsdk/page_bare_attributeref.go | 59 ++++++++++ .../modelsdk/page_bare_attributeref_test.go | 47 ++++++++ mdl/backend/modelsdk/page_write.go | 9 +- mdl/backend/modelsdk/snippet_write.go | 9 +- mdl/executor/cmd_pages_builder.go | 13 +-- mdl/executor/cmd_pages_builder_v3.go | 8 +- .../cmd_pages_builder_visible_when.go | 24 +++-- .../cmd_pages_describe_flowcontext.go | 37 +++++++ mdl/executor/cmd_pages_describe_objectlist.go | 2 +- mdl/executor/cmd_pages_describe_output.go | 10 +- mdl/executor/cmd_pages_describe_parse.go | 31 ++++-- mdl/executor/cmd_pages_describe_pluggable.go | 8 +- ..._pages_describe_unresolved_context_test.go | 94 ++++++++++++++++ mdl/executor/cmd_pages_visible_when_test.go | 25 +++++ mdl/executor/exec_context.go | 6 ++ mdl/executor/validate.go | 102 ++++++++++++++++-- mdl/executor/validate_excluded_page_test.go | 63 +++++++++-- mdl/grammar/domains/MDLPage.g4 | 2 +- mdl/visitor/visitor_page_v3.go | 2 +- 23 files changed, 560 insertions(+), 56 deletions(-) create mode 100644 mdl-examples/bug-tests/excluded-page-unresolved-flow-qualified-bindings.mdl create mode 100644 mdl/backend/modelsdk/page_bare_attributeref.go create mode 100644 mdl/backend/modelsdk/page_bare_attributeref_test.go create mode 100644 mdl/executor/cmd_pages_describe_unresolved_context_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 8d6bc88da1..acbb71c1ca 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -702,3 +702,4 @@ {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`container c1 (dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ])` - the first-class spelling mendixlabs/mxcli#750 proposes - checked clean and `exec` reported `Created page`, but the widget was stored with no DynamicClasses at all. `alter page ... set DynamicClasses = [ ... ] on w` reported `Altered page` and changed nothing; a column's `DynamicCellClass` on ALTER stored `[if$x/Ythen'a'else'b']` as its expression", "cause": "`[ ... ]` parses as propertyValueV3's array alternative, so the value reaches the AST as a []string. The create writers read only a string (GetStringProp for DynamicClasses, `v.(string)` for columnClass); the mutator's dynamicclasses case returned nil when the type check failed; setColumnPropertyMut formatted the list with %v, after the visitor's GetText() had already fused its tokens", "file": "`mdl/executor/validate_widget_expression_list.go` (MDL-WIDGET32), `mdl/backend/pagemutator/mutator.go` (`errExpressionNotAString`)", "insight": "**A proposal's example syntax is worth running through `check` before designing around it** - here the proposed spelling already parsed, and the parse was the bug. Same class as #999 / MDL-WIDGET27 (empty `[]`): a value shape no writer claims. Fix at both ends and they stay in step: a no-project check rule for CREATE, and an error (not a nil return) from the ALTER setter, which `check -p` reports for free because validateAlterSetProperties dry-runs the setter. Key the rule on the property, never on the brackets - `visible: [cond]` is valid MDL. Measured with pre-fix and fixed binaries on copies of ako/TestApp: pre-fix `describe` shows the container without DynamicClasses and the quoted control intact. Tests `validate_widget_expression_list_test.go`, `pagemutator/expression_list_value_test.go`", "refs": ["mendixlabs/mxcli#750", "#999"], "rules": ["MDL-WIDGET32"]} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_Edit takes the page's 8 Enumerations$Condition entries to 0: every widget with Studio Pro's \"Visible: based on attribute value\" becomes ALWAYS visible; check, exec and mx check all report success", "cause": "MDL had no spelling for attribute-based conditional visibility. extractConditionalSettings read only Expression, conditionalVisibilityToGen wrote only Expression, and the settings' list markers were the default [3] where Studio Pro stores Conditions [2] and ModuleRoles [1]", "file": "`mdl/grammar/domains/MDLPage.g4` (`VISIBLE COLON attributePathV3 IN (…)`), `mdl/executor/cmd_pages_builder_visible_when.go` (`applyVisibleWhen`), `mdl/backend/modelsdk/widget_write.go` (`conditionalVisibilityToGen`, TypeDefaults), `mdl/executor/cmd_pages_describe_parse.go` / `_output.go` (`visibleWhenProp`)", "insight": "**A dropped visibility setting is invisible to every check**: the model stays valid, the widget just shows for everyone — count `Enumerations$Condition` in `bson dump --format ndsl` before and after a round trip, since mx check never will. Survey the corpus before designing syntax: all 12 settings in the stock project were attribute-based (booleans and one enum), none role-based or editability, which scoped the feature. Studio Pro stores EVERY value (enum values in declaration order plus \"(empty)\", or true/false) with a flag, so MDL lists only the shown values and the writer fills the rest from the domain model — and a byte-identical before/after diff of the settings block (markers included) on 3 pages is the proof. mdlIdent quotes `empty`/`true`/`false` as keywords; emit them bare in a value list. A new AST property key must be added to the known-property list (validate_widgets.go) or MDL-WIDGET07 falsely warns it is dropped", "refs": ["Administration.Account_Edit", "Administration.ScheduledEvents"], "rules": ["MDL-WIDGET07"], "date": "2026-09-25"} {"area":"mdl/executor","date":"2026-09-25","symptom":"describe → exec of an EXCLUDED page (Feedback v4.0.2 FeedbackModule.ShareFeedback_Logo, 11.13.0) refused: `page '…' has reference errors: - nanoflow not found: FeedbackModule.DS_FeedbackForm …`, though the untouched project passes mx check at 0 errors (Mendix does not validate excluded documents). With --no-check the page builder refused the same names again (`failed to resolve nanoflow`).","cause":"Two refusals, not one: validate.go's CreatePageStmtV3/CreateSnippetStmtV3 cases ignored exclusion (microflow/nanoflow/rule had been exempt since #312, silently), and pageBuilder.resolveMicroflow/resolveNanoflowByName/resolvePageRef/resolveSnippetRef fail on a missing name though the writer only ever stores the qualified NAME (IDs are never serialized).","file":"mdl/executor/validate.go (relaxExcludedWidgetRefs, carriedExclusion, warnExcluded), mdl/executor/cmd_pages_builder.go (tolerateDanglingRefs/danglingRefOK), cmd/mxcli/cmd_exec.go + cmd_check.go (ValidateProgramWithWarnings)","insight":"Relaxing the check is NOT safe for a DATA SOURCE, and only a real run shows it: the source flow's return type is the entity in scope, describe prints the nested bindings as bare names (`Attribute: Subject`, `ImageUrlParams: [{1} = ImageB64]`), and writing them without the entity left a bare `ImageB64` AttributeRef that made mx unable to LOAD the project (ArgumentNullException setting 'Attribute') — excluded page or not, where the pre-fix refusal had been protecting it by accident. So dangling action targets/snippet calls are warnings, dangling data sources and entities still block with the reason. 'Excluded' must mean what exec WRITES — @excluded OR the #914 carry (every stored namesake excluded) — or check and exec disagree. A/B on 11.13.0: identical page with 3 dangling action targets, excluded → 0 errors; live → 3x CE1613. Follow-up not fixed: describe loses attribute qualification inside a container whose flow is unresolvable, so ShareFeedback_Logo itself still cannot round-trip.","refs":["mdl-examples/bug-tests/excluded-page-dangling-references.mdl","#312","#914"]} +{"area": "mdl/executor", "symptom": "describe → exec of Feedback v4.0.2's EXCLUDED FeedbackModule.ShareFeedback_Logo refused `nanoflow not found: FeedbackModule.DS_FeedbackForm (data source)`; forcing it through left a project `mx check` could not LOAD (ArgumentNullException setting 'Attribute')", "cause": "The data view's flow is not in the project, so DESCRIBE had no context entity and printed every binding inside it bare (`Attribute: Subject`, `{1} = ImageB64`, `Visible: _showEmail in (…)`); exec had nothing to qualify them against, and a bare DomainModels$AttributeRef makes Mendix's loader throw. The stored model always had the full names", "file": "`mdl/executor/cmd_pages_describe_flowcontext.go` (`withQualifiedAttrs`, `describeAttr`), `mdl/executor/validate.go` (`unscopedBindings`), `mdl/executor/cmd_pages_builder_v3.go` (dangling DS flow kept by name), `mdl/backend/modelsdk/page_bare_attributeref.go` (`refuseBareAttributeRefs`)", "insight": "**The information was never lost — DESCRIBE threw it away**: every AttributeRef inside the unresolvable container still stored Module.Entity.Attr; shortening to the bare name is only safe where the reader can re-derive the entity, so key the shortening on whether the context resolved, not on habit. Measure the loader's tolerance before adding a write guard: 72 of 72 Studio Pro AttributeRefs in the project are qualified, so refusing a bare one refuses only writes that were already fatal — and turns a load-time stack trace into a statement-level error naming the attribute. Pair a blanket AST-level refusal with the exact failing slots (Attribute, CaptionAttribute, Visible-in, *Params) so the refusal names the widget, and keep the writer guard as the net for slots the walk does not know. Forced-fault run: hand-edit one qualified binding back to bare and confirm the refusal names it", "refs": ["ako/mxcli#675", "FeedbackModule.ShareFeedback_Logo"], "date": "2026-09-25"} diff --git a/.claude/skills/mendix/check-syntax/SKILL.md b/.claude/skills/mendix/check-syntax/SKILL.md index be4b1cee6f..973912c2e6 100644 --- a/.claude/skills/mendix/check-syntax/SKILL.md +++ b/.claude/skills/mendix/check-syntax/SKILL.md @@ -63,9 +63,13 @@ does not validate excluded documents (Feedback v4.0.2 ships an excluded page bou to nanoflows it lacks, and the project checks at 0 errors), so `check` and `exec` print them as `Reference warning` lines for excluded microflows, nanoflows, rules, and pages/snippets exec will write excluded (`@excluded`, or a stored namesake that -is). **A page's or snippet's missing data source (or entity) still blocks:** the -widgets inside bind against it, and written without it their bindings are bare -names — on 11.13.0 that left a project `mx` could not load. +is). **A missing data-source flow is a warning only when the bindings inside it are +qualified** (`Attribute: Module.Entity.Attr`, `{1} = Module.Entity.Attr`, +`Visible: Module.Entity.Attr in (…)`) — the form `describe` writes there. The +widgets inside bind against the entity that flow returns, so with the flow missing +a bare binding cannot be resolved; it is refused, naming the widget, because on +11.13.0 a bare attribute reference left a project `mx` could not load. A missing +entity still blocks. ### It also reports a name the PROJECT already has diff --git a/docs-site/src/tutorial/validation.md b/docs-site/src/tutorial/validation.md index bdb7e8a4ab..3aad848373 100644 --- a/docs-site/src/tutorial/validation.md +++ b/docs-site/src/tutorial/validation.md @@ -50,9 +50,11 @@ This is the check you should run before executing a script. It's fast (reads the References inside an **excluded** document (`@excluded`, or a page or snippet that stays excluded because its stored namesake is) are reported as `Reference warnings` rather than errors, because Mendix does not validate -excluded documents. A page's or snippet's missing *data source* still fails the -check: the widgets inside it bind against that source's entity, and cannot be -written without it. +excluded documents. A missing *data-source flow* is a warning too, but only when +every binding inside that container is qualified (`Module.Entity.Attribute`) — +`describe page` writes them that way there. The widgets bind against the entity +the flow returns, so with the flow missing a bare binding cannot be resolved and +the check fails, naming the widget. A missing entity still fails the check. ### Name conflicts with the project diff --git a/mdl-examples/bug-tests/excluded-page-unresolved-flow-qualified-bindings.mdl b/mdl-examples/bug-tests/excluded-page-unresolved-flow-qualified-bindings.mdl new file mode 100644 index 0000000000..37b8d32ae1 --- /dev/null +++ b/mdl-examples/bug-tests/excluded-page-unresolved-flow-qualified-bindings.mdl @@ -0,0 +1,46 @@ +-- ============================================================================ +-- An excluded page over a flow the project lacks: bindings kept qualified +-- ============================================================================ +-- +-- Symptom: describe → exec of FeedbackModule.ShareFeedback_Logo (Feedback +-- v4.0.2, Mendix 11.13.0), an EXCLUDED example page whose data view is sourced +-- by a nanoflow the module does not ship, was refused: +-- nanoflow not found: FeedbackModule.DS_FeedbackForm (data source) +-- and writing it anyway left a project `mx check` could not LOAD +-- (ArgumentNullException setting 'Attribute'), because DESCRIBE printed the +-- bindings inside that data view bare (`Attribute: Subject`, `{1} = ImageB64`) +-- and with the flow missing there is no entity to qualify them against. +-- +-- Fix: DESCRIBE keeps the stored Module.Entity.Attr inside a data container +-- whose flow cannot be resolved (and `Visible: Mod.Entity.Attr in (…)` accepts +-- it); the missing flow is then a warning on an excluded page. A bare binding +-- there is still refused, naming the widget, and the page writer refuses any +-- bare attribute reference outright. +-- +-- Verify: exec on a project WITHOUT MyFirstModule.DS_MissingForm → warning +-- only, "Created page"; `mxcli docker check` — 0 errors (the page stays +-- excluded). Replace one qualified binding with a bare name → refused. +-- ============================================================================ + +create entity MyFirstModule.FeedbackDraft ( + Subject: String(200), + ShowEmail: Boolean default false, + Email: String(200) +); +/ + +@excluded +create or modify page MyFirstModule.FeedbackDraft_Example +( Title: 'Feedback (example)', Layout: Atlas_Core.Atlas_Default ) +{ + dataview dv (DataSource: nanoflow MyFirstModule.DS_MissingForm) { + textbox txtSubject (Label: 'Subject', Attribute: MyFirstModule.FeedbackDraft.Subject) + dynamictext txtEcho (Content: 'About: {1}', ContentParams: [{1} = MyFirstModule.FeedbackDraft.Subject]) + textbox txtEmail ( + Label: 'Email', + Attribute: MyFirstModule.FeedbackDraft.Email, + Visible: MyFirstModule.FeedbackDraft.ShowEmail in (true) + ) + } +} +/ diff --git a/mdl/backend/modelsdk/page_bare_attributeref.go b/mdl/backend/modelsdk/page_bare_attributeref.go new file mode 100644 index 0000000000..ead2078258 --- /dev/null +++ b/mdl/backend/modelsdk/page_bare_attributeref.go @@ -0,0 +1,59 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "fmt" + "strings" + + "go.mongodb.org/mongo-driver/bson" +) + +// refuseBareAttributeRefs refuses a page or snippet whose encoded form holds a +// DomainModels$AttributeRef that is not Module.Entity.Attribute. +// +// Mendix rebuilds each stored reference into a typed identifier as it loads, +// and an attribute that does not parse as one takes the loader down before any +// validation runs: a bare `ImageB64` image parameter left `mx check` unable to +// load the project (ArgumentNullException setting 'Attribute', 11.13.0) — the +// page was excluded, which does not help, as loading is not validating. Studio +// Pro qualifies every one (72 of 72 across a stock project's pages, snippets +// and layouts). A bare name reaches here when nothing could qualify it — inside +// a data container whose flow the project lacks — so this is the last line +// under the check that refuses it first (checkUnscopedBindings). +func refuseBareAttributeRefs(contents []byte) error { + var bad []string + var walk func(v bson.RawValue, path string) + walk = func(v bson.RawValue, path string) { + switch v.Type { + case bson.TypeEmbeddedDocument: + doc := v.Document() + if t, ok := doc.Lookup("$Type").StringValueOK(); ok && t == "DomainModels$AttributeRef" { + if a, ok := doc.Lookup("Attribute").StringValueOK(); ok && a != "" && strings.Count(a, ".") < 2 { + bad = append(bad, fmt.Sprintf("%q at %s", a, path)) + } + } + name, _ := doc.Lookup("Name").StringValueOK() + elems, _ := doc.Elements() + for _, e := range elems { + p := path + "/" + e.Key() + if name != "" { + p = path + "/" + name + "." + e.Key() + } + walk(e.Value(), p) + } + case bson.TypeArray: + vals, _ := v.Array().Values() + for _, x := range vals { + walk(x, path) + } + } + } + walk(bson.RawValue{Type: bson.TypeEmbeddedDocument, Value: contents}, "") + if len(bad) == 0 { + return nil + } + return fmt.Errorf("attribute reference not qualified as Module.Entity.Attribute — Mendix cannot load a "+ + "project holding one, so it is not written: %s. Qualify it in the script; inside a data container "+ + "whose flow the project lacks there is no entity to resolve a bare name against", strings.Join(bad, "; ")) +} diff --git a/mdl/backend/modelsdk/page_bare_attributeref_test.go b/mdl/backend/modelsdk/page_bare_attributeref_test.go new file mode 100644 index 0000000000..83c2c2f001 --- /dev/null +++ b/mdl/backend/modelsdk/page_bare_attributeref_test.go @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "strings" + "testing" + + "go.mongodb.org/mongo-driver/bson" +) + +// A DomainModels$AttributeRef whose Attribute is not Module.Entity.Attr makes +// the project unloadable: an image URL parameter written as a bare `ImageB64` +// (Feedback v4.0.2's ShareFeedback_Logo, under a data view whose flow the +// project lacks) left `mx check` unable to LOAD the project — +// ArgumentNullException setting 'Attribute', Mendix 11.13.0 — while the page +// was excluded. Studio Pro qualifies every one: 72 of 72 AttributeRefs across +// that project's pages, snippets and layouts. So the writer refuses the bare +// form, naming it, instead of storing it. +func TestRefuseBareAttributeRefs(t *testing.T) { + attrRef := func(a string) bson.D { + return bson.D{{Key: "$Type", Value: "DomainModels$AttributeRef"}, {Key: "Attribute", Value: a}, {Key: "EntityRef", Value: nil}} + } + doc := func(a string) []byte { + b, err := bson.Marshal(bson.D{ + {Key: "$Type", Value: "Forms$Page"}, + {Key: "Widgets", Value: bson.A{int32(2), bson.D{ + {Key: "$Type", Value: "CustomWidgets$CustomWidget"}, + {Key: "Name", Value: "image1"}, + {Key: "Params", Value: bson.A{int32(2), bson.D{{Key: "AttributeRef", Value: attrRef(a)}}}}, + }}}, + }) + if err != nil { + t.Fatal(err) + } + return b + } + err := refuseBareAttributeRefs(doc("ImageB64")) + if err == nil || !strings.Contains(err.Error(), "ImageB64") { + t.Fatalf("a bare attribute reference must be refused, naming it; got %v", err) + } + for _, ok := range []string{"FeedbackModule.Feedback.ImageB64", ""} { + if err := refuseBareAttributeRefs(doc(ok)); err != nil { + t.Errorf("%q must be accepted: %v", ok, err) + } + } +} diff --git a/mdl/backend/modelsdk/page_write.go b/mdl/backend/modelsdk/page_write.go index 94e80bdd89..6c15d5b803 100644 --- a/mdl/backend/modelsdk/page_write.go +++ b/mdl/backend/modelsdk/page_write.go @@ -256,7 +256,14 @@ func encodePage(page *pages.Page, pv *types.ProjectVersion, carry func(*genPg.Pa if carry != nil { carry(g) } - return docEncoder("Forms$Page", pv).Encode(g) + contents, err := docEncoder("Forms$Page", pv).Encode(g) + if err != nil { + return nil, err + } + if err := refuseBareAttributeRefs(contents); err != nil { + return nil, fmt.Errorf("page %q: %w", page.Name, err) + } + return contents, nil } // pageToGen builds the full gen Page: header, layout call, the widget tree (under diff --git a/mdl/backend/modelsdk/snippet_write.go b/mdl/backend/modelsdk/snippet_write.go index 907315610c..2921ad0025 100644 --- a/mdl/backend/modelsdk/snippet_write.go +++ b/mdl/backend/modelsdk/snippet_write.go @@ -33,7 +33,14 @@ func encodeSnippet(snippet *pages.Snippet, pv *types.ProjectVersion) ([]byte, er return nil, err } g.SetID(element.ID(snippet.ID)) - return docEncoder("Forms$Snippet", pv).Encode(g) + contents, err := docEncoder("Forms$Snippet", pv).Encode(g) + if err != nil { + return nil, err + } + if err := refuseBareAttributeRefs(contents); err != nil { // see page_bare_attributeref.go + return nil, fmt.Errorf("snippet %q: %w", snippet.Name, err) + } + return contents, nil } // CreateSnippet inserts a new Forms$Snippet document — a reusable widget tree with diff --git a/mdl/executor/cmd_pages_builder.go b/mdl/executor/cmd_pages_builder.go index d468f89b16..77e70dc33a 100644 --- a/mdl/executor/cmd_pages_builder.go +++ b/mdl/executor/cmd_pages_builder.go @@ -81,12 +81,13 @@ type pageBuilder struct { // page. The writer stores an ACTION's or snippet call's target BY NAME, so // an unresolved one is kept as written instead of failing the build. // - // A DATA SOURCE is deliberately not tolerated: its flow's return type is - // what puts an entity in scope, and without it every attribute binding - // inside the container is written unqualified. Measured on Mendix 11.13.0: - // a bare `ImageB64` in an image's URL parameter made `mx check` fail to - // LOAD the project (ArgumentNullException setting 'Attribute'), even - // though the page was excluded. See validateExcludedWidgetRefs. + // A data-source FLOW is kept by name too, but it is what puts an entity in + // scope: without it a bare attribute binding inside the container cannot + // be qualified, and one written bare made `mx check` fail to LOAD the + // project (ArgumentNullException setting 'Attribute', Mendix 11.13.0). + // DESCRIBE writes those bindings qualified there, the check refuses a bare + // one (checkUnscopedBindings), and the page writer refuses any bare + // attribute reference as a last line (refuseBareAttributeRefs). tolerateDanglingRefs bool // Local page/snippet variables (Variables: { $name: Type = 'default' }). diff --git a/mdl/executor/cmd_pages_builder_v3.go b/mdl/executor/cmd_pages_builder_v3.go index 12b57cda58..2c7f7c86f3 100644 --- a/mdl/executor/cmd_pages_builder_v3.go +++ b/mdl/executor/cmd_pages_builder_v3.go @@ -916,7 +916,11 @@ func (pb *pageBuilder) buildDataSourceV3(ds *ast.DataSourceV3) (pages.DataSource case "microflow": // Microflow source mfID, err := pb.resolveMicroflow(ds.Reference) - if err != nil { + // An excluded page may name a flow the project lacks (see + // tolerateDanglingRefs). It is written by name with NO entity in scope, + // so the bindings inside must be qualified; checkUnscopedBindings + // refuses a bare one before anything is written. + if err != nil && !pb.danglingRefOK(err) { return nil, "", mdlerrors.NewBackend("resolve microflow", err) } @@ -936,7 +940,7 @@ func (pb *pageBuilder) buildDataSourceV3(ds *ast.DataSourceV3) (pages.DataSource case "nanoflow": // Nanoflow source - resolve by listing all nanoflows nfID, err := pb.resolveNanoflowByName(ds.Reference) - if err != nil { + if err != nil && !pb.danglingRefOK(err) { // kept by name: see the microflow case return nil, "", mdlerrors.NewBackend("resolve nanoflow", err) } diff --git a/mdl/executor/cmd_pages_builder_visible_when.go b/mdl/executor/cmd_pages_builder_visible_when.go index 1cefe7fe0a..cf6060b5c9 100644 --- a/mdl/executor/cmd_pages_builder_visible_when.go +++ b/mdl/executor/cmd_pages_builder_visible_when.go @@ -44,18 +44,28 @@ func (pb *pageBuilder) applyVisibleWhen(widget pages.Widget, w *ast.WidgetV3) er return mdlerrors.NewValidationf("%s %s: `Visible: %s in (…)` is not supported on this widget", w.Type, w.Name, vw.Attribute) } where := fmt.Sprintf("%s %s: Visible: %s in (…)", w.Type, w.Name, vw.Attribute) - if pb.entityContext == "" { - return mdlerrors.NewValidationf("%s: the attribute is read from the enclosing data container's object — place the widget inside a data container", where) + if strings.Contains(vw.Attribute, "/") { + return mdlerrors.NewValidationf("%s: association paths are not supported — name an attribute of the data container's own entity", where) } - if strings.ContainsAny(vw.Attribute, "/.") { - return mdlerrors.NewValidationf("%s: name an attribute of the data container's own entity (%s); association paths are not supported", where, pb.entityContext) + + // The entity is the data container's, or — qualified, Module.Entity.Attr — + // named outright. DESCRIBE writes the qualified form under a container whose + // flow cannot be resolved, where there is no entity in scope at all. + entity, attrName := pb.entityContext, vw.Attribute + if parts := strings.Split(vw.Attribute, "."); len(parts) == 3 { + entity, attrName = parts[0]+"."+parts[1], parts[2] + } else if len(parts) != 1 { + return mdlerrors.NewValidationf("%s: name the attribute bare, or as Module.Entity.Attribute", where) + } + if entity == "" { + return mdlerrors.NewValidationf("%s: the attribute is read from the enclosing data container's object — place the widget inside a data container, or qualify it (Module.Entity.%s)", where, attrName) } - declaring, ok := pb.declaringEntityFor(pb.entityContext, vw.Attribute) + declaring, ok := pb.declaringEntityFor(entity, attrName) if !ok { - return mdlerrors.NewValidationf("%s: %s has no attribute %s", where, pb.entityContext, vw.Attribute) + return mdlerrors.NewValidationf("%s: %s has no attribute %s", where, entity, attrName) } - attrQN := declaring + "." + vw.Attribute + attrQN := declaring + "." + attrName var all []string switch t := pb.findAttributeType(attrQN).(type) { diff --git a/mdl/executor/cmd_pages_describe_flowcontext.go b/mdl/executor/cmd_pages_describe_flowcontext.go index 73547933de..4b71321b07 100644 --- a/mdl/executor/cmd_pages_describe_flowcontext.go +++ b/mdl/executor/cmd_pages_describe_flowcontext.go @@ -3,6 +3,8 @@ package executor import ( + "strings" + "github.com/mendixlabs/mxcli/sdk/microflows" ) @@ -30,6 +32,41 @@ func dataSourceEntityContext(ctx *ExecContext, ds *rawDataSource) string { return ds.Reference } +// flowContextUnresolved reports whether a data container's source is a flow +// whose returned entity cannot be determined — the flow is not in the project +// (Feedback v4.0.2's excluded ShareFeedback_Logo names a nanoflow the module +// does not ship), or it returns no object. +func flowContextUnresolved(ctx *ExecContext, ds *rawDataSource) bool { + if ds == nil || ds.Reference == "" || (ds.Type != "microflow" && ds.Type != "nanoflow") { + return false + } + return flowReturnEntity(ctx, ds.Type, ds.Reference) == "" +} + +// withQualifiedAttrs runs parse with attribute bindings kept qualified when +// ds leaves no entity in scope, restoring the previous setting afterwards. +func withQualifiedAttrs[T any](ctx *ExecContext, ds *rawDataSource, parse func() T) T { + if ctx == nil || !flowContextUnresolved(ctx, ds) { + return parse() + } + prev := ctx.describeQualifyAttrs + ctx.describeQualifyAttrs = true + defer func() { ctx.describeQualifyAttrs = prev }() + return parse() +} + +// describeAttr renders a stored attribute name for MDL: bare, as exec resolves +// it against the data container's entity — or, where there is no entity to +// resolve against (describeQualifyAttrs), the stored Module.Entity.Attr. A bare +// name there cannot be qualified on the way back: written anyway, a bare image +// parameter left a project `mx check` could not load. +func describeAttr(ctx *ExecContext, qn string) string { + if ctx != nil && ctx.describeQualifyAttrs && strings.Count(qn, ".") >= 2 { + return qn + } + return shortAttributeName(qn) +} + // flowReturnEntity resolves the entity a microflow or nanoflow returns, by object // or list return type. Returns "" when the project is unavailable, the flow is not // found, or it returns something other than an object/list. diff --git a/mdl/executor/cmd_pages_describe_objectlist.go b/mdl/executor/cmd_pages_describe_objectlist.go index 66940caced..5508d38144 100644 --- a/mdl/executor/cmd_pages_describe_objectlist.go +++ b/mdl/executor/cmd_pages_describe_objectlist.go @@ -212,7 +212,7 @@ func extractObjectListItem(ctx *ExecContext, itemObj map[string]any, nestedMap m // Attribute binding (staticXAttribute, staticYAttribute, …). if attrRef, ok := value["AttributeRef"].(map[string]any); ok && len(attrRef) > 0 { if a := extractString(attrRef["Attribute"]); a != "" { - item.Props = append(item.Props, rawExplicitProp{Key: objectListMDLKey(key), Value: shortAttributeName(a), IsRef: true}) + item.Props = append(item.Props, rawExplicitProp{Key: objectListMDLKey(key), Value: describeAttr(ctx, a), IsRef: true}) } continue } diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index 676ecd0958..e0225d8686 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -1860,7 +1860,7 @@ func extractClientTemplateParameters(ctx *ExecContext, w map[string]any, fieldNa result = append(result, "$"+sourceVarName+"."+attrName) } else { // No SourceVariable - use short attribute name - result = append(result, shortAttributeName(attr)) + result = append(result, describeAttr(ctx, attr)) } continue } @@ -2077,5 +2077,11 @@ func visibleWhenProp(w rawWidget) string { } vals[i] = mdlIdent(v) } - return fmt.Sprintf("Visible: %s in (%s)", mdlIdent(w.VisibleAttr), strings.Join(vals, ", ")) + attr := mdlIdent(w.VisibleAttr) + if strings.Contains(w.VisibleAttr, ".") { + // Qualified (Module.Entity.Attr) where no entity is in scope — written + // as a qualified name, which takes keyword segments bare. + attr = w.VisibleAttr + } + return fmt.Sprintf("Visible: %s in (%s)", attr, strings.Join(vals, ", ")) } diff --git a/mdl/executor/cmd_pages_describe_parse.go b/mdl/executor/cmd_pages_describe_parse.go index 5ddf678c8a..f2b78c1215 100644 --- a/mdl/executor/cmd_pages_describe_parse.go +++ b/mdl/executor/cmd_pages_describe_parse.go @@ -27,7 +27,7 @@ func asActionMap(v any) map[string]any { // parseRawWidget parses a raw widget map into rawWidget structs. // extractConditionalSettings extracts ConditionalVisibility/Editability from raw BSON. -func extractConditionalSettings(widget *rawWidget, w map[string]any) { +func extractConditionalSettings(ctx *ExecContext, widget *rawWidget, w map[string]any) { if cvs, ok := w["ConditionalVisibilitySettings"].(map[string]any); ok && cvs != nil { if expr, ok := cvs["Expression"].(string); ok && expr != "" { widget.VisibleIf = expr @@ -36,7 +36,7 @@ func extractConditionalSettings(widget *rawWidget, w map[string]any) { // setting was never read, so describe → exec wrote the widget always // visible (Administration.Account_Edit: 8 conditions → 0). if attr, ok := cvs["Attribute"].(string); ok && attr != "" { - widget.VisibleAttr = shortAttributeName(attr) + widget.VisibleAttr = describeAttr(ctx, attr) for _, c := range getBsonArrayElements(cvs["Conditions"]) { cm, ok := c.(map[string]any) if !ok { @@ -117,7 +117,7 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s } widget.DesignProperties = extractDesignProperties(appearance) } - extractConditionalSettings(&widget, w) + extractConditionalSettings(ctx, &widget, w) // Regions are five named slots, not a list: Top, Right, Bottom, Left // and CenterRegion (the last spelled differently from its siblings). // Each occupied one becomes a synthetic intermediate widget, the same @@ -182,7 +182,7 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s } widget.DesignProperties = extractDesignProperties(appearance) } - extractConditionalSettings(&widget, w) + extractConditionalSettings(ctx, &widget, w) for _, tp := range getBsonArrayElements(w["TabPages"]) { tpMap, ok := tp.(map[string]any) if !ok { @@ -253,7 +253,7 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s widget.HeaderMode = headerMode } } - extractConditionalSettings(&widget, w) + extractConditionalSettings(ctx, &widget, w) children := getBsonArrayElements(w["Widgets"]) if children != nil { for _, c := range children { @@ -269,7 +269,7 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s Type: typeName, Name: name, } - extractConditionalSettings(&widget, w) + extractConditionalSettings(ctx, &widget, w) // Extract CSS class, style, and design properties from Appearance if appearance, ok := w["Appearance"].(map[string]any); ok { @@ -351,7 +351,9 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s widget.LabelWidth = extractDataViewLabelWidth(w) widget.ReadOnlyStyle = extractReadOnlyStyle(ctx, w) widget.ShowFooter, _ = w["ShowFooter"].(bool) - widget.Children = parseDataViewChildren(ctx, w, widget.EntityContext) + widget.Children = withQualifiedAttrs(ctx, widget.DataSource, func() []rawWidget { + return parseDataViewChildren(ctx, w, widget.EntityContext) + }) return []rawWidget{widget} case "Forms$TextBox", "Pages$TextBox": @@ -469,7 +471,9 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s } else if inheritedCtx != "" { widget.EntityContext = inheritedCtx } - widget.Children = extractGalleryContent(ctx, w, widget.EntityContext) + widget.Children = withQualifiedAttrs(ctx, widget.DataSource, func() []rawWidget { + return extractGalleryContent(ctx, w, widget.EntityContext) + }) widget.FilterWidgets = extractGalleryFilters(ctx, w) } // For filter widgets, extract filter attributes and expression @@ -696,7 +700,9 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s if onClick := asActionMap(w["ClickAction"]); onClick != nil { widget.Action = extractButtonAction(ctx, map[string]any{"Action": onClick}) } - widget.Children = parseListViewContent(ctx, w, widget.EntityContext) + widget.Children = withQualifiedAttrs(ctx, widget.DataSource, func() []rawWidget { + return parseListViewContent(ctx, w, widget.EntityContext) + }) return []rawWidget{widget} default: @@ -1027,7 +1033,12 @@ func extractAttributeRef(ctx *ExecContext, w map[string]any) string { if _, ok := attrRef["Attribute"].(string); !ok { return "" } - return columnAttributeFromRef(attrRef) + path := columnAttributeFromRef(attrRef) + if !strings.Contains(path, "/") { + // A plain binding: qualified where no entity is in scope (describeAttr). + return describeAttr(ctx, extractString(attrRef["Attribute"])) + } + return path } // parseGalleryContent extracts the content widget from a Gallery. diff --git a/mdl/executor/cmd_pages_describe_pluggable.go b/mdl/executor/cmd_pages_describe_pluggable.go index ecbded2054..24258e5cb8 100644 --- a/mdl/executor/cmd_pages_describe_pluggable.go +++ b/mdl/executor/cmd_pages_describe_pluggable.go @@ -634,7 +634,7 @@ func extractTextTemplateParameters(ctx *ExecContext, textTemplate map[string]any result = append(result, "$"+sourceVarName+"."+attrName) } else { // No SourceVariable - use short attribute name - result = append(result, shortAttributeName(attr)) + result = append(result, describeAttr(ctx, attr)) } continue } @@ -893,7 +893,7 @@ func extractCustomWidgetPropertyAttributeRef(ctx *ExecContext, w map[string]any, } if attrRef, ok := value["AttributeRef"].(map[string]any); ok && attrRef != nil { if attr, ok := attrRef["Attribute"].(string); ok && attr != "" { - return shortAttributeName(attr) + return describeAttr(ctx, attr) } } } @@ -1098,7 +1098,7 @@ func extractCustomWidgetPropertyAttributes(ctx *ExecContext, w map[string]any, p // Check for AttributeRef if attrRef, ok := objValue["AttributeRef"].(map[string]any); ok && attrRef != nil { if attr, ok := attrRef["Attribute"].(string); ok && attr != "" { - result = append(result, shortAttributeName(attr)) + result = append(result, describeAttr(ctx, attr)) } } } @@ -1167,7 +1167,7 @@ func extractExplicitProperties(ctx *ExecContext, w map[string]any) []rawExplicit if attr := extractString(attrRef["Attribute"]); attr != "" { result = append(result, rawExplicitProp{ Key: propKey, - Value: shortAttributeName(attr), + Value: describeAttr(ctx, attr), IsRef: true, }) continue diff --git a/mdl/executor/cmd_pages_describe_unresolved_context_test.go b/mdl/executor/cmd_pages_describe_unresolved_context_test.go new file mode 100644 index 0000000000..72f2fbede4 --- /dev/null +++ b/mdl/executor/cmd_pages_describe_unresolved_context_test.go @@ -0,0 +1,94 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "context" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// A data view over a flow the project does not contain — Feedback v4.0.2's +// excluded ShareFeedback_Logo, over FeedbackModule.DS_FeedbackForm — has no +// entity in scope for DESCRIBE or for exec. Every binding inside it still +// carries its full name in storage (`FeedbackModule.Feedback.Subject`), but +// DESCRIBE printed the bare attribute, which exec cannot qualify: written +// anyway, a bare `ImageB64` image parameter left a project `mx check` could not +// LOAD. So inside such a container the qualified name is kept. +func unresolvedContextDataView(ds map[string]any) map[string]any { + attrRef := func(qn string) map[string]any { + return map[string]any{"$Type": "DomainModels$AttributeRef", "Attribute": qn, "EntityRef": nil} + } + return map[string]any{ + "$Type": "Forms$DataView", + "Name": "dataView5", + "DataSource": ds, + "Widgets": []any{int32(2), + map[string]any{"$Type": "Forms$TextBox", "Name": "feedback_subject", + "AttributeRef": attrRef("FeedbackModule.Feedback.Subject")}, + map[string]any{"$Type": "Forms$DynamicText", "Name": "text1", + "Content": map[string]any{"$Type": "Forms$ClientTemplate", + "Template": map[string]any{"$Type": "Texts$Text", "Items": []any{int32(3), + map[string]any{"$Type": "Texts$Translation", "LanguageCode": "en_US", "Text": "Image: {1}"}}}, + "Parameters": []any{int32(2), + map[string]any{"$Type": "Forms$ClientTemplateParameter", "AttributeRef": attrRef("FeedbackModule.Feedback.ImageB64")}}, + }}, + map[string]any{"$Type": "Forms$TextBox", "Name": "textBox1", + "AttributeRef": attrRef("FeedbackModule.Feedback.SubmitterEmail"), + "ConditionalVisibilitySettings": map[string]any{ + "$Type": "Forms$ConditionalVisibilitySettings", + "Attribute": "FeedbackModule.Feedback._showEmail", + "Conditions": []any{int32(2), + map[string]any{"$Type": "Enumerations$Condition", "AttributeValue": "true", "EditableVisible": true}, + map[string]any{"$Type": "Enumerations$Condition", "AttributeValue": "false", "EditableVisible": false}}, + }}, + }, + } +} + +func describeWidget(t *testing.T, w map[string]any) string { + t.Helper() + var buf bytes.Buffer + ctx := (&Executor{}).newExecContext(context.Background()) + ctx.Output = &buf + for _, rw := range parseRawWidget(ctx, w) { + outputWidgetMDLV3(ctx, rw, 1) + } + return buf.String() +} + +func TestDescribe_UnresolvedFlowContext_KeepsQualifiedBindings(t *testing.T) { + got := describeWidget(t, unresolvedContextDataView(map[string]any{ + "$Type": "Forms$NanoflowSource", "Nanoflow": "FeedbackModule.DS_FeedbackForm", + })) + for _, want := range []string{ + "Attribute: FeedbackModule.Feedback.Subject", + "{1} = FeedbackModule.Feedback.ImageB64", + "Attribute: FeedbackModule.Feedback.SubmitterEmail", + "Visible: FeedbackModule.Feedback._showEmail in (true)", + } { + if !strings.Contains(got, want) { + t.Errorf("describe output lacks %q — a bare binding under an unresolvable flow cannot be qualified on exec:\n%s", want, got) + } + } + if _, errs := visitor.Build("create page M.P (Title: 'x', Layout: A.L) {\n" + got + "}\n"); len(errs) > 0 { + t.Fatalf("describe output does not parse: %v\n%s", errs, got) + } +} + +// Where the entity IS known the short form is unchanged. +func TestDescribe_ResolvedContext_KeepsShortBindings(t *testing.T) { + got := describeWidget(t, unresolvedContextDataView(map[string]any{ + "$Type": "Forms$DataViewSource", + "EntityRef": map[string]any{"$Type": "DomainModels$DirectEntityRef", "Entity": "FeedbackModule.Feedback"}, + "SourceVariable": map[string]any{"$Type": "Forms$PageVariable", "PageParameter": "Feedback"}, + })) + for _, want := range []string{"Attribute: Subject", "{1} = ImageB64", "Visible: _showEmail in (true)"} { + if !strings.Contains(got, want) { + t.Errorf("describe output lacks %q:\n%s", want, got) + } + } +} diff --git a/mdl/executor/cmd_pages_visible_when_test.go b/mdl/executor/cmd_pages_visible_when_test.go index dd2a4e1fde..37cc6e3e69 100644 --- a/mdl/executor/cmd_pages_visible_when_test.go +++ b/mdl/executor/cmd_pages_visible_when_test.go @@ -150,3 +150,28 @@ func TestDescribe_VisibleWhen(t *testing.T) { t.Errorf("describe output does not parse: %v\n%s", errs, got) } } + +// Under a data container whose flow cannot be resolved there is no entity in +// scope, and DESCRIBE emits the attribute qualified: `Visible: +// Mod.Entity.Attr in (…)` must build from the name alone. +func TestVisibleWhen_QualifiedWithoutContext(t *testing.T) { + cvs, err := buildVisibleWhen(t, visibleWhenPB(""), "M.Job.IsLocal", "true") + if err != nil { + t.Fatalf("build: %v", err) + } + if cvs == nil || cvs.Attribute != "M.Job.IsLocal" || len(cvs.Conditions) != 2 { + t.Fatalf("got %+v", cvs) + } + // An inherited attribute named on the specialization is stored against + // the DECLARING entity, as the bare form is. + cvs, err = buildVisibleWhen(t, visibleWhenPB(""), "M.SpecialJob.Status", "Running") + if err != nil { + t.Fatalf("build: %v", err) + } + if cvs.Attribute != "M.Job.Status" { + t.Errorf("Attribute = %q, want M.Job.Status", cvs.Attribute) + } + if _, err := buildVisibleWhen(t, visibleWhenPB(""), "M.Job.Nope", "true"); err == nil { + t.Error("an unknown qualified attribute must be refused") + } +} diff --git a/mdl/executor/exec_context.go b/mdl/executor/exec_context.go index b225f2236a..9508a966c2 100644 --- a/mdl/executor/exec_context.go +++ b/mdl/executor/exec_context.go @@ -34,6 +34,12 @@ type ExecContext struct { // Output is the writer for user-visible output (with line-limit guard). Output io.Writer + // describeQualifyAttrs is set by DESCRIBE PAGE while it reads the widgets + // inside a data container whose flow cannot be resolved: no entity is in + // scope there, so an attribute binding keeps its stored Module.Entity.Attr + // name rather than the bare one exec could not qualify. See describeAttr. + describeQualifyAttrs bool + // Format controls output formatting (table, json, etc.). Format OutputFormat diff --git a/mdl/executor/validate.go b/mdl/executor/validate.go index abf45b7515..f1ebeee7dc 100644 --- a/mdl/executor/validate.go +++ b/mdl/executor/validate.go @@ -898,7 +898,14 @@ func (sc *scriptContext) relaxExcludedWidgetRefs(kind, name string, widgets []*a ref := e[strings.LastIndex(e, ": ")+2:] switch { case refs.dataSources[ref]: - blocking = append(blocking, e+" (data source)") + // The flow is kept by name, but it is what puts an entity in scope: + // only a container whose bindings are all qualified can be written. + if bare := unscopedBindings(widgets, ref); len(bare) > 0 { + blocking = append(blocking, e+" (data source) — and these bindings inside it are not qualified, "+ + "so nothing can resolve them: "+strings.Join(bare, ", ")) + continue + } + warnings = append(warnings, e+" (data source; the bindings inside it are qualified)") case strings.HasPrefix(e, "entity not found"): // The builder resolves an entity to write it (create_object and the // like), so exec would refuse it anyway; say so here instead of @@ -915,11 +922,94 @@ func (sc *scriptContext) relaxExcludedWidgetRefs(kind, name string, widgets []*a return nil } return mdlerrors.NewValidationf("%s '%s' is excluded, but a data source or entity it names does not exist:\n - %s\n"+ - " An excluded document may keep a dangling action target, but not a dangling data source: the\n"+ - " flow or entity decides what the widgets inside it bind to, and without it those bindings are\n"+ - " written unqualified — which leaves a project Mendix cannot load. Create the missing document,\n"+ - " or leave this %s as it is stored.", - kind, name, strings.Join(blocking, "\n - "), kind) + " An excluded document may keep a dangling flow, but the widgets inside a data container bind\n"+ + " against the entity its flow returns — with the flow missing, a bare binding cannot be\n"+ + " qualified, and one written bare leaves a project Mendix cannot load. Qualify those bindings\n"+ + " (Module.Entity.Attribute, as `describe` writes them there), or create the missing document.", + kind, name, strings.Join(blocking, "\n - ")) +} + +// unscopedBindings names the attribute bindings that cannot be resolved inside +// the data container(s) whose data source is the missing flow ref: bare +// attributes, `$currentObject/…` paths and association hops, all of which +// resolve against the entity that flow would have returned. Descent stops at a +// nested container with a data source of its own, which scopes its children. +// +// Covered: `Attribute:`, `CaptionAttribute:`, `Visible: Attr in (…)` and +// template parameters (`…Params: [{1} = Attr]`). Anything else is caught by +// the page writer's refusal of a bare attribute reference. +func unscopedBindings(widgets []*ast.WidgetV3, ref string) []string { + var out []string + var inScope func(ws []*ast.WidgetV3) + inScope = func(ws []*ast.WidgetV3) { + for _, w := range ws { + if w == nil { + continue + } + if _, own := w.Properties["DataSource"].(*ast.DataSourceV3); own { + continue // its own data source decides its children's scope + } + for _, b := range bareBindingsOf(w) { + out = append(out, fmt.Sprintf("%s `%s` (%s)", strings.ToLower(w.Type), w.Name, b)) + } + inScope(w.Children) + } + } + var find func(ws []*ast.WidgetV3) + find = func(ws []*ast.WidgetV3) { + for _, w := range ws { + if w == nil { + continue + } + if ds, ok := w.Properties["DataSource"].(*ast.DataSourceV3); ok && ds.Reference == ref && + (ds.Type == "microflow" || ds.Type == "nanoflow") { + for _, b := range bareBindingsOf(w) { // the container's own bindings, e.g. its visibility + out = append(out, fmt.Sprintf("%s `%s` (%s)", strings.ToLower(w.Type), w.Name, b)) + } + inScope(w.Children) + continue + } + find(w.Children) + } + } + find(widgets) + return out +} + +// bareBindingsOf lists a widget's attribute bindings that need an entity in +// scope to resolve. +func bareBindingsOf(w *ast.WidgetV3) []string { + var out []string + needsScope := func(v string) bool { + switch { + case v == "", strings.HasPrefix(v, "'"): + return false // unset, or a literal + case strings.HasPrefix(v, "$"): + return strings.HasPrefix(strings.ToLower(v), "$currentobject/") + } + return strings.Contains(v, "/") || strings.Count(v, ".") < 2 + } + for _, key := range []string{"Attribute", "CaptionAttribute"} { + if v, ok := w.Properties[key].(string); ok && needsScope(v) { + out = append(out, key+": "+v) + } + } + if vw, ok := w.Properties["VisibleWhen"].(*ast.VisibleWhenV3); ok && needsScope(vw.Attribute) { + out = append(out, "Visible: "+vw.Attribute+" in (…)") + } + for key, v := range w.Properties { + params, ok := v.([]ast.ParamAssignmentV3) + if !ok { + continue + } + for _, p := range params { + if s, ok := p.Value.(string); ok && needsScope(s) { + out = append(out, fmt.Sprintf("%s {%d} = %s", key, p.Index, s)) + } + } + } + sort.Strings(out) + return out } // carriedExclusion reports whether exec will write the page or snippet named diff --git a/mdl/executor/validate_excluded_page_test.go b/mdl/executor/validate_excluded_page_test.go index 2b772141c8..373a4b04fe 100644 --- a/mdl/executor/validate_excluded_page_test.go +++ b/mdl/executor/validate_excluded_page_test.go @@ -143,15 +143,27 @@ func TestBuildExcludedPage_DanglingFlowIsKeptByName(t *testing.T) { }) } - // A data source is never tolerated, excluded or not. + // A dangling data-source flow on an excluded page is kept by name too, with + // no entity in scope: the bindings inside must then be qualified, which + // DESCRIBE now emits there and the check verifies (checkUnscopedBindings). ds := &ast.DataSourceV3{Type: "nanoflow", Reference: "Feedback.DS_FeedbackForm"} - if _, _, err := newPB(true).buildDataSourceV3(ds); err == nil { - t.Error("a dangling data source must be refused even on an excluded page") + src, entity, err := newPB(true).buildDataSourceV3(ds) + if err != nil { + t.Fatalf("excluded page, dangling data-source flow: %v", err) + } + if nf, ok := src.(*pages.NanoflowSource); !ok || nf.Nanoflow != "Feedback.DS_FeedbackForm" || entity != "" { + t.Errorf("want the nanoflow source kept by name with no entity in scope; got %#v, %q", src, entity) + } + // CONTROL: a live page still refuses it. + if _, _, err := newPB(false).buildDataSourceV3(ds); err == nil { + t.Error("a live page must still refuse a dangling data source") } } -// A dangling data source on an excluded page still blocks, with the reason; -// the action targets beside it are still only warnings. +// A dangling data source on an excluded page blocks when a binding inside it is +// BARE — no entity is in scope to qualify it, and a bare attribute reference +// makes the project unloadable; the refusal names it. The action targets +// beside it are still only warnings. func TestValidateExcludedPage_DanglingDataSourceBlocks(t *testing.T) { ctx, _ := newMockCtx(t) sc := newScriptContext() @@ -162,12 +174,13 @@ func TestValidateExcludedPage_DanglingDataSourceBlocks(t *testing.T) { Properties: map[string]any{ "DataSource": &ast.DataSourceV3{Type: "nanoflow", Reference: "Feedback.DS_FeedbackForm"}, }, - Children: actionWidget("nanoflow", "Feedback.ACT_ClearForm"), + Children: append(actionWidget("nanoflow", "Feedback.ACT_ClearForm"), + &ast.WidgetV3{Name: "feedback_subject", Type: "textbox", Properties: map[string]any{"Attribute": "Subject"}}), }} err := validateWithContext(ctx, s, sc) - if err == nil || !strings.Contains(err.Error(), "nanoflow not found: Feedback.DS_FeedbackForm (data source)") || - !strings.Contains(err.Error(), "is excluded, but a data source") { + if err == nil || !strings.Contains(err.Error(), "Feedback.DS_FeedbackForm") || + !strings.Contains(err.Error(), "feedback_subject") || !strings.Contains(err.Error(), "Subject") { t.Fatalf("want the data source refused with its reason; got %v", err) } if strings.Contains(err.Error(), "ACT_ClearForm") { @@ -177,3 +190,37 @@ func TestValidateExcludedPage_DanglingDataSourceBlocks(t *testing.T) { t.Errorf("the action target must still be a warning; got %q", sc.warnings) } } + +// The same page with every binding inside the container QUALIFIED — the form +// DESCRIBE emits under an unresolvable flow — is written: the missing flow is a +// warning like any other dangling reference on an excluded page. +func TestValidateExcludedPage_DanglingDataSource_QualifiedBindingsAreWarnings(t *testing.T) { + ctx, _ := newMockCtx(t) + sc := newScriptContext() + sc.modules["Feedback"] = true + s := excludedPageStmt(true) + s.Widgets = []*ast.WidgetV3{{ + Name: "dv", Type: "dataview", + Properties: map[string]any{ + "DataSource": &ast.DataSourceV3{Type: "nanoflow", Reference: "Feedback.DS_FeedbackForm"}, + }, + Children: []*ast.WidgetV3{ + {Name: "feedback_subject", Type: "textbox", Properties: map[string]any{"Attribute": "Feedback.Feedback.Subject"}}, + {Name: "textBox1", Type: "textbox", Properties: map[string]any{ + "Attribute": "Feedback.Feedback.SubmitterEmail", + "VisibleWhen": &ast.VisibleWhenV3{Attribute: "Feedback.Feedback._showEmail", Values: []string{"true"}}, + }}, + {Name: "text1", Type: "dynamictext", Properties: map[string]any{ + "Content": "Image: {1}", + "ContentParams": []ast.ParamAssignmentV3{{Index: 1, Value: "Feedback.Feedback.ImageB64"}}, + }}, + }, + }} + if err := validateWithContext(ctx, s, sc); err != nil { + t.Fatalf("qualified bindings under a dangling flow must not block an excluded page; got:\n%v", err) + } + joined := strings.Join(sc.warnings, "\n") + if !strings.Contains(joined, "Feedback.DS_FeedbackForm") { + t.Errorf("the missing data-source flow must still be reported; got %q", sc.warnings) + } +} diff --git a/mdl/grammar/domains/MDLPage.g4 b/mdl/grammar/domains/MDLPage.g4 index 9c10023216..e717507a53 100644 --- a/mdl/grammar/domains/MDLPage.g4 +++ b/mdl/grammar/domains/MDLPage.g4 @@ -513,7 +513,7 @@ widgetPropertyV3 | WIDTH COLON NUMBER_LITERAL // Width: 200 | HEIGHT COLON NUMBER_LITERAL // Height: 100 | VISIBLE COLON xpathConstraint // Visible: [IsActive = true] - | VISIBLE COLON attributePathV3 IN LPAREN visibleValueV3 (COMMA visibleValueV3)* RPAREN // Visible: Status in (Running, empty) + | VISIBLE COLON qualifiedName IN LPAREN visibleValueV3 (COMMA visibleValueV3)* RPAREN // Visible: Status in (Running, empty) | Mod.Entity.Attr in (…) | VISIBLE COLON propertyValueV3 // Visible: false | EDITABLE COLON xpathConstraint // Editable: [Status != 'Closed'] | EDITABLE COLON propertyValueV3 // Editable: Never | Always diff --git a/mdl/visitor/visitor_page_v3.go b/mdl/visitor/visitor_page_v3.go index 3bdc4a31e0..94ffd8be4e 100644 --- a/mdl/visitor/visitor_page_v3.go +++ b/mdl/visitor/visitor_page_v3.go @@ -833,7 +833,7 @@ func parseWidgetPropertyV3(ctx parser.IWidgetPropertyV3Context, widget *ast.Widg if propCtx.VISIBLE() != nil { // `Visible: Attr in (v1, …)` — Studio Pro's "based on attribute value". if propCtx.IN() != nil { - vw := &ast.VisibleWhenV3{Attribute: buildAttributePathV3(propCtx.AttributePathV3())} + vw := &ast.VisibleWhenV3{Attribute: getQualifiedNameText(propCtx.QualifiedName())} for _, v := range propCtx.AllVisibleValueV3() { vw.Values = append(vw.Values, unquoteIdentifier(v.GetText())) } From db949df8441453f94135783c6f8aa4d7bf75ae6b Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 09:36:59 +0000 Subject: [PATCH 31/47] style: gofmt the unresolved-context describe test Co-Authored-By: Claude Opus 5.5 --- mdl/executor/cmd_pages_describe_unresolved_context_test.go | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mdl/executor/cmd_pages_describe_unresolved_context_test.go b/mdl/executor/cmd_pages_describe_unresolved_context_test.go index 72f2fbede4..dd63af4e5a 100644 --- a/mdl/executor/cmd_pages_describe_unresolved_context_test.go +++ b/mdl/executor/cmd_pages_describe_unresolved_context_test.go @@ -82,8 +82,8 @@ func TestDescribe_UnresolvedFlowContext_KeepsQualifiedBindings(t *testing.T) { // Where the entity IS known the short form is unchanged. func TestDescribe_ResolvedContext_KeepsShortBindings(t *testing.T) { got := describeWidget(t, unresolvedContextDataView(map[string]any{ - "$Type": "Forms$DataViewSource", - "EntityRef": map[string]any{"$Type": "DomainModels$DirectEntityRef", "Entity": "FeedbackModule.Feedback"}, + "$Type": "Forms$DataViewSource", + "EntityRef": map[string]any{"$Type": "DomainModels$DirectEntityRef", "Entity": "FeedbackModule.Feedback"}, "SourceVariable": map[string]any{"$Type": "Forms$PageVariable", "PageParameter": "Feedback"}, })) for _, want := range []string{"Attribute: Subject", "{1} = ImageB64", "Visible: _showEmail in (true)"} { From 713c5d7d8f3f37a3c021d532cb394f62914bade5 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 10:53:08 +0000 Subject: [PATCH 32/47] feat(pages): write DynamicClasses and DynamicCellClass as expressions PROPOSAL_first_class_expressions.md slice 2, named properties (Q4 decided: schema-driven, shipped named-properties first). A widget's DynamicClasses and a datagrid column's DynamicCellClass hold one Mendix expression, and MDL now writes it as-is: dynamicclasses: if $currentObject/Featured then 'is-featured' else '' dynamicclasses: 'is-featured' -- the string: the class is-featured A quoted value is a Mendix string - the rule the OData client's credentials follow - so the doubled-quote spelling is gone. - grammar: widgetPropertyV3 and alterPageAssignment accept `expression` LAST, so every existing value form keeps its parse. - visitor: the two properties store the source text of whichever value alternative matched (a `$x/Y + ...` can be claimed by the datasource or action alternative). A bracketed list still reaches MDL-WIDGET27/32. An expression on any other widget property, or in any other ALTER SET, is an error, so the wider rule cannot open a silent empty value. - describe prints the stored expression as-is. - MDL-WIDGET33 refuses the old spelling - a quoted string whose content looks like expression text ($, a quote, or a leading `if`) - on create and alter; it would now store that text as a class name. The suggestion is the unquoted expression. - MDL-WIDGET32 and the ALTER setter's bracketed-list error now point at the unbracketed expression instead of the quoted form. Measured on a copy of ako/TestApp (11.14.0): Appearance.DynamicClasses stores `if $currentObject/Featured then 'is-featured' else ''` and `'plain-class'`, columnClass stores its expression; describe prints the same MDL; describe -> exec leaves describe byte-identical; ALTER with the expression stores it; ALTER with the old spelling is refused with nothing written. The 28 uses in skills, docs-site, the quick reference, the syntax help, examples and bug tests are migrated by lexing each MDL string and rewriting only values whose content is expression text; proposals and the changelog keep the old spelling as history. A column's pluggable `Visible` keeps the quoted form until the schema-driven step, and the alter-page skill says so. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .claude/skills/mendix/alter-page/SKILL.md | 24 ++- .claude/skills/mendix/create-page/SKILL.md | 17 +- .../mendix/create-page/reference/widgets.md | 2 +- .../mendix/migrate-design-prototype/SKILL.md | 10 +- CHANGELOG.md | 1 + cmd/mxcli/syntax/features_page.go | 2 +- docs-site/src/appendixes/quick-reference.md | 4 +- docs-site/src/language/alter-page.md | 2 +- docs-site/src/language/widget-types.md | 4 +- docs-site/src/reference/page/create-page.md | 2 +- docs/01-project/MDL_QUICK_REFERENCE.md | 4 +- .../PROPOSAL_first_class_expressions.md | 10 + ...cclasses-legacy-quoted-expression.fail.mdl | 16 ++ .../alter-page-lowercase-set-on-builtin.mdl | 2 +- .../bug-tests/bug10-dynamic-css-classes.mdl | 4 +- .../bug-tests/widget-dynamicclasses.mdl | 6 +- .../bug-tests/widget-unknown-property.mdl | 2 +- .../doctype-tests/03-page-examples.mdl | 2 +- .../doctype-tests/12-styling-examples.mdl | 8 +- .../doctype-tests/29-datagrid-examples.mdl | 2 +- .../pagemutator/expression_list_value_test.go | 8 +- mdl/backend/pagemutator/mutator.go | 4 +- mdl/executor/cmd_pages_describe_output.go | 4 +- .../validate_widget_expression_list.go | 85 ++++++++- .../validate_widget_expression_list_test.go | 14 +- mdl/executor/validate_widgets.go | 5 + mdl/executor/widget_expression_props_test.go | 171 ++++++++++++++++++ mdl/grammar/MDLParser.g4 | 1 + mdl/grammar/domains/MDLPage.g4 | 7 + mdl/visitor/visitor_alter_page.go | 14 ++ mdl/visitor/visitor_page_v3.go | 26 +++ mdl/visitor/visitor_widget_expression.go | 64 +++++++ 32 files changed, 466 insertions(+), 61 deletions(-) create mode 100644 mdl-examples/bug-tests/750-dynamicclasses-legacy-quoted-expression.fail.mdl create mode 100644 mdl/executor/widget_expression_props_test.go create mode 100644 mdl/visitor/visitor_widget_expression.go diff --git a/.claude/skills/mendix/alter-page/SKILL.md b/.claude/skills/mendix/alter-page/SKILL.md index cf43be8ebc..d19707e4a9 100644 --- a/.claude/skills/mendix/alter-page/SKILL.md +++ b/.claude/skills/mendix/alter-page/SKILL.md @@ -470,21 +470,27 @@ Property names resolve against the keys the installed widget declares, so both the schema key and mxcli's MDL alias work (`DynamicCellClass` and `ColumnClass` both reach `columnClass`). An unknown name lists what *is* settable on that grid. -**Expression-valued properties take a Mendix expression, not a literal.** -`DynamicCellClass` and `Visible` are expressions, so a literal CSS class has to be -a quoted string *inside* the expression — doubled quotes in MDL: +**`DynamicCellClass` (and a widget's `DynamicClasses`) take a Mendix expression, +written as-is.** A quoted value is a Mendix string, so a literal CSS class is just +the quoted class name, and a computed one is the expression itself: ```mdl --- WRONG: the expression becomes a bare identifier, mxbuild reports CE0117 +-- a literal class: the string 'highlight' alter page Mod.P { SET DynamicCellClass = 'highlight' ON dg1.Label } --- correct: the expression is the string literal 'highlight' -alter page Mod.P { SET DynamicCellClass = '''highlight''' ON dg1.Label } +-- a computed class +alter page Mod.P { SET DynamicCellClass = if $currentObject/Price > 100 then 'highlight' else '' ON dg1.Label } + +-- WRONG: a bare name is an identifier, not a string — mxbuild reports CE0117 +alter page Mod.P { SET DynamicCellClass = highlight ON dg1.Label } ``` -This applies equally to `create page`; the two paths behave identically. A bare -identifier is not a valid Mendix expression, and mxbuild reports CE0117 against -the column. +The old spelling — the expression's text in quotes, `'if … then ''a'' else '''''` +— is refused as MDL-WIDGET33, because it would now store that text as a class +name. This applies equally to `create page`; the two paths behave identically. + +A column's pluggable `Visible` expression is not converted yet: there a quoted +value is still the expression's text, so a literal needs the doubled quotes. Properties holding a **structured** value — `attribute`, `filter`, `content`, actions — cannot be set by ALTER at all. It refuses them and points at diff --git a/.claude/skills/mendix/create-page/SKILL.md b/.claude/skills/mendix/create-page/SKILL.md index d4d7d232ed..c1a4432d3a 100644 --- a/.claude/skills/mendix/create-page/SKILL.md +++ b/.claude/skills/mendix/create-page/SKILL.md @@ -134,26 +134,29 @@ actionbutton btn (caption: 'Save', designproperties: ['Size': 'Large', 'Full wid ``` **Dynamic Classes** — a Mendix expression evaluated at runtime that returns a -class list (applied on top of the static `class`). Root attributes in -`$currentObject` and escape single quotes by doubling them (`''`): +class list (applied on top of the static `class`). Write the expression as-is — +no outer quotes, no doubled ones — and root attributes in `$currentObject`. A +quoted value is a Mendix string: `dynamicclasses: 'is-featured'` is the class +`is-featured`. ```sql dynamictext ovChip ( content: 'chip', class: 'ss-chip', - dynamicclasses: 'if $currentObject/VesselClass = Mod.BoatClass.Astute then ''ss-chip--astute'' else ''''' + dynamicclasses: if $currentObject/VesselClass = Mod.BoatClass.Astute then 'ss-chip--astute' else '' ) ``` -Write it quoted, not in brackets: `dynamicclasses: [ … ]` (and a column's -`DynamicCellClass: [ … ]`) parses as a list, which no writer reads — `check` -reports it as MDL-WIDGET32 rather than letting the value be dropped. +Not in brackets: `dynamicclasses: [ … ]` (and a column's `DynamicCellClass: [ … ]`) +parses as a list, which no writer reads — `check` reports it as MDL-WIDGET32. And +not the old quoted spelling `'if … then ''a'' else '''''`, which would now store +the expression's text as a class name — `check` reports it as MDL-WIDGET33. **All can be combined on a single widget:** ```sql container ctnHero ( class: 'card', style: 'border-left: 4px solid #264AE5;', - dynamicclasses: 'if $currentObject/Featured then ''is-featured'' else ''''', + dynamicclasses: if $currentObject/Featured then 'is-featured' else '', designproperties: ['Spacing top': 'Large', 'Full width': on] ) { dynamictext txtTitle (content: 'Styled Container', rendermode: H3) diff --git a/.claude/skills/mendix/create-page/reference/widgets.md b/.claude/skills/mendix/create-page/reference/widgets.md index 61b50ddbdd..256bb57d31 100644 --- a/.claude/skills/mendix/create-page/reference/widgets.md +++ b/.claude/skills/mendix/create-page/reference/widgets.md @@ -340,7 +340,7 @@ column colPrice ( Sortable: false, Resizable: false, Hidable: hidden, ColumnWidth: manual, Size: 150, - DynamicCellClass: 'if($currentObject/Price > 100) then ''highlight'' else '''' ', + DynamicCellClass: if($currentObject/Price > 100) then 'highlight' else '' , tooltip: 'Price in USD' ) ``` diff --git a/.claude/skills/mendix/migrate-design-prototype/SKILL.md b/.claude/skills/mendix/migrate-design-prototype/SKILL.md index 568e419180..423769afbd 100644 --- a/.claude/skills/mendix/migrate-design-prototype/SKILL.md +++ b/.claude/skills/mendix/migrate-design-prototype/SKILL.md @@ -501,13 +501,13 @@ top of** `Class:`. ```sql container heatCell ( Class: 'ss-heat-cell', - DynamicClasses: 'if $currentObject/M01 >= 100 then ''ss-heat--over'' - else if $currentObject/M01 >= 80 then ''ss-heat--warn'' - else ''ss-heat--ok''' + DynamicClasses: if $currentObject/M01 >= 100 then 'ss-heat--over' + else if $currentObject/M01 >= 80 then 'ss-heat--warn' + else 'ss-heat--ok' ) ``` -(Note the doubled single-quotes for string literals inside an MDL expression.) +(Written as-is: plain single quotes inside, no outer quotes around the expression.) ### Computed dimensions — the bucket-class idiom @@ -526,7 +526,7 @@ bucket and generate one class per bucket**: ``` 3. Select the class from the bucket: - `DynamicClasses: '''ss-pb-'' + toString($currentObject/PctBucket)'`. + `DynamicClasses: 'ss-pb-' + toString($currentObject/PctBucket)`. Trade-off worth noting: this adds one bucket attribute per animated dimension to the domain model. Pick a bucket count that matches the visual precision you need (20 → 5% diff --git a/CHANGELOG.md b/CHANGELOG.md index c2e278d325..3ac4a90451 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Changed +- **`DynamicClasses` and a column's `DynamicCellClass` are written as Mendix expressions** (mendixlabs/mxcli#750) — the expression is written as-is, so the doubled-quote spelling is gone: `dynamicclasses: if $currentObject/Featured then 'is-featured' else ''`, and `dynamicclasses: 'is-featured'` is the string — the class `is-featured`. The same rule as the OData client's credentials. `create page`, `alter page … set` and `describe` all use it, and a describe → exec round trip stores identical values (measured on a Mendix 11.14.0 project). **Migrating a script:** the old spelling, the expression's text in quotes (`'if … then ''a'' else '''''`), still parses but would now store that text as a class name, so `check` and `exec` refuse it as **MDL-WIDGET33** and give the unquoted expression. An expression in any other widget property is an error rather than an empty value; a pluggable property whose schema kind is Expression (a column's `Visible`, for one) keeps the quoted form until a following change. - **An OData client's credentials and header values are written as Mendix expressions** (mendixlabs/mxcli#750) — `HttpUsername`, `HttpPassword`, `ClientCertificate` and every `headers (…)` value hold an expression, and MDL now writes it as-is: `HttpUsername: 'admin'` is the string `'admin'`, `@Module.Const` reads a constant, and `'Bearer ' + @Module.Token` concatenates. Before, a quoted value was the expression's *text*, so `'admin'` stored the identifier `admin` and a string needed `'''admin'''`. `describe` prints the stored expression as-is, so Studio Pro's `'abc'` now reads `HttpUsername: 'abc'`; measured against a Studio Pro-authored client, and a describe → exec round trip stores identical values. **Migrating a script:** `'''admin'''` becomes `'admin'`, and a quoted constant `'@Module.Const'` becomes `@Module.Const` — both old forms still parse but would now store something else, so `check` and `exec` refuse them as **MDL-ODATA07**. A compound expression in any other OData property (`Path: 'a' + 'b'`) is an error rather than an empty value. `ServiceUrl` is a constant reference, not an expression — see the next entry. - **An OData client's `ServiceUrl` names a constant, like `ProxyHost`** (mendixlabs/mxcli#750) — Studio Pro picks the service URL as a constant and stores it as `@Module.Name`. `ServiceUrl: Module.Location` is now accepted alongside `@Module.Location` and `'@Module.Location'` (the bare name used to be refused as "not a constant reference"); all three store the same value, and `describe` prints the bare name, as it does for the proxy references. A literal URL is still refused (CE6825). diff --git a/cmd/mxcli/syntax/features_page.go b/cmd/mxcli/syntax/features_page.go index ecd42e9915..f8f1e46fce 100644 --- a/cmd/mxcli/syntax/features_page.go +++ b/cmd/mxcli/syntax/features_page.go @@ -317,7 +317,7 @@ CREATE PAGE Sales.Detail (Title: 'Detail', Layout: Atlas_Core.Atlas_Default) { "Reach for ALTER STYLING rather than CREATE OR REPLACE PAGE whenever only\n" + "the look changes: replacing the page rewrites every widget in it, so the\n" + "diff is the whole document and anything MDL cannot yet spell is lost.", - Example: "CONTAINER ctn (\n Class: 'my-card',\n DynamicClasses: 'if $currentObject/Priority = ''High'' then ''card-danger'' else ''card-normal'''\n) {\n DYNAMICTEXT txt (Content: 'Styled text')\n}\n\n" + + Example: "CONTAINER ctn (\n Class: 'my-card',\n DynamicClasses: if $currentObject/Priority = 'High' then 'card-danger' else 'card-normal'\n) {\n DYNAMICTEXT txt (Content: 'Styled text')\n}\n\n" + "-- Restyle one widget on a page that already exists\n" + "alter styling on page Sales.OrderOverview widget btnSave\n" + " set Class = 'btn-primary', 'Spacing top' = 'Large';\n\n" + diff --git a/docs-site/src/appendixes/quick-reference.md b/docs-site/src/appendixes/quick-reference.md index 24e4914f8f..30f8116625 100644 --- a/docs-site/src/appendixes/quick-reference.md +++ b/docs-site/src/appendixes/quick-reference.md @@ -359,7 +359,7 @@ MDL uses explicit property declarations for pages: | Selection binding | `DataSource: SELECTION widget` | `DATAVIEW dv (DataSource: SELECTION galleryList)` | | CSS class | `Class: 'classes'` | `CONTAINER c (Class: 'card mx-spacing-top-large')` | | Inline style | `Style: 'css'` | `CONTAINER c (Style: 'padding: 16px;')` | -| Dynamic classes | `DynamicClasses: 'expr'` | `CONTAINER c (DynamicClasses: 'if $currentObject/IsActive then ''is-active'' else ''''')` — runtime-computed; stacks on `Class` | +| Dynamic classes | `DynamicClasses: 'expr'` | `CONTAINER c (DynamicClasses: if $currentObject/IsActive then 'is-active' else '')` — runtime-computed; stacks on `Class` | | Design properties | `DesignProperties: [...]` | `CONTAINER c (DesignProperties: ['Spacing top': 'Large', 'Full width': ON])` | | Width (pixels) | `Width: integer` | `IMAGE img (Width: 200)` | | Height (pixels) | `Height: integer` | `IMAGE img (Height: 150)` | @@ -383,7 +383,7 @@ MDL uses explicit property declarations for pages: | `ColumnWidth` | `autoFill`, `autoFit`, `manual` | `autoFill` | `ColumnWidth: manual` | | `Size` | integer (px) | `1` | `Size: 200` | | `Visible` | expression string | `true` | `Visible: '$showColumn'` (page variable, not $currentObject) | -| `DynamicCellClass` | expression string | (empty) | `DynamicCellClass: 'if(...) then ... else ...'` | +| `DynamicCellClass` | expression string | (empty) | `DynamicCellClass: if(...) then ... else ...` | | `Tooltip` | text string | (empty) | `Tooltip: 'Price in USD'` | **Page Example:** diff --git a/docs-site/src/language/alter-page.md b/docs-site/src/language/alter-page.md index 819f3cb13a..8e4898af7a 100644 --- a/docs-site/src/language/alter-page.md +++ b/docs-site/src/language/alter-page.md @@ -43,7 +43,7 @@ ALTER PAGE Module.EditPage { | `ButtonStyle` | Button visual style | `SET ButtonStyle = Danger ON btnDelete` | | `Class` | CSS class names | `SET Class = 'card p-3' ON cMain` | | `Style` | Inline CSS | `SET Style = 'margin: 8px;' ON cBox` | -| `DynamicClasses` | Runtime-computed CSS classes | `SET DynamicClasses = 'if $currentObject/IsActive then ''is-active'' else ''''' ON cMain` | +| `DynamicClasses` | Runtime-computed CSS classes | `SET DynamicClasses = if $currentObject/IsActive then 'is-active' else '' ON cMain` | | `Editable` | Editability mode | `SET Editable = ReadOnly ON txtEmail` | | `Visible` | Visibility expression | `SET Visible = '$showField' ON txtPhone` | | `Name` | Widget name | `SET Name = 'txtFullName' ON txtName` | diff --git a/docs-site/src/language/widget-types.md b/docs-site/src/language/widget-types.md index e697de7953..8e13a2248e 100644 --- a/docs-site/src/language/widget-types.md +++ b/docs-site/src/language/widget-types.md @@ -69,7 +69,7 @@ CONTAINER cCard (Class: 'card mx-spacing-top-large') { |----------|-------------|---------| | `Class` | CSS class names | `Class: 'card p-3'` | | `Style` | Inline CSS styles | `Style: 'padding: 16px;'` | -| `DynamicClasses` | Runtime-computed CSS classes (expression; stacks on `Class`) | `DynamicClasses: 'if $currentObject/IsActive then ''is-active'' else '''''` | +| `DynamicClasses` | Runtime-computed CSS classes (expression; stacks on `Class`) | `DynamicClasses: if $currentObject/IsActive then 'is-active' else ''` | | `DesignProperties` | Design property values | `DesignProperties: ['Spacing top': 'Large']` | ### CUSTOMCONTAINER @@ -566,7 +566,7 @@ These properties are shared across many widget types: |----------|-------------|---------| | `Class` | CSS class names | `Class: 'card p-3'` | | `Style` | Inline CSS styles | `Style: 'margin-top: 8px;'` | -| `DynamicClasses` | Runtime-computed CSS classes (expression; stacks on `Class`) | `DynamicClasses: 'if $currentObject/IsActive then ''is-active'' else '''''` | +| `DynamicClasses` | Runtime-computed CSS classes (expression; stacks on `Class`) | `DynamicClasses: if $currentObject/IsActive then 'is-active' else ''` | | `DesignProperties` | Atlas design properties | `DesignProperties: ['Spacing top': 'Large', 'Full width': ON]` | | `Visible` | Visibility expression | `Visible: '$showSection'` | | `Editable` | Editability mode | `Editable: ReadOnly` | diff --git a/docs-site/src/reference/page/create-page.md b/docs-site/src/reference/page/create-page.md index 4cbd52c384..6c27500945 100644 --- a/docs-site/src/reference/page/create-page.md +++ b/docs-site/src/reference/page/create-page.md @@ -190,7 +190,7 @@ These properties are available on most widget types: |----------|-------------|---------| | `Class` | CSS class names | `Class: 'card mx-spacing-top-large'` | | `Style` | Inline CSS | `Style: 'padding: 16px;'` | -| `DynamicClasses` | Runtime-computed CSS classes (expression; stacks on `Class`) | `DynamicClasses: 'if $currentObject/IsActive then ''is-active'' else '''''` | +| `DynamicClasses` | Runtime-computed CSS classes (expression; stacks on `Class`) | `DynamicClasses: if $currentObject/IsActive then 'is-active' else ''` | | `Editable` | Edit control | `Editable: NEVER` or `Editable: ALWAYS` | | `Visible` | Visibility expression | `Visible: '$showField'` | | `DesignProperties` | Atlas design properties | `DesignProperties: ['Spacing top': 'Large']` | diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index 3ecf990031..6bff72d057 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -1455,7 +1455,7 @@ MDL uses explicit property declarations for pages: | Association source ("data from context") | `datasource: $currentObject/Module.Assoc` | nested `dataview dvCust (datasource: $currentObject/Order_Customer)` shows the to-one referenced object; a list widget shows the to-many collection | | CSS class | `class: 'classes'` | `container c (class: 'card mx-spacing-top-large')` | | Inline style | `style: 'css'` | `container c (style: 'padding: 16px;')` | -| Dynamic classes | `dynamicclasses: 'expr'` | `container c (dynamicclasses: 'if $currentObject/IsActive then ''is-active'' else ''''')` — runtime-computed classes; stacks on `class` | +| Dynamic classes | `dynamicclasses: 'expr'` | `container c (dynamicclasses: if $currentObject/IsActive then 'is-active' else '')` — runtime-computed classes; stacks on `class` | | Design properties | `designproperties: [...]` | `container c (designproperties: ['Spacing top': 'Large', 'full width': on])` | | Width (pixels) | `width: integer` | `image img (width: 200)` | | Height (pixels) | `height: integer` | `image img (height: 150)` | @@ -1521,7 +1521,7 @@ MDL uses explicit property declarations for pages: | `ColumnWidth` | `autofill`, `autoFit`, `manual` | `autofill` | `ColumnWidth: manual` | | `Size` | integer (px) | `1` | `Size: 200` | | `visible` | expression string | `true` | `visible: '$showColumn'` (page variable, not $currentObject) | -| `DynamicCellClass` | expression string | (empty) | `DynamicCellClass: 'if(...) then ... else ...'` | +| `DynamicCellClass` | expression string | (empty) | `DynamicCellClass: if(...) then ... else ...` | | `tooltip` | text string | (empty) | `tooltip: 'Price in USD'` | **Page Example:** diff --git a/docs/11-proposals/PROPOSAL_first_class_expressions.md b/docs/11-proposals/PROPOSAL_first_class_expressions.md index 09f4e4b0a4..220c2d1c4c 100644 --- a/docs/11-proposals/PROPOSAL_first_class_expressions.md +++ b/docs/11-proposals/PROPOSAL_first_class_expressions.md @@ -232,6 +232,12 @@ proposals compose rather than compete. otherwise quote. Confirm this output variance is acceptable. 4. **Pluggable expression properties are resolved by schema, not by name.** + **Decided 2026-09-25: (a)**, shipped in two steps. First the named + properties, `DynamicClasses` and a column's `DynamicCellClass`: written + as-is, a quoted value is a string (the Q5 rule), and the old quoted-text + spelling is refused as MDL-WIDGET33. Then the schema-driven extension to every + pluggable property whose kind is Expression. + Slice 2 either (a) lets any generic property take a bare expression and rejects it at check time when the widget schema says the slot is not `expression`-typed, or (b) adds the bare form only for the named properties @@ -376,6 +382,10 @@ quoted output with the bare form. **Slice 2 — expression family, widget slots.** +*Named-property step done (2026-09-25):* `DynamicClasses` and `DynamicCellClass` +take the expression as written through `create page`, `alter page … set` and +`describe`. The table below is the schema-driven remainder. + | File | Change | |---|---| | `mdl/grammar/domains/MDLPage.g4` | `widgetPropertyV3`: add `(IDENTIFIER \| keyword) COLON expression` **after** every existing generic branch, so `'text'`, numbers, booleans, qualified names and `[ … ]` keep their current parse and only what those reject (`if …`, `$v/Attr + …`, calls) reaches it. `make grammar`; watch for new ambiguity reports. | diff --git a/mdl-examples/bug-tests/750-dynamicclasses-legacy-quoted-expression.fail.mdl b/mdl-examples/bug-tests/750-dynamicclasses-legacy-quoted-expression.fail.mdl new file mode 100644 index 0000000000..18a6a1acae --- /dev/null +++ b/mdl-examples/bug-tests/750-dynamicclasses-legacy-quoted-expression.fail.mdl @@ -0,0 +1,16 @@ +-- mendixlabs/mxcli#750, PROPOSAL_first_class_expressions.md slice 2 (named +-- properties): DynamicClasses and a datagrid column's DynamicCellClass hold a +-- Mendix expression written as-is. +-- +-- dynamicclasses: if $currentObject/Featured then 'is-featured' else '' +-- dynamicclasses: 'is-featured' -- the string: the class is-featured +-- +-- The old spelling — the expression's text in quotes, with its own quotes +-- doubled — still parses, and would now store that text as a class name. check +-- and exec refuse it as MDL-WIDGET33 and name the unquoted expression. +-- +-- This file must FAIL `mxcli check`. + +create page Legacy.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + container c1 (dynamicclasses: 'if $currentObject/Featured then ''is-featured'' else ''''') { } +} diff --git a/mdl-examples/bug-tests/alter-page-lowercase-set-on-builtin.mdl b/mdl-examples/bug-tests/alter-page-lowercase-set-on-builtin.mdl index 0faf6077cb..4d2ab2b30b 100644 --- a/mdl-examples/bug-tests/alter-page-lowercase-set-on-builtin.mdl +++ b/mdl-examples/bug-tests/alter-page-lowercase-set-on-builtin.mdl @@ -55,7 +55,7 @@ create or replace page MyFirstModule.P_LowerSet -- Every one of these used to hard-error "widget has no pluggable Object". alter page MyFirstModule.P_LowerSet { set class = 'fl-topbar' on topBar; - set dynamicclasses = 'if $currentObject/Name != '''' then ''is-named'' else ''''' on topBar; + set dynamicclasses = if $currentObject/Name != '' then 'is-named' else '' on topBar; set class = 'fl-badge' on rowBadge; set caption = 'Shortcuts' on btnKeys; } diff --git a/mdl-examples/bug-tests/bug10-dynamic-css-classes.mdl b/mdl-examples/bug-tests/bug10-dynamic-css-classes.mdl index 526707f2fb..a75cde2e67 100644 --- a/mdl-examples/bug-tests/bug10-dynamic-css-classes.mdl +++ b/mdl-examples/bug-tests/bug10-dynamic-css-classes.mdl @@ -39,7 +39,7 @@ create page BugTests.Bug10 ( dynamictext t1 ( content: 'hi', class: 'base', - dynamicclasses: 'if 1 = 1 then ''mod-a'' else ''mod-b''' + dynamicclasses: if 1 = 1 then 'mod-a' else 'mod-b' ) -- 10a: DataGrid2 column DynamicCellClass must persist as an expression datagrid dg1 ( @@ -49,7 +49,7 @@ create page BugTests.Bug10 ( column colStatus ( Attribute: FullName, Caption: 'Name', - DynamicCellClass: 'if $currentObject/FullName = ''x'' then ''ea-ok'' else ''ea-draft''' + DynamicCellClass: if $currentObject/FullName = 'x' then 'ea-ok' else 'ea-draft' ) } } diff --git a/mdl-examples/bug-tests/widget-dynamicclasses.mdl b/mdl-examples/bug-tests/widget-dynamicclasses.mdl index 6268f91525..82ce02a553 100644 --- a/mdl-examples/bug-tests/widget-dynamicclasses.mdl +++ b/mdl-examples/bug-tests/widget-dynamicclasses.mdl @@ -40,12 +40,12 @@ create or replace page MyFirstModule.P_DynClass dynamictext ovClass ( Content: 'chip', Class: 'ss-chip', - DynamicClasses: 'if $currentObject/Name = ''Astute'' then ''ss-chip--astute'' else ''''' + DynamicClasses: if $currentObject/Name = 'Astute' then 'ss-chip--astute' else '' ) -- DynamicClasses on a container container ovBox ( Class: 'ss-box', - DynamicClasses: 'if $currentObject/Name = '''' then ''ss-box--empty'' else ''''' + DynamicClasses: if $currentObject/Name = '' then 'ss-box--empty' else '' ) { dynamictext boxLabel (content: 'inner') } @@ -58,7 +58,7 @@ create or replace page MyFirstModule.P_DynClass -- hard error "widget has no pluggable Object" on core widgets). alter page MyFirstModule.P_DynClass { set Class = 'ss-later' on ovLater; - set DynamicClasses = 'if $currentObject/Name != '''' then ''ss-later--named'' else ''''' on ovLater; + set DynamicClasses = if $currentObject/Name != '' then 'ss-later--named' else '' on ovLater; } describe page MyFirstModule.P_DynClass; diff --git a/mdl-examples/bug-tests/widget-unknown-property.mdl b/mdl-examples/bug-tests/widget-unknown-property.mdl index afde96beaa..f33cb726aa 100644 --- a/mdl-examples/bug-tests/widget-unknown-property.mdl +++ b/mdl-examples/bug-tests/widget-unknown-property.mdl @@ -25,7 +25,7 @@ create or replace page MyFirstModule.P_UnknownProp ( Title: 'Unknown prop', Layout: Atlas_Core.Atlas_Default ) { -- all recognized → no warning - dynamictext ok ( Content: 'hi', Class: 'c', DynamicClasses: 'if true then ''a'' else ''''', RenderMode: H1 ) + dynamictext ok ( Content: 'hi', Class: 'c', DynamicClasses: if true then 'a' else '', RenderMode: H1 ) -- typo of a real property → MDL-WIDGET07 "did you mean `Content`?" dynamictext typo ( Content: 'hi', Contnet: 'oops' ) -- genuinely unknown property → MDL-WIDGET07 (no suggestion) diff --git a/mdl-examples/doctype-tests/03-page-examples.mdl b/mdl-examples/doctype-tests/03-page-examples.mdl index cf2c5da237..992c21e960 100644 --- a/mdl-examples/doctype-tests/03-page-examples.mdl +++ b/mdl-examples/doctype-tests/03-page-examples.mdl @@ -2327,7 +2327,7 @@ create page PgTest.P033b_DataGrid_ColumnProperties attribute: Stock, caption: 'In Stock', Alignment: center, visible: '$showStockColumn', - DynamicCellClass: 'if($currentObject/Stock < 10) then ''text-danger'' else '''' ' + DynamicCellClass: if($currentObject/Stock < 10) then 'text-danger' else '' ) -- Non-hidable column with auto-fit width diff --git a/mdl-examples/doctype-tests/12-styling-examples.mdl b/mdl-examples/doctype-tests/12-styling-examples.mdl index 1938bc80b0..b443cb7f38 100644 --- a/mdl-examples/doctype-tests/12-styling-examples.mdl +++ b/mdl-examples/doctype-tests/12-styling-examples.mdl @@ -165,17 +165,17 @@ create page StyleTest.P002b_Dynamic_Classes dynamictext dcStatus ( content: 'Status', class: 'badge', - dynamicclasses: 'if $currentObject/IsActive then ''badge-success'' else ''badge-muted''' + dynamicclasses: if $currentObject/IsActive then 'badge-success' else 'badge-muted' ) -- runtime class on a container from a string attribute container dcCard ( class: 'card', - dynamicclasses: 'if $currentObject/Department = ''Sales'' then ''card-sales'' else ''card-default''' + dynamicclasses: if $currentObject/Department = 'Sales' then 'card-sales' else 'card-default' ) { dynamictext dcName ( content: 'Name', - dynamicclasses: 'if $currentObject/Name = '''' then ''is-empty'' else ''''' + dynamicclasses: if $currentObject/Name = '' then 'is-empty' else '' ) } } @@ -506,7 +506,7 @@ update widgets -- ALTER PAGE ... SET DynamicClasses ON is the surgical alternative to a -- bulk update — set (or change) a single widget's runtime class expression in place. alter page StyleTest.P002b_Dynamic_Classes { - set DynamicClasses = 'if $currentObject/Department = ''HR'' then ''card-hr'' else ''''' on dcCard; + set DynamicClasses = if $currentObject/Department = 'HR' then 'card-hr' else '' on dcCard; } -- MARK: Roundtrip diff --git a/mdl-examples/doctype-tests/29-datagrid-examples.mdl b/mdl-examples/doctype-tests/29-datagrid-examples.mdl index db8e119ae3..f77cb08b3a 100644 --- a/mdl-examples/doctype-tests/29-datagrid-examples.mdl +++ b/mdl-examples/doctype-tests/29-datagrid-examples.mdl @@ -377,7 +377,7 @@ create page DgTest.DG07_Column_Properties ( -- Status column with dynamic cell class based on value column colStatus ( attribute: Status, caption: 'Status', - DynamicCellClass: 'if ($currentObject/Status = DgTest.Status.Active) then ''text-success'' else ''text-muted''' + DynamicCellClass: if ($currentObject/Status = DgTest.Status.Active) then 'text-success' else 'text-muted' ) } }; diff --git a/mdl/backend/pagemutator/expression_list_value_test.go b/mdl/backend/pagemutator/expression_list_value_test.go index 8b1100974d..e5ee2c919c 100644 --- a/mdl/backend/pagemutator/expression_list_value_test.go +++ b/mdl/backend/pagemutator/expression_list_value_test.go @@ -37,8 +37,8 @@ func TestSetWidgetProperty_DynamicClassesRefusesAList(t *testing.T) { if err == nil { t.Fatal("a bracketed list was accepted for DynamicClasses and reported as success") } - if !strings.Contains(err.Error(), "quoted") { - t.Errorf("error = %q, want it to name the quoted spelling that works", err) + if !strings.Contains(err.Error(), "without brackets") { + t.Errorf("error = %q, want it to name the spelling that works", err) } app := bsonnav.DGetDoc(findBsonWidget(rawData, "ctn1").widget, "Appearance") if got := bsonnav.DGetString(app, "DynamicClasses"); got != stored { @@ -58,8 +58,8 @@ func TestSetColumnProperty_ExpressionRefusesAList(t *testing.T) { if err == nil { t.Fatal("a bracketed list was accepted for DynamicCellClass and reported as success") } - if !strings.Contains(err.Error(), "quoted") { - t.Errorf("error = %q, want it to name the quoted spelling that works", err) + if !strings.Contains(err.Error(), "without brackets") { + t.Errorf("error = %q, want it to name the spelling that works", err) } if got := fieldOf(t, col, idClass, "Expression"); got != "'kept'" { t.Errorf("Expression = %v after a refused set, want 'kept' unchanged", got) diff --git a/mdl/backend/pagemutator/mutator.go b/mdl/backend/pagemutator/mutator.go index ae7c5be3fb..32eaa2bbce 100644 --- a/mdl/backend/pagemutator/mutator.go +++ b/mdl/backend/pagemutator/mutator.go @@ -3205,7 +3205,7 @@ func (m *Mutator) lookupParameter(name string) (entity string, isSnippetParam bo func errExpressionNotAString(propName string, _ any) error { return fmt.Errorf( "property %q takes a single value, but was given a bracketed list — "+ - "write an expression as a quoted string, doubling the quotes inside it: "+ - "set %s = 'if $currentObject/Featured then ''a'' else ''b'''", + "write the expression itself, without brackets: "+ + "set %s = if $currentObject/Featured then 'a' else 'b'", propName, propName) } diff --git a/mdl/executor/cmd_pages_describe_output.go b/mdl/executor/cmd_pages_describe_output.go index 7d7a64acfe..4ee83ba432 100644 --- a/mdl/executor/cmd_pages_describe_output.go +++ b/mdl/executor/cmd_pages_describe_output.go @@ -181,7 +181,7 @@ func appendAppearanceProps(props []string, w rawWidget) []string { props = append(props, fmt.Sprintf("Style: %s", mdlQuote(w.Style))) } if w.DynamicClasses != "" { - props = append(props, fmt.Sprintf("DynamicClasses: %s", mdlQuote(w.DynamicClasses))) + props = append(props, fmt.Sprintf("DynamicClasses: %s", w.DynamicClasses)) // an expression, printed as-is } if len(w.DesignProperties) > 0 { props = append(props, formatDesignPropertiesMDL(w.DesignProperties)) @@ -1173,7 +1173,7 @@ func outputDataGrid2ColumnV3(ctx *ExecContext, prefix, colName string, col rawDa props = append(props, fmt.Sprintf("Visible: %s", mdlQuote(col.Visible))) } if col.DynamicCellClass != "" { - props = append(props, fmt.Sprintf("DynamicCellClass: %s", mdlQuote(col.DynamicCellClass))) + props = append(props, fmt.Sprintf("DynamicCellClass: %s", col.DynamicCellClass)) // an expression, printed as-is } if col.Tooltip != "" { props = append(props, fmt.Sprintf("Tooltip: %s", mdlQuote(col.Tooltip))) diff --git a/mdl/executor/validate_widget_expression_list.go b/mdl/executor/validate_widget_expression_list.go index 732172e56f..d51895cc58 100644 --- a/mdl/executor/validate_widget_expression_list.go +++ b/mdl/executor/validate_widget_expression_list.go @@ -4,6 +4,7 @@ package executor import ( "fmt" + "regexp" "sort" "strings" @@ -58,8 +59,8 @@ func validateExpressionPropertyLists(w *ast.WidgetV3, locationPrefix string) []l "the value is discarded on write", locationPrefix, w.Name, key), Suggestion: fmt.Sprintf( - "write the expression as a quoted string, doubling the quotes inside it: "+ - "%s: 'if $currentObject/Featured then ''is-featured'' else '''''", key), + "write the expression itself, without brackets: "+ + "%s: if $currentObject/Featured then 'is-featured' else ''", key), }) } return out @@ -73,3 +74,83 @@ func isListValuedExpressionProp(key string) bool { } return false } + +// legacyExpressionTextRe recognises the content of the OLD spelling of an +// expression property: a quoted string holding the expression's text. A class +// name or class list never contains a `$` (a variable) or a quote character, and +// does not start with `if`; the old expression text nearly always does one of +// the three. +var legacyExpressionTextRe = regexp.MustCompile(`\$|'|^\s*if\b`) + +// validateLegacyExpressionText (MDL-WIDGET33) reports the old spelling of +// DynamicClasses / DynamicCellClass: +// +// dynamicclasses: 'if $currentObject/F then ''a'' else ''''' -- old +// dynamicclasses: if $currentObject/F then 'a' else '' -- now +// +// The property holds a Mendix expression written as-is, so a quoted value is a +// Mendix string (PROPOSAL_first_class_expressions.md slice 2, the same rule as +// the OData client's credentials). The old spelling still parses and would now +// store the expression's TEXT as a class-name string — valid, silent, and never +// the class the author meant. An error, so exec refuses it too; the suggestion +// is the expression with the quoting removed. +func validateLegacyExpressionText(w *ast.WidgetV3, locationPrefix string) []linter.Violation { + if w == nil || len(w.Properties) == 0 { + return nil + } + keys := make([]string, 0, len(w.Properties)) + for k := range w.Properties { + keys = append(keys, k) + } + sort.Strings(keys) + var out []linter.Violation + for _, key := range keys { + if !isListValuedExpressionProp(key) { + continue + } + s, ok := w.Properties[key].(string) + if !ok { + continue + } + if v, bad := legacyExpressionTextViolation(fmt.Sprintf("%s: widget `%s`", locationPrefix, w.Name), key, s); bad { + out = append(out, v) + } + } + return out +} + +func legacyExpressionTextViolation(where, key, expr string) (linter.Violation, bool) { + content, isLiteral := mendixStringLiteral(expr) + if !isLiteral || !legacyExpressionTextRe.MatchString(content) { + return linter.Violation{}, false + } + return linter.Violation{ + RuleID: "MDL-WIDGET33", + Severity: linter.SeverityError, + Message: fmt.Sprintf( + "%s property `%s` is a quoted string holding an expression — %s now takes the expression itself, "+ + "so this would store the text as a class name", where, key, key), + Suggestion: fmt.Sprintf("write the expression without the outer quotes and the doubled ones: %s: %s", key, content), + }, true +} + +// validateAlterSetLegacyExpressionText is MDL-WIDGET33 for ALTER PAGE … SET. +func validateAlterSetLegacyExpressionText(op *ast.SetPropertyOp, locationPrefix string) []linter.Violation { + keys := make([]string, 0, len(op.Properties)) + for k := range op.Properties { + keys = append(keys, k) + } + sort.Strings(keys) + var out []linter.Violation + for _, key := range keys { + s, ok := op.Properties[key].(string) + if !ok || !isListValuedExpressionProp(key) { + continue + } + where := fmt.Sprintf("%s: set on `%s`", locationPrefix, op.Target.Widget) + if v, bad := legacyExpressionTextViolation(where, key, s); bad { + out = append(out, v) + } + } + return out +} diff --git a/mdl/executor/validate_widget_expression_list_test.go b/mdl/executor/validate_widget_expression_list_test.go index 09d4d81733..9f5749ffb6 100644 --- a/mdl/executor/validate_widget_expression_list_test.go +++ b/mdl/executor/validate_widget_expression_list_test.go @@ -46,19 +46,19 @@ func TestMDLWIDGET32_ExpressionPropertyWrittenAsList(t *testing.T) { want: 1, }, { - // The quoted expression is how the property is written today, and + // The expression written as-is is how the property is spelled, and // is what reaches storage: the control. - name: "control: quoted expression", + name: "control: the expression", src: `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { - container c1 (dynamicclasses: 'if $currentObject/Featured then ''is-featured'' else ''''') { } + container c1 (dynamicclasses: if $currentObject/Featured then 'is-featured' else '') { } }`, want: 0, }, { - name: "control: quoted column expression", + name: "control: the column expression", src: `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { datagrid dg (datasource: database M.Thing) { - column c1 (attribute: Name, caption: 'N', DynamicCellClass: 'if $currentObject/Featured then ''hot'' else ''''') + column c1 (attribute: Name, caption: 'N', DynamicCellClass: if $currentObject/Featured then 'hot' else '') } }`, want: 0, @@ -82,9 +82,9 @@ func TestMDLWIDGET32_ExpressionPropertyWrittenAsList(t *testing.T) { if tc.want == 0 { return } - // The message has to carry its own remedy: the quoted spelling that + // The message has to carry its own remedy: the unbracketed spelling that // does reach storage. - for _, s := range []string{"discarded", "quoted"} { + for _, s := range []string{"discarded", "without brackets"} { if !strings.Contains(got[0].Message+got[0].Suggestion, s) { t.Errorf("message should mention %q: %s / %s", s, got[0].Message, got[0].Suggestion) } diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index aac5dfa73c..9ef2ee5c3a 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -124,6 +124,8 @@ func ValidateWidgetPropertiesForStatement(stmt ast.Statement, registry *WidgetRe out = append(out, validateWidgetSubtree(o.Widgets, registry, "alter "+s.PageName.String())...) case *ast.ReplaceWidgetOp: out = append(out, validateWidgetSubtree(o.NewWidgets, registry, "alter "+s.PageName.String())...) + case *ast.SetPropertyOp: + out = append(out, validateAlterSetLegacyExpressionText(o, "alter "+s.PageName.String())...) } } return out @@ -181,6 +183,9 @@ func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, loc // An expression property written in brackets — the spelling #750 // proposes — parses as a list and was discarded on write. out = append(out, validateExpressionPropertyLists(w, locationPrefix)...) + // …and the OLD spelling, a quoted string holding the expression's text, + // which now stores that text as a class name (MDL-WIDGET33). + out = append(out, validateLegacyExpressionText(w, locationPrefix)...) // #1062: an action slot holding something that is not an action, which // used to check clean, exec clean, build clean and render dead. Runs for // every widget kind and needs no definition, for the same reason as the diff --git a/mdl/executor/widget_expression_props_test.go b/mdl/executor/widget_expression_props_test.go new file mode 100644 index 0000000000..d16cb9325f --- /dev/null +++ b/mdl/executor/widget_expression_props_test.go @@ -0,0 +1,171 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/visitor" +) + +// PROPOSAL_first_class_expressions.md slice 2, named properties: DynamicClasses +// (any widget) and a datagrid column's DynamicCellClass hold ONE Mendix +// expression, and MDL writes it as-is. A quoted value is a Mendix string — the +// same rule as the OData client's credentials (#676) — so the doubled-quote +// spelling `'if … then ''a'' else '''''` is no longer needed, and is refused. + +func pageWidgets(t *testing.T, src string) map[string]*ast.WidgetV3 { + t.Helper() + out := map[string]*ast.WidgetV3{} + var walk func(ws []*ast.WidgetV3) + walk = func(ws []*ast.WidgetV3) { + for _, w := range ws { + out[w.Name] = w + walk(w.Children) + } + } + for _, s := range parseMDL(t, src).Statements { + if p, ok := s.(*ast.CreatePageStmtV3); ok { + walk(p.Widgets) + } + } + return out +} + +func TestWidgetExpressionProps_StoreTheExpressionAsWritten(t *testing.T) { + ws := pageWidgets(t, `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + container c1 (dynamicclasses: if $currentObject/Featured then 'is-featured' else '') { } + container c2 (dynamicclasses: 'is-featured') { } + container c3 (DynamicClasses: $currentObject/Style + ' card') { } + datagrid dg (datasource: database M.Thing) { + column col1 (attribute: Name, caption: 'N', DynamicCellClass: if $currentObject/Price > 100 then 'highlight' else '') + } +}`) + for name, want := range map[string]string{ + "c1": "if $currentObject/Featured then 'is-featured' else ''", + "c2": "'is-featured'", + "c3": "$currentObject/Style + ' card'", + "col1": "if $currentObject/Price > 100 then 'highlight' else ''", + } { + w := ws[name] + if w == nil { + t.Fatalf("widget %s not parsed", name) + } + got := w.GetDynamicClasses() + if name == "col1" { + got = w.GetStringProp("DynamicCellClass") + } + if got != want { + t.Errorf("%s stored %q, want the expression %q", name, got, want) + } + } +} + +func TestWidgetExpressionProps_AlterStoresTheExpression(t *testing.T) { + prog := parseMDL(t, `alter page M.P { set DynamicClasses = if $currentObject/Featured then 'a' else 'b' on c1 };`) + stmt := prog.Statements[0].(*ast.AlterPageStmt) + var got any + for _, op := range stmt.Operations { + if set, ok := op.(*ast.SetPropertyOp); ok { + got = set.Properties["DynamicClasses"] + } + } + if got != "if $currentObject/Featured then 'a' else 'b'" { + t.Errorf("set DynamicClasses stored %#v", got) + } +} + +// Widening the value rule for two properties must not open a silent empty value +// for every other one. +func TestWidgetExpressionInAPlainProperty_IsAnError(t *testing.T) { + for _, src := range []string{ + `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { combobox cb (emptyOptionText: 'a' + 'b') }`, + `alter page M.P { set Caption = 'Save' + ' now' on btn1 };`, + } { + _, errs := visitor.Build(src) + if len(errs) == 0 { + t.Errorf("accepted an expression in a plain-value property: %s", src) + continue + } + if !strings.Contains(errs[0].Error(), "expression") { + t.Errorf("error %q should say the property does not take an expression", errs[0]) + } + } +} + +// MDL-WIDGET33: the old spelling — a quoted string holding the expression's +// text — now stores a string, so it is refused and the message names the +// unquoted expression. +func TestMDLWIDGET33_LegacyQuotedExpression(t *testing.T) { + cases := []struct { + name, value string + want int + }{ + {"legacy if-expression", `'if $currentObject/Featured then ''is-featured'' else '''''`, 1}, + {"legacy attribute concatenation", `'$currentObject/Style + '' card'''`, 1}, + {"control: a class-name string", `'is-featured'`, 0}, + {"control: a class list", `'btn btn-lg'`, 0}, + {"control: the unquoted expression", `if $currentObject/Featured then 'is-featured' else ''`, 0}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + src := `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + container c1 (dynamicclasses: ` + tc.value + `) { } + datagrid dg (datasource: database M.Thing) { + column col1 (attribute: Name, caption: 'N', DynamicCellClass: ` + tc.value + `) + } +}` + got := widgetViolations(t, src, "MDL-WIDGET33") + if len(got) != 2*tc.want { + t.Fatalf("MDL-WIDGET33: got %d, want %d: %#v", len(got), 2*tc.want, got) + } + if tc.want > 0 && !strings.Contains(got[0].Suggestion, "if $currentObject") && + !strings.Contains(got[0].Suggestion, "$currentObject/Style") { + t.Errorf("suggestion should give the unquoted expression: %s", got[0].Suggestion) + } + }) + } +} + +func TestMDLWIDGET33_LegacyQuotedExpressionOnAlter(t *testing.T) { + prog := parseMDL(t, `alter page M.P { set DynamicClasses = 'if $currentObject/F then ''a'' else ''b''' on c1 };`) + var got []string + for _, v := range ValidateWidgetProperties(prog, "") { + if v.RuleID == "MDL-WIDGET33" { + got = append(got, v.Message) + } + } + if len(got) != 1 { + t.Fatalf("MDL-WIDGET33 on ALTER: got %v, want one", got) + } +} + +// describe prints the stored expression as-is, and that output re-stores it. +func TestDescribeWidgetExpressionProps_RoundTrip(t *testing.T) { + const expr = "if $currentObject/Featured then 'is-featured' else ''" + props := appendAppearanceProps(nil, rawWidget{DynamicClasses: expr}) + var line string + for _, p := range props { + if strings.HasPrefix(p, "DynamicClasses:") { + line = p + } + } + if line != "DynamicClasses: "+expr { + t.Fatalf("describe printed %q, want the expression unquoted", line) + } + ws := pageWidgets(t, `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + container c1 (`+line+`) { } +}`) + if got := ws["c1"].GetDynamicClasses(); got != expr { + t.Errorf("re-exec of describe output stores %q, want %q", got, expr) + } + + var out bytes.Buffer + outputDataGrid2ColumnV3(&ExecContext{Output: &out}, "", "col1", rawDataGridColumn{DynamicCellClass: expr}) + if !strings.Contains(out.String(), "DynamicCellClass: "+expr) { + t.Errorf("column describe should print the expression unquoted, got:\n%s", out.String()) + } +} diff --git a/mdl/grammar/MDLParser.g4 b/mdl/grammar/MDLParser.g4 index 2d5836a5c8..fc2f79d57e 100644 --- a/mdl/grammar/MDLParser.g4 +++ b/mdl/grammar/MDLParser.g4 @@ -328,6 +328,7 @@ alterPageAssignment | identifierOrKeyword EQUALS actionExprV3 // createFileAction = MICROFLOW Module.MF | identifierOrKeyword EQUALS propertyValueV3 // Caption = 'Save' | STRING_LITERAL EQUALS propertyValueV3 // 'showLabel' = false + | identifierOrKeyword EQUALS expression // DynamicClasses = if $x/F then 'a' else '' (see widgetPropertyV3) ; alterPageInsert diff --git a/mdl/grammar/domains/MDLPage.g4 b/mdl/grammar/domains/MDLPage.g4 index e717507a53..39dc786373 100644 --- a/mdl/grammar/domains/MDLPage.g4 +++ b/mdl/grammar/domains/MDLPage.g4 @@ -538,6 +538,13 @@ widgetPropertyV3 | (IDENTIFIER | keyword) COLON actionExprV3 | IDENTIFIER COLON propertyValueV3 // Generic: any other property | keyword COLON propertyValueV3 // Generic: keyword as property name (for pluggable widgets) + // A Mendix expression, written as-is: `dynamicclasses: if $currentObject/F + // then 'a' else ''`. LAST, so every value form above keeps its parse and only + // what they all reject reaches it. The visitor accepts it only for the + // expression-typed properties (DynamicClasses, a column's DynamicCellClass) + // and refuses it elsewhere, so no plain property can read it as empty + // (PROPOSAL_first_class_expressions.md, slice 2). + | (IDENTIFIER | keyword) COLON expression ; diff --git a/mdl/visitor/visitor_alter_page.go b/mdl/visitor/visitor_alter_page.go index a45aecc065..f8d8015846 100644 --- a/mdl/visitor/visitor_alter_page.go +++ b/mdl/visitor/visitor_alter_page.go @@ -124,6 +124,20 @@ func (b *Builder) buildAlterPageSetLayout(ctx *parser.AlterPageSetContext) *ast. // buildAlterPageAssignment extracts property name and value from an assignment context. func (b *Builder) buildAlterPageAssignment(ctx *parser.AlterPageAssignmentContext) (string, interface{}) { + // An expression-typed property takes the expression as written, from + // whichever value alternative matched (visitor_widget_expression.go). + if id := ctx.IdentifierOrKeyword(); id != nil && ctx.STRING_LITERAL() == nil { + name := identifierOrKeywordText(id) + if isWidgetExpressionProp(name) { + if v := lastRuleChild(ctx); v != nil { + return name, widgetExpressionValue(v) + } + } + if expr := ctx.Expression(); expr != nil { + b.addError(widgetExpressionNotAllowed(name, expr)) + return "", nil + } + } // DataSource = dataSourceExprV3 if dsCtx := ctx.DataSourceExprV3(); dsCtx != nil { return "DataSource", buildDataSourceV3(dsCtx) diff --git a/mdl/visitor/visitor_page_v3.go b/mdl/visitor/visitor_page_v3.go index 94ffd8be4e..238b0bbc34 100644 --- a/mdl/visitor/visitor_page_v3.go +++ b/mdl/visitor/visitor_page_v3.go @@ -868,6 +868,20 @@ func parseWidgetPropertyV3(ctx parser.IWidgetPropertyV3Context, widget *ast.Widg // Generic property: Identifier: value if id := propCtx.IDENTIFIER(); id != nil { + // An expression-typed property takes the expression as written, from + // whichever value alternative matched (visitor_widget_expression.go). + if isWidgetExpressionProp(id.GetText()) { + if v := lastRuleChild(propCtx); v != nil { + widget.Properties[id.GetText()] = widgetExpressionValue(v) + } + return + } + if expr := propCtx.Expression(); expr != nil { + if b != nil { + b.addError(widgetExpressionNotAllowed(id.GetText(), expr)) + } + return + } // `Params: [{1} = Attr]` — the parameters of a text-template // sub-property whose name belongs to the WIDGET rather than to MDL (a // File Uploader custom button's ButtonCaptionParams). ContentParams and @@ -901,6 +915,18 @@ func parseWidgetPropertyV3(ctx parser.IWidgetPropertyV3Context, widget *ast.Widg // Generic property with keyword name: keyword: value (for pluggable widget property keys // that happen to be MDL keywords, e.g., type, datasource, content) if kw := propCtx.Keyword(); kw != nil { + if isWidgetExpressionProp(kw.GetText()) { + if v := lastRuleChild(propCtx); v != nil { + widget.Properties[kw.GetText()] = widgetExpressionValue(v) + } + return + } + if expr := propCtx.Expression(); expr != nil { + if b != nil { + b.addError(widgetExpressionNotAllowed(kw.GetText(), expr)) + } + return + } if plCtx := propCtx.ParamListV3(); plCtx != nil { widget.Properties[kw.GetText()] = buildParamListV3(plCtx) return diff --git a/mdl/visitor/visitor_widget_expression.go b/mdl/visitor/visitor_widget_expression.go new file mode 100644 index 0000000000..a0590ecdec --- /dev/null +++ b/mdl/visitor/visitor_widget_expression.go @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "fmt" + "strings" + + "github.com/antlr4-go/antlr/v4" + + "github.com/mendixlabs/mxcli/mdl/grammar/parser" +) + +// Widget properties that hold ONE Mendix expression, written as-is: +// +// dynamicclasses: if $currentObject/Featured then 'is-featured' else '' +// dynamicclasses: 'is-featured' -- the string: the class is-featured +// +// A quoted value is a Mendix string, as for the OData client's credentials, not +// the expression's text — the old spelling, a quoted string holding the +// expression with its own quotes doubled, is +// refused at check time (MDL-WIDGET33). This is the named-property slice of +// PROPOSAL_first_class_expressions.md §6.2 slice 2; pluggable properties whose +// schema kind is Expression follow in the schema-driven slice. +var widgetExpressionProps = map[string]bool{ + "dynamicclasses": true, // any widget's Appearance.DynamicClasses + "dynamiccellclass": true, // a datagrid column's columnClass +} + +func isWidgetExpressionProp(name string) bool { + return widgetExpressionProps[strings.ToLower(name)] +} + +// widgetExpressionValue returns an expression property's value: the source text +// of whatever the grammar matched after the `:` or `=`, whitespace kept. Which +// alternative matched is not a signal — `$x/Cls + ' x'` may be claimed by the +// datasource or action alternatives, which also start with a variable — so the +// text is taken from the value node itself. A bracketed list is the one +// exception: it is returned as the list the other readers build, so MDL-WIDGET27 +// (empty) and MDL-WIDGET32 (non-empty) still report it. +func widgetExpressionValue(valueNode antlr.ParserRuleContext) any { + if pv, ok := valueNode.(*parser.PropertyValueV3Context); ok && (pv.LBRACKET() != nil || pv.ObjectEntryListV3() != nil) { + return buildPropertyValueV3(pv) + } + return ruleSourceText(valueNode) +} + +// lastRuleChild is the rule node after `name :` / `name =` — the value. +func lastRuleChild(ctx antlr.ParserRuleContext) antlr.ParserRuleContext { + children := ctx.GetChildren() + for i := len(children) - 1; i >= 0; i-- { + if rc, ok := children[i].(antlr.ParserRuleContext); ok { + return rc + } + } + return nil +} + +func widgetExpressionNotAllowed(name string, expr parser.IExpressionContext) error { + return fmt.Errorf( + "property %s takes a plain value, not an expression: %s — "+ + "only DynamicClasses and a column's DynamicCellClass take an expression", + name, ruleSourceText(expr.(antlr.ParserRuleContext))) +} From 66c84a2419690d68126078df679137bd08b55bcf Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 10:54:52 +0000 Subject: [PATCH 33/47] feat(lint): scope a lint run to named documents with -d/--documents MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Asking a structural question about one document meant a project-wide lint — ~13 s on the reporting project — plus a baseline diff to see which findings were new. At that price the gate gets batched to once per session, and three CONV011 violations shipped under a clean mxbuild log (upstream #1186). Two flags already helped and were not mentioned in the report, so they are worth stating: `-r` picks rules, and `RequiredCatalogMode` builds only the depth those rules need (CONV011/MPR002/MPR006 declare none, so `fast`); `-m` runs through `IsExcluded`, which those rules test BEFORE `FullMicroflow`, so other modules skip the per-microflow BSON read that makes lint scale. What was missing is granularity WITHIN a module. The filter goes in the ITERATOR, not in each rule: one SQL predicate on Microflows/Pages/Widgets covers CONV011, MPR002, CONV010, QUAL003 and every other rule that walks them, with no rule edited and no second copy to drift. Naming a document also implies its module, so rules that are not iterator-narrowed still skip the rest at their existing guard. Findings are post-filtered as well, since a rule reporting from project settings rather than a document iterator is untouched by the SQL — and the matcher accepts both spellings of Location.DocumentName, because CONV010 sets the qualified name and MPR011 the short one. Two traps, both caught by trying to break my own tests: - An empty inclusion map means "no filter" to IsExcluded, so intersecting `--modules A` with `--documents B.C` — an empty set — scanned the WHOLE project instead of nothing. Distinguishing "no allowlist" from "allowlist that matched nothing" needs an explicit bool, not len(map) > 0. - The narrowing test passed against code with the SQL filter reverted: the shared fixture has one microflow per module, so the module implication alone explained the result. A sibling in the SAME module is the only fixture that isolates the two, and reverting is what exposed it. A bare name is refused before connecting: it matches nothing, and a scoped lint that matches nothing reports a clean document. Refs #681 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01MgcYSQrLLbUcnAMaCHpyqQ --- .../skills/fix-issue/findings/mdl-other.jsonl | 1 + cmd/mxcli/cmd_lint.go | 34 +++++ cmd/mxcli/lint_document_filter.go | 32 ++++ cmd/mxcli/main.go | 1 + mdl/linter/context.go | 94 ++++++++++-- mdl/linter/context_document_filter_test.go | 140 ++++++++++++++++++ 6 files changed, 293 insertions(+), 9 deletions(-) create mode 100644 cmd/mxcli/lint_document_filter.go create mode 100644 mdl/linter/context_document_filter_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-other.jsonl b/.claude/skills/fix-issue/findings/mdl-other.jsonl index a271382b25..1778b7b34e 100644 --- a/.claude/skills/fix-issue/findings/mdl-other.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-other.jsonl @@ -70,3 +70,4 @@ {"area":"mdl/executor","date":"2026-09-18","symptom":"A page parameter passed as an argument to a nanoflow/microflow BUTTON action is not wired — Studio Pro reports CE1571 \"No argument has been selected for parameter 'X' and no default is available\" on opening the page, while `mx check`, `mxcli check --references` and `mxcli lint` are all clean. Reported as an asymmetry: of two arguments, the one matching the enclosing dataview's DataSource 'works' and the other does not","cause":"Mendix stores a flow argument in one of TWO slots of Forms$MicroflowParameterMapping / Forms$NanoflowParameterMapping: a reference to a page parameter, snippet parameter or page variable goes in `Variable` as a Forms$PageVariable; a literal or expression goes in `Expression`. mxcli only ever wrote `Expression: \"$Name\"`, which binds nothing. The read side was wrong in the mirror image — the three action describers and flowSourceArgs looked for a `Name` key on that sub-document, which Forms$PageVariable does not have","file":"`sdk/pages/pages_widgets_action.go` (VariableKind on both mapping types), `mdl/executor/cmd_pages_flow_args.go` (new: classifyFlowArgValue + pageVariableArgValue), `mdl/executor/cmd_pages_builder_v3.go` (3 of the 4 copies of the $-rule), `mdl/backend/modelsdk/widget_write.go` (bindParameterMappingValue), `mdl/executor/cmd_pages_describe_output.go` + `cmd_pages_describe_datasource.go` (read)","insight":"**The reported asymmetry is a red herring — both arguments were written identically and NEITHER was bound.** Studio Pro supplies a default for the one that is the dataview's object and reports the other; 'and no default is available' in CE1571 says exactly that. Time spent on why $Dto worked is wasted. **mxbuild is not a detector here**: `mx check` on the reported project is 0 errors before AND after the fix, so the usual two-copies-of-a-real-project run proves nothing and the reporter is right that it only shows in Studio Pro. **Get the reference from a Marketplace .mpk — it contains a whole Studio Pro-authored `project.mpr`**: `mxcli marketplace download --output x.mpk && unzip -o x.mpk project.mpr`, then `mxcli bson dump` it. A blank app is useless for this (every mapping list in it is empty); Workflow Commons 4.11.0 gave 101 flow parameter mappings, of which 95 bind through Variable and 6 through Expression — and all 6 of those are Boolean literals, so the $-prefixed Expression mxcli wrote occurs ZERO times. `marketplace install` refuses that package (javasource path guard), so extract rather than install. **The PageVariable slot follows what the name refers to** (PageParameter 20, SnippetParameter 58, Widget 17) — a snippet is the COMMON case, not the corner, and `paramScope` is the right oracle because it holds only entity-typed parameters, which is the same set Mendix binds this way. **Leave $currentObject alone**: no reference for the bare form was measured and show_page already depends on the context object being inferred (MDL-PAGEARG01), so changing it on a guess risks the case that works. **The read bug hid the write bug**: describe printed `Action: microflow M.F` with no arguments for Studio Pro content, so a round-trip looked lossless and the missing binding never showed up as a diff","refs":["mendixlabs/mxcli#1140","mendixlabs/mxcli#835"],"ce":["CE1571"]} {"area": "mdl/versions", "date": "2026-09-21", "symptom": "A version gate copied from the issue text (\"Workflow Groups are GA from Mendix 11.6\") is wrong by four minors", "cause": "Mendix's release notes date the FEATURE's general availability; the metamodel floor is when the type and its property were introduced, and that is what decides whether the document loads. `Settings$WorkflowGroup` and `WorkflowsProjectSettingsPart.groups` are both `introduced: \"11.2.0\"`", "file": "`sdk/versions/mendix-11.yaml` (`workflows.groups`)", "insight": "The arbiter for a metamodel floor is the Model SDK's own StructureVersionInfo: `npm pack mendixmodelsdk && tar xzf \u2026 && grep -n '' package/src/gen/.js`, then read BOTH the class's `versionInfo.introduced` and its `properties..introduced` \u2014 a property can arrive later than its type. Release notes, proposal text and a number already written down in this repo are all downstream of it (same trap as mendixlabs/mxcli#1121). Corroborate it against two real projects rather than trusting one source: `mxcli new` at a version either side of the floor and diff the document's keys \u2014 an 11.1.0 workflows settings part has no `Groups` key at all, an 11.13.0 one carries `Groups: [2]`, which also proves the refusal is right rather than over-cautious (writing the property below the floor would be inventing a key). mendixlabs/mxcli#272", "refs": ["mendixlabs/mxcli#272"]} {"area": "mdl/linter/rules", "date": "2026-09-23", "symptom": "CONV010 flagged an ACT_ nanoflow that delegated to a sub-flow \u2014 the very thing the rule demands. A real project patched its own copy of the rule and asked for the fix upstream. An ACT_ nanoflow could satisfy CONV010 in NO way: delegate and be flagged, or inline the logic and be flagged.", "cause": "ALLOWED_ACTIONS held `MicroflowCallAction` but not `NanoflowCallAction`. `microflows()` yields nanoflows too \u2014 the catalog's `microflows` table carries a MicroflowType column \u2014 so CONV010 lints ACT_ nanoflows, and a nanoflow delegates with a nanoflow call.", "file": "`.claude/lint-rules/conv010_act_microflow_content.star` (NanoflowCallAction added to ALLOWED_ACTIONS; cmd/mxcli/lint-rules/ is gitignored and regenerated by `make sync-lint-rules`), test `mdl/catalog/lint_rule_vocabulary_test.go` (added to the `permitted` list)", "insight": "Third time this one allowlist has been short, and the rule's own comments record the previous two: the wrong vocabulary entirely (storage names vs SDK names, matching nothing, 11 false positives of 13 findings) and a missing ExclusiveMerge that a permitted ExclusiveSplit necessarily creates (122 hits on one project). The recurring shape is an UNSATISFIABLE rule, and its cost is asymmetric: a rule that cannot be satisfied does not read as a broken rule, it reads as broken CODE, so users refactor around it or patch the rule locally and the defect never comes back upstream \u2014 which is exactly what happened here until someone wrote 'report upstream' in their findings. A vocabulary pin test (TestCONV010AllowsWhatTheCatalogCallsUIActions) already existed to stop this class and did not, because its `permitted` list is hand-maintained and was itself incomplete: pinning a rule to a hand-written list of what SHOULD be allowed only moves the completeness problem. Worth considering: enumerate the delegation actions from the type system rather than listing them.", "refs": ["ako/mxcli#644"]} +{"area": "mdl/linter", "date": "2026-09-25", "symptom": "No way to ask a structural question about ONE document: `lint` scopes by module and by rule but not by document, so validating a microflow after each `exec` meant a project-wide lint (~13 s measured by the reporter) plus a baseline diff to see which findings were new. At that price the gate gets batched to once per session \u2014 three CONV011 violations shipped under a clean mxbuild log, six across three microflows before anyone looked.", "cause": "Feature gap, but the shape of the fix is not obvious: every rule guards its expensive per-document read (`FullMicroflow`) with `IsExcluded(moduleName)`, so module scoping already skipped the costly part for other modules \u2014 the missing granularity was WITHIN a module.", "file": "`mdl/linter/context.go` (`SetIncludedDocuments`, `IsDocumentExcluded`, `documentFilterSQL`; `Microflows`/`Pages`/`Widgets` narrowed in SQL; new `includeActive` flag), `cmd/mxcli/cmd_lint.go` + `main.go` (`-d/--documents`), `cmd/mxcli/lint_document_filter.go` (violation post-filter), tests `mdl/linter/context_document_filter_test.go`", "insight": "**Scope in the ITERATOR, not in each rule** \u2014 one SQL predicate on `Microflows()` covers CONV011, MPR002, CONV010, QUAL003 and every other rule that walks it, with no rule edited and no second copy of the filter to drift. **Two traps, both found by tests I wrote and then tried to break.** (1) An empty inclusion map means 'no filter' to `IsExcluded`, so intersecting `--modules A` with `--documents B.C` \u2014 an empty set \u2014 made lint scan the WHOLE project instead of nothing; distinguishing 'no allowlist' from 'allowlist that matched nothing' needs an explicit bool, not `len(map) > 0`. (2) The narrowing test PASSED against code with the SQL filter reverted, because the shared fixture has one microflow per module and the module implication alone explained the result \u2014 a sibling document in the SAME module is the only fixture that isolates document narrowing from module narrowing. Prove-by-revert is what caught both; the second would otherwise have shipped a test that could never fail. Also: a rule that reports from project settings rather than a document iterator is untouched by SQL narrowing, so a scoped run needs a violation post-filter too \u2014 and it must accept BOTH spellings of Location.DocumentName, since CONV010 sets the qualified name and MPR011 the short one.", "refs": ["ako/mxcli#681", "mendixlabs/mxcli#1186"]} diff --git a/cmd/mxcli/cmd_lint.go b/cmd/mxcli/cmd_lint.go index a517cc5f10..f2ff170aad 100644 --- a/cmd/mxcli/cmd_lint.go +++ b/cmd/mxcli/cmd_lint.go @@ -88,6 +88,13 @@ Examples: mxcli lint -p app.mpr -r MPR001,SEC001 mxcli lint -p app.mpr -m MyModule mxcli lint -p app.mpr -m MyModule -m AnotherModule + mxcli lint -p app.mpr -d Sales.ACT_Order + mxcli lint -p app.mpr -d Sales.ACT_Order -r CONV011 + +--documents scopes a run to named documents, so a structural question about one +microflow costs a fraction of a project-wide lint and needs no baseline diff to +see what is new. It narrows the document iterators and implies the documents' +modules, and findings that are not about those documents are not reported. `, Run: func(cmd *cobra.Command, args []string) { projectPath, _ := cmd.Flags().GetString("project") @@ -97,6 +104,18 @@ Examples: excludeModules, _ := cmd.Flags().GetStringSlice("exclude") onlyRules, _ := cmd.Flags().GetStringSlice("rules") moduleFilter, _ := cmd.Flags().GetStringSlice("modules") + documentFilter, _ := cmd.Flags().GetStringSlice("documents") + // Checked before connecting: a malformed filter is the caller's typo, and + // making them wait for a catalog build to hear about it is the slow half + // of the problem --documents exists to fix. + for _, d := range documentFilter { + if !strings.Contains(d, ".") { + fmt.Fprintf(os.Stderr, "Error: --documents takes QUALIFIED names; %q names no module.\n", d) + fmt.Fprintln(os.Stderr, " Use Module.Document, e.g. Sales.ACT_Order. A bare name matches nothing,") + fmt.Fprintln(os.Stderr, " and a scoped lint that matches nothing reports a clean document.") + os.Exit(1) + } + } if projectPath == "" { fmt.Fprintln(os.Stderr, "Error: --project (-p) is required") @@ -151,6 +170,12 @@ Examples: if len(moduleFilter) > 0 { ctx.SetIncludedModules(moduleFilter) } + // --documents narrows the document iterators AND implies their modules, + // so rules this package does not narrow still skip the other modules + // before their per-document read (ako/mxcli#681). + if len(documentFilter) > 0 { + ctx.SetIncludedDocuments(documentFilter) + } // Safety net: if a rule needs data the catalog still lacks (e.g. a project // with no cross-references, or a build that couldn't populate the graph), @@ -240,6 +265,15 @@ Examples: os.Exit(1) } + // A rule that reports without walking a document iterator (a project + // setting, a security policy) is unaffected by the SQL narrowing above, + // so a scoped run would still carry its findings. Drop anything that is + // not one of the named documents: `lint -d A.B` reporting something about + // C is the same class of lie as a gate that passes what it did not read. + if len(documentFilter) > 0 { + violations = keepDocumentViolations(violations, documentFilter) + } + // Output results outputFormat := linter.OutputFormat(format) formatter := linter.GetFormatter(outputFormat, useColor) diff --git a/cmd/mxcli/lint_document_filter.go b/cmd/mxcli/lint_document_filter.go new file mode 100644 index 0000000000..37d3d754b7 --- /dev/null +++ b/cmd/mxcli/lint_document_filter.go @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import "github.com/mendixlabs/mxcli/mdl/linter" + +// keepDocumentViolations drops findings that are not about one of the named +// documents. +// +// The SQL narrowing in LintContext covers the rules that walk a document +// iterator, which is where the time goes. It cannot cover a rule that reports +// from project settings or from a security policy — those still run, and a +// scoped lint would carry their findings alongside the one document the caller +// asked about. `lint -d Sales.ACT_Order` answering about anything else is the +// same class of lie as a gate that passes what it never read (ako/mxcli#681). +// +// Matching accepts BOTH spellings of Location.DocumentName, because the rules +// disagree: CONV010 sets it to the qualified name and MPR011 to the short name, +// so a matcher that picked one would silently drop half the rules' findings. +func keepDocumentViolations(violations []linter.Violation, documents []string) []linter.Violation { + want := make(map[string]bool, len(documents)) + for _, d := range documents { + want[d] = true + } + kept := make([]linter.Violation, 0, len(violations)) + for _, v := range violations { + if want[v.Location.DocumentName] || want[v.Location.QualifiedName()] { + kept = append(kept, v) + } + } + return kept +} diff --git a/cmd/mxcli/main.go b/cmd/mxcli/main.go index 13bb1c81ec..ffc9fb009a 100644 --- a/cmd/mxcli/main.go +++ b/cmd/mxcli/main.go @@ -373,6 +373,7 @@ func init() { lintCmd.Flags().StringSliceP("exclude", "e", nil, "Modules to exclude from linting") lintCmd.Flags().StringSliceP("rules", "r", nil, "Only run these rule IDs (e.g. -r MPR001 -r SEC001)") lintCmd.Flags().StringSliceP("modules", "m", nil, "Only lint the specified modules (comma-separated or repeated)") + lintCmd.Flags().StringSliceP("documents", "d", nil, "Only lint these documents, by qualified name (e.g. Sales.ACT_Order); comma-separated or repeated") // Report command flags reportCmd.Flags().StringP("format", "f", "markdown", "Output format: markdown, json, html") diff --git a/mdl/linter/context.go b/mdl/linter/context.go index 0ae76e048b..a9ff0e9347 100644 --- a/mdl/linter/context.go +++ b/mdl/linter/context.go @@ -6,6 +6,7 @@ import ( "database/sql" "fmt" "iter" + "sort" "strings" "sync" @@ -36,8 +37,17 @@ type LintContext struct { catalog *catalog.Catalog db catalog.CatalogDB excluded map[string]bool - included map[string]bool // when non-empty, only these modules are linted - reader LintReader + included map[string]bool // the module allowlist; see includeActive + // includeActive distinguishes "no module allowlist" from "an allowlist that + // matched nothing". Without it an empty `included` map means the former, so + // `--modules A --documents B.C` — which names an empty set — would lint the + // WHOLE project instead of nothing. Caught by its own test. + includeActive bool + // documents is the --document allowlist, by QUALIFIED name. When non-empty + // the document iterators below return only these, so a rule that walks + // microflows stops paying for the ones nobody asked about (ako/mxcli#681). + documents map[string]bool + reader LintReader // fullMFCache memoizes every fully-parsed microflow for the lifetime of the // lint run (see FullMicroflow). Populated lazily on first FullMicroflow call. @@ -119,12 +129,78 @@ func (ctx *LintContext) SetExcludedModules(modules []string) { // SetIncludedModules sets an allowlist of modules to lint. When non-empty, // only modules in this list are linted (modules not in the list are skipped). func (ctx *LintContext) SetIncludedModules(modules []string) { + if len(modules) == 0 { + return + } ctx.included = make(map[string]bool) + ctx.includeActive = true for _, m := range modules { ctx.included[m] = true } } +// SetIncludedDocuments sets an allowlist of documents to lint, by qualified +// name ("Sales.ACT_Order"). It also narrows the module allowlist to the modules +// those documents live in: every rule already guards its expensive per-document +// read with IsExcluded(moduleName), so the module implication is what makes a +// scoped lint faster in rules this package does not otherwise narrow. +func (ctx *LintContext) SetIncludedDocuments(docs []string) { + ctx.documents = make(map[string]bool) + mods := map[string]bool{} + for _, d := range docs { + ctx.documents[d] = true + if i := strings.LastIndex(d, "."); i > 0 { + mods[d[:i]] = true + } + } + if len(mods) == 0 { + return + } + // Intersect rather than replace: --modules A --documents B.C names an empty + // set and must lint nothing, not all of B. Replacing would silently widen an + // explicit filter, which is the direction that turns a scoped run into a + // project-wide one without saying so. + if !ctx.includeActive { + ctx.included = mods + ctx.includeActive = true + return + } + for m := range ctx.included { + if !mods[m] { + delete(ctx.included, m) + } + } +} + +// IsDocumentExcluded reports whether a document is outside the --document +// allowlist. With no allowlist nothing is excluded, so an unscoped lint is +// unaffected. +// +// qualifiedName is "Module.Document". A caller holding the two halves separately +// should join them; a caller holding only a short name cannot use this. +func (ctx *LintContext) IsDocumentExcluded(qualifiedName string) bool { + return len(ctx.documents) > 0 && !ctx.documents[qualifiedName] +} + +// documentFilterSQL returns a SQL predicate narrowing an iterator to the +// --document allowlist, or "1=1" when there is none. +// +// It is applied in the QUERY rather than in each rule because that is the one +// place it covers every rule at once: CONV011 and MPR002 both walk Microflows() +// and both call FullMicroflow() per row, which is the read that makes lint scale +// with project size. +func (ctx *LintContext) documentFilterSQL(column string) string { + if len(ctx.documents) == 0 { + return "1=1" + } + quoted := make([]string, 0, len(ctx.documents)) + for d := range ctx.documents { + quoted = append(quoted, "'"+strings.ReplaceAll(d, "'", "''")+"'") + } + sort.Strings(quoted) // deterministic SQL, so a failure is reproducible + return column + " IN (" + strings.Join(quoted, ", ") + ")" +} + // IsExcluded returns true if the module should be skipped during linting. // A module is skipped when it is explicitly excluded, or when an inclusion // filter is active and the module is not in it. @@ -132,7 +208,7 @@ func (ctx *LintContext) IsExcluded(moduleName string) bool { if ctx.excluded[moduleName] { return true } - if len(ctx.included) > 0 && !ctx.included[moduleName] { + if ctx.includeActive && !ctx.included[moduleName] { return true } return false @@ -553,9 +629,9 @@ func (ctx *LintContext) Microflows() iter.Seq[Microflow] { mf.ParameterCount, mf.ActivityCount, mf.Complexity FROM microflows mf LEFT JOIN modules m ON mf.ModuleName = m.Name - WHERE %s + WHERE %s AND %s ORDER BY mf.ModuleName, mf.Name - `, notPlatformModule("m"))) + `, notPlatformModule("m"), ctx.documentFilterSQL("mf.QualifiedName"))) if err != nil { ctx.recordQueryError("Microflows", err) return @@ -607,9 +683,9 @@ func (ctx *LintContext) Pages() iter.Seq[Page] { p.Title, p.URL, p.Description, p.WidgetCount FROM pages p LEFT JOIN modules m ON p.ModuleName = m.Name - WHERE %s + WHERE %s AND %s ORDER BY p.ModuleName, p.Name - `, notPlatformModule("m"))) + `, notPlatformModule("m"), ctx.documentFilterSQL("p.QualifiedName"))) if err != nil { ctx.recordQueryError("Pages", err) return @@ -773,9 +849,9 @@ func (ctx *LintContext) Widgets() iter.Seq[Widget] { w.MicroflowRef, w.NanoflowRef FROM widgets w LEFT JOIN modules m ON w.ModuleName = m.Name - WHERE %s + WHERE %s AND %s ORDER BY w.ModuleName, w.ContainerQualifiedName, w.Name - `, notPlatformModule("m"))) + `, notPlatformModule("m"), ctx.documentFilterSQL("w.ContainerQualifiedName"))) if err != nil { ctx.recordQueryError("Widgets", err) return diff --git a/mdl/linter/context_document_filter_test.go b/mdl/linter/context_document_filter_test.go new file mode 100644 index 0000000000..caa921795a --- /dev/null +++ b/mdl/linter/context_document_filter_test.go @@ -0,0 +1,140 @@ +// SPDX-License-Identifier: Apache-2.0 + +// --documents (ako/mxcli#681, upstream mendixlabs/mxcli#1186). +// +// A project-wide lint was the only way to ask a structural question about one +// document: a reporter measured ~13 s per exec and maintained a baseline diff on +// top, so the gate got batched to once per session and three CONV011 violations +// shipped under a clean mxbuild log. +// +// Scoping is in the ITERATOR rather than in each rule because that is the one +// place it covers every rule at once. CONV011 and MPR002 both walk Microflows() +// and both call FullMicroflow() per row — that read is what makes lint scale +// with project size, and narrowing the query skips it without touching a rule. +package linter + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/catalog" +) + +// twoInOneModuleDB adds a SECOND microflow to ModB. +// +// The shared fixture has one microflow per module, which cannot distinguish +// iterator narrowing from the module implication — naming ModB.Flow excludes +// ModA and ModC either way, so a test on that fixture passes against code with +// no document filter at all. Reverting the SQL narrowing proved exactly that, +// and the first version of the test below was green against it. +// +// A sibling in the SAME module is the only thing that isolates the two. +func twoInOneModuleDB(t *testing.T) catalog.CatalogDB { + t.Helper() + db := setupModuleFilterDB(t) + if _, err := db.Exec(`INSERT INTO microflows VALUES (?, ?, ?, ?, '', 'Microflow', '', '', 0, 0, 0)`, + "ModB_mf2", "ModB_Sibling", "ModB.Sibling", "ModB"); err != nil { + t.Fatalf("insert sibling microflow: %v", err) + } + return db +} + +func collectMicroflowNames(ctx *LintContext) []string { + var out []string + for mf := range ctx.Microflows() { + out = append(out, mf.QualifiedName) + } + return out +} + +func TestDocumentFilter_NarrowsTheIterator(t *testing.T) { + ctx := NewLintContextFromDB(twoInOneModuleDB(t)) + ctx.SetIncludedDocuments([]string{"ModB.Flow"}) + + got := collectMicroflowNames(ctx) + if len(got) != 1 || got[0] != "ModB.Flow" { + t.Errorf("Microflows() = %v, want exactly [ModB.Flow] — ModB.Sibling is in the "+ + "SAME module, so only the iterator filter can exclude it, and a rule walking "+ + "this iterator must not pay FullMicroflow() for documents nobody asked about", got) + } +} + +// CONTROL: with no allowlist the iterator is unchanged. Without this the filter +// could be "return nothing unless asked", which every existing lint run would hit. +func TestDocumentFilter_AbsentMeansEverything(t *testing.T) { + ctx := NewLintContextFromDB(twoInOneModuleDB(t)) + if got := collectMicroflowNames(ctx); len(got) != 4 { + t.Errorf("Microflows() = %v with no --documents, want all 4", got) + } +} + +// The module implication is what makes a scoped lint faster in the rules this +// package does not narrow: they all guard their per-document read with +// IsExcluded(moduleName), so naming a document must exclude the other modules. +func TestDocumentFilter_ImpliesItsModule(t *testing.T) { + ctx := NewLintContextFromDB(setupModuleFilterDB(t)) + ctx.SetIncludedDocuments([]string{"ModB.Flow"}) + + if ctx.IsExcluded("ModB") { + t.Error("ModB excluded, but a document in it was named") + } + for _, other := range []string{"ModA", "ModC"} { + if !ctx.IsExcluded(other) { + t.Errorf("%s not excluded — a rule that is not iterator-narrowed would "+ + "still read every document in it", other) + } + } +} + +// --modules and --documents INTERSECT. Replacing would silently widen an +// explicit filter, which is the direction that turns a scoped run into a +// project-wide one without saying so. +func TestDocumentFilter_IntersectsWithModuleFilter(t *testing.T) { + ctx := NewLintContextFromDB(setupModuleFilterDB(t)) + ctx.SetIncludedModules([]string{"ModA"}) + ctx.SetIncludedDocuments([]string{"ModB.Flow"}) + + for _, m := range []string{"ModA", "ModB", "ModC"} { + if !ctx.IsExcluded(m) { + t.Errorf("%s not excluded: --modules ModA with --documents ModB.Flow "+ + "names an empty set, and must lint nothing rather than all of either", m) + } + } +} + +// An exclude still wins. IsExcluded checks the exclude set first, and +// --documents must not be a way around a config that excludes a module. +func TestDocumentFilter_DoesNotOverrideAnExclude(t *testing.T) { + ctx := NewLintContextFromDB(setupModuleFilterDB(t)) + ctx.SetExcludedModules([]string{"ModB"}) + ctx.SetIncludedDocuments([]string{"ModB.Flow"}) + + if !ctx.IsExcluded("ModB") { + t.Error("an explicitly excluded module became lintable by naming a document in it") + } +} + +// A qualified name is data from the command line and reaches a SQL string. +func TestDocumentFilter_QuoteInNameDoesNotBreakTheQuery(t *testing.T) { + ctx := NewLintContextFromDB(twoInOneModuleDB(t)) + ctx.SetIncludedDocuments([]string{"ModB.O'Brien", "ModB.Flow"}) + + got := collectMicroflowNames(ctx) + if len(got) != 1 || got[0] != "ModB.Flow" { + t.Errorf("Microflows() = %v, want [ModB.Flow]; an apostrophe in a sibling "+ + "name must not break the query (which would yield nothing and read as clean)", got) + } +} + +func TestIsDocumentExcluded(t *testing.T) { + ctx := NewLintContextFromDB(setupModuleFilterDB(t)) + if ctx.IsDocumentExcluded("ModA.Flow") { + t.Error("nothing is document-excluded before an allowlist is set") + } + ctx.SetIncludedDocuments([]string{"ModA.Flow"}) + if ctx.IsDocumentExcluded("ModA.Flow") { + t.Error("the named document reports as excluded") + } + if !ctx.IsDocumentExcluded("ModA.Other") { + t.Error("an unnamed document in the same module reports as included") + } +} From 61a90bb4ec59e7c855a585f3eff0ad31ff0d22f1 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 10:54:52 +0000 Subject: [PATCH 34/47] feat(check): warn on a commit inside a loop before it is written (MDL-PERF01) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mxcli check` passed a microflow committing once per iteration that `lint` already knew as CONV011 (upstream #1186). The gap is temporal, not a missing capability: CONV011 reads the STORED model, so it cannot speak until `exec` has written the microflow. MDL-PERF01 reads the MDL before it is applied, so the answer arrives in the call that would otherwise have written the defect. The two are complementary — this cannot see a microflow it is not being asked to write, and CONV011 cannot see one before it exists — so the message names CONV011 and a reader hitting one recognises the other rather than filing it twice. **The boundary is deliberately CONV011's, not a better one.** A `while true` is built as an ExclusiveMerge back-edge rather than a LoopedActivity, so CONV011 does not flag a commit inside one, and neither does this. A commit there is arguably still N+1 at runtime, and being more correct is the tempting move — but two rules for one concept that disagree on what counts is how a pair drifts, and this repo already has CONV010's three successive short allowlists as the worked example. If that case is worth reporting it is worth reporting in both, and CONV011 is the one that sees the built flow. The test pinning it carries a control on the control: `while true` exempt, `while ` flagged, so the exemption cannot quietly become "never flag a while". `checkReturnInLoop` already had the depth-tracking walk over Loop/While/If/EnumSplit/InheritanceSplit, so the nesting cases came free. A warning, not an error: committing per iteration is sometimes deliberate. What it must not be is invisible. Closes #681 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01MgcYSQrLLbUcnAMaCHpyqQ --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../bug-tests/1186-commit-in-loop.mdl | 47 ++++++++ mdl/executor/validate_commit_in_loop.go | 76 ++++++++++++ mdl/executor/validate_commit_in_loop_test.go | 112 ++++++++++++++++++ mdl/executor/validate_microflow.go | 5 + 5 files changed, 241 insertions(+) create mode 100644 mdl-examples/bug-tests/1186-commit-in-loop.mdl create mode 100644 mdl/executor/validate_commit_in_loop.go create mode 100644 mdl/executor/validate_commit_in_loop_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 97246e4d4e..ed70b75f46 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -705,3 +705,4 @@ {"area":"mdl/executor","date":"2026-09-25","symptom":"describe → exec of an EXCLUDED page (Feedback v4.0.2 FeedbackModule.ShareFeedback_Logo, 11.13.0) refused: `page '…' has reference errors: - nanoflow not found: FeedbackModule.DS_FeedbackForm …`, though the untouched project passes mx check at 0 errors (Mendix does not validate excluded documents). With --no-check the page builder refused the same names again (`failed to resolve nanoflow`).","cause":"Two refusals, not one: validate.go's CreatePageStmtV3/CreateSnippetStmtV3 cases ignored exclusion (microflow/nanoflow/rule had been exempt since #312, silently), and pageBuilder.resolveMicroflow/resolveNanoflowByName/resolvePageRef/resolveSnippetRef fail on a missing name though the writer only ever stores the qualified NAME (IDs are never serialized).","file":"mdl/executor/validate.go (relaxExcludedWidgetRefs, carriedExclusion, warnExcluded), mdl/executor/cmd_pages_builder.go (tolerateDanglingRefs/danglingRefOK), cmd/mxcli/cmd_exec.go + cmd_check.go (ValidateProgramWithWarnings)","insight":"Relaxing the check is NOT safe for a DATA SOURCE, and only a real run shows it: the source flow's return type is the entity in scope, describe prints the nested bindings as bare names (`Attribute: Subject`, `ImageUrlParams: [{1} = ImageB64]`), and writing them without the entity left a bare `ImageB64` AttributeRef that made mx unable to LOAD the project (ArgumentNullException setting 'Attribute') — excluded page or not, where the pre-fix refusal had been protecting it by accident. So dangling action targets/snippet calls are warnings, dangling data sources and entities still block with the reason. 'Excluded' must mean what exec WRITES — @excluded OR the #914 carry (every stored namesake excluded) — or check and exec disagree. A/B on 11.13.0: identical page with 3 dangling action targets, excluded → 0 errors; live → 3x CE1613. Follow-up not fixed: describe loses attribute qualification inside a container whose flow is unresolvable, so ShareFeedback_Logo itself still cannot round-trip.","refs":["mdl-examples/bug-tests/excluded-page-dangling-references.mdl","#312","#914"]} {"area": "mdl/executor", "symptom": "describe → exec of Feedback v4.0.2's EXCLUDED FeedbackModule.ShareFeedback_Logo refused `nanoflow not found: FeedbackModule.DS_FeedbackForm (data source)`; forcing it through left a project `mx check` could not LOAD (ArgumentNullException setting 'Attribute')", "cause": "The data view's flow is not in the project, so DESCRIBE had no context entity and printed every binding inside it bare (`Attribute: Subject`, `{1} = ImageB64`, `Visible: _showEmail in (…)`); exec had nothing to qualify them against, and a bare DomainModels$AttributeRef makes Mendix's loader throw. The stored model always had the full names", "file": "`mdl/executor/cmd_pages_describe_flowcontext.go` (`withQualifiedAttrs`, `describeAttr`), `mdl/executor/validate.go` (`unscopedBindings`), `mdl/executor/cmd_pages_builder_v3.go` (dangling DS flow kept by name), `mdl/backend/modelsdk/page_bare_attributeref.go` (`refuseBareAttributeRefs`)", "insight": "**The information was never lost — DESCRIBE threw it away**: every AttributeRef inside the unresolvable container still stored Module.Entity.Attr; shortening to the bare name is only safe where the reader can re-derive the entity, so key the shortening on whether the context resolved, not on habit. Measure the loader's tolerance before adding a write guard: 72 of 72 Studio Pro AttributeRefs in the project are qualified, so refusing a bare one refuses only writes that were already fatal — and turns a load-time stack trace into a statement-level error naming the attribute. Pair a blanket AST-level refusal with the exact failing slots (Attribute, CaptionAttribute, Visible-in, *Params) so the refusal names the widget, and keep the writer guard as the net for slots the walk does not know. Forced-fault run: hand-edit one qualified binding back to bare and confirm the refusal names it", "refs": ["ako/mxcli#675", "FeedbackModule.ShareFeedback_Logo"], "date": "2026-09-25"} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_New fails `mx check` with [CE0642] \"Property 'Caption' is required.\" at Combo box 'comboBox2' (Account_Edit comboBox4 too); check and exec report success", "cause": "The ComboBox caption is an EXPRESSION (optionsSourceAssociationCaptionType=expression, optionsSourceAssociationCaptionExpression='$currentObject/Description'); describe read only the attribute caption, so the widget was rewritten with none. The write side already worked via the explicit-property pass, but MDL-WIDGET06 claimed both keys 'will be dropped'", "file": "`mdl/executor/cmd_pages_describe_parse.go` (combobox branch), `cmd_pages_describe_output.go`, `validate_widget_explicit_writable.go` (`persistedByExplicitPass`), `validate_widgets.go` (MDL-WIDGET06)", "insight": "**Resolve pluggable-widget properties by key before theorising**: a 20-line script mapping each Property's TypePointer to its PropertyKey through the widget's own Type.ObjectType settled the cause in one run, where the ndsl dump shows only anonymous values. Then TEST THE WRITE SIDE before building one: adding the two storage keys to the describe output by hand persisted both and built clean, which shrank the fix to describe + a false warning — no new syntax. A validator rule that asserts 'not persisted' must be tied to what the write path handles (here: the explicit pass writes Expression/TextTemplate/Attribute and scalar types), or it goes stale when the writer grows; the #643 test pinned the stale claim. A dedicated extractor for a 'known' pluggable widget silently drops every property it does not map — generic widgets emit them, known ones do not", "refs": ["ako/mxcli#664", "ako/mxcli#643"], "ce": ["CE0642"], "rules": ["MDL-WIDGET06"], "date": "2026-09-25"} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "`mxcli check` passed a microflow with a commit inside a loop \u2014 one database round trip per iteration \u2014 that `mxcli lint` already flagged as CONV011. The defect surfaced only at project-wide lint time, long after the write.", "cause": "CONV011 reads the STORED model, so it cannot speak until `exec` has written the microflow; `check` reads the MDL and had no equivalent rule. The gap is temporal, not a missing capability on either side.", "file": "`mdl/executor/validate_commit_in_loop.go` (MDL-PERF01, hooked in `validate_microflow.go`), test `validate_commit_in_loop_test.go`, example `mdl-examples/bug-tests/1186-commit-in-loop.mdl`", "insight": "**When adding a check-time rule that anticipates an existing lint rule, pin the BOUNDARY to the lint rule's, not to the better one, and say why in the code.** A `while true` is built as an ExclusiveMerge back-edge rather than a LoopedActivity, so CONV011 (which walks LoopedActivity) does not flag a commit inside one. A commit there is arguably still N+1, and the tempting move is to be more correct \u2014 but two rules for one concept that disagree on what counts is precisely how a pair drifts, and this repo already has CONV010's three successive short allowlists as the worked example. If the case is worth reporting it is worth reporting in BOTH, and the stored-model rule is the one that sees the built flow. The test that pins this carries a control on the control: `while true` is exempt, `while ` is not, so the exemption cannot silently become 'never flag a while'. Name the sibling rule in the message (`lint reports this as CONV011`) so a reader hitting one recognises the other rather than filing it twice. Also worth reusing: `checkReturnInLoop` already had the depth-tracking walk over Loop/While/If/EnumSplit/InheritanceSplit \u2014 copying its shape got the nesting cases right for free.", "refs": ["ako/mxcli#681", "mendixlabs/mxcli#1186"], "rules": ["MDL-PERF01"]} diff --git a/mdl-examples/bug-tests/1186-commit-in-loop.mdl b/mdl-examples/bug-tests/1186-commit-in-loop.mdl new file mode 100644 index 0000000000..1da05419ad --- /dev/null +++ b/mdl-examples/bug-tests/1186-commit-in-loop.mdl @@ -0,0 +1,47 @@ +-- upstream mendixlabs/mxcli#1186: `mxcli check` passed a commit inside a loop +-- that `lint` already knew as CONV011. +-- +-- CONV011 reads the STORED model, so it can only speak after `exec` has written +-- the microflow, and it had no document scope — asking about one microflow meant +-- a project-wide lint (~13 s on the reporting project) plus a baseline diff to +-- see what was new. At that price the gate gets batched to once per session: +-- three CONV011 violations shipped under a clean mxbuild log, and six +-- accumulated across three microflows before anyone looked. +-- +-- MDL-PERF01 reads the MDL before it is applied, so the answer arrives in the +-- call that would otherwise have written the defect. The two are complementary: +-- this cannot see a microflow it is not being asked to write, and CONV011 cannot +-- see one before it exists. +-- +-- The companion is `mxcli lint -d Module.Document`, which scopes a lint to named +-- documents for the after-the-write case. +-- +-- Verify (no project needed — this is a check-time rule): +-- mxcli check mdl-examples/bug-tests/1186-commit-in-loop.mdl +-- expect: MDL-PERF01 warning on SUB_Commit_Per_Row, nothing on SUB_Commit_Once. + +create module CommitLoop; + +create entity CommitLoop.Row ( + "Name": string, + Done: boolean +); + +-- The reported shape: one database round trip per iteration. +create microflow CommitLoop.SUB_Commit_Per_Row($Rows: list of CommitLoop.Row) +BEGIN + LOOP $Row IN $Rows BEGIN + CHANGE $Row (Done = true); + COMMIT $Row; + END LOOP; +END + +-- The fix the rule suggests, and the control that it clears the warning: +-- change inside the loop, commit the list once after it. +create microflow CommitLoop.SUB_Commit_Once($Rows: list of CommitLoop.Row) +BEGIN + LOOP $Row IN $Rows BEGIN + CHANGE $Row (Done = true); + END LOOP; + COMMIT $Rows; +END diff --git a/mdl/executor/validate_commit_in_loop.go b/mdl/executor/validate_commit_in_loop.go new file mode 100644 index 0000000000..d699db433e --- /dev/null +++ b/mdl/executor/validate_commit_in_loop.go @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// checkCommitInLoop (MDL-PERF01) reports a commit inside a loop, which is one +// database round trip per iteration. +// +// `lint` has known this as CONV011 since long before this rule, and that is the +// problem it exists to solve rather than duplicate: CONV011 reads the STORED +// model, so it can only speak after `exec` has written the microflow, and it has +// no document scope — answering for one microflow meant a project-wide lint +// (~13 s measured on a real project) plus a baseline diff to see what was new. +// At that price the gate gets batched to once per session, and on the project +// that reported it (upstream mendixlabs/mxcli#1186) three CONV011 violations +// shipped under a clean mxbuild log, six accumulating across three microflows +// before anyone looked. +// +// This one reads the MDL the author just wrote, before it is applied, so the +// answer arrives in the same call that would have written the defect. The two +// are complementary, not alternatives: this cannot see a microflow it is not +// being asked to write, and CONV011 cannot see one before it exists. +// +// **The boundary is deliberately CONV011's, not a better one.** A `while true` +// is built as an ExclusiveMerge back-edge rather than a LoopedActivity, so +// CONV011 — which walks LoopedActivity — does not flag a commit inside one, and +// neither does this. A commit there is arguably still N+1 at runtime, but two +// rules for one concept that disagree on what counts is how a pair like this +// starts drifting; if that case is worth reporting it is worth reporting in +// both, and CONV011 is the one that sees the built flow. +// +// A warning, not an error: committing per iteration is sometimes what the author +// means (a long-running job that must not lose work on failure). What it must not +// be is invisible. +func (v *microflowValidator) checkCommitInLoop(body []ast.MicroflowStatement) { + var walk func(stmts []ast.MicroflowStatement, inLoop bool) + walk = func(stmts []ast.MicroflowStatement, inLoop bool) { + for _, s := range stmts { + switch st := s.(type) { + case *ast.MfCommitStmt: + if inLoop { + v.addViolation("MDL-PERF01", linter.SeverityWarning, + fmt.Sprintf("commit of $%s is inside a loop, so it runs one database round trip "+ + "per iteration (`lint` reports this as CONV011)", st.Variable), + fmt.Sprintf("Add $%s to a list inside the loop and commit the list once after it", st.Variable)) + } + case *ast.LoopStmt: + walk(st.Body, true) + case *ast.WhileStmt: + // See the note above: a `while true` is a back-edge, not a loop + // object, and CONV011 does not count it either. + walk(st.Body, inLoop || !isUnconditionalTrueWhile(st)) + case *ast.IfStmt: + walk(st.ThenBody, inLoop) + walk(st.ElseBody, inLoop) + case *ast.EnumSplitStmt: + for _, c := range st.Cases { + walk(c.Body, inLoop) + } + walk(st.ElseBody, inLoop) + case *ast.InheritanceSplitStmt: + for _, c := range st.Cases { + walk(c.Body, inLoop) + } + walk(st.ElseBody, inLoop) + } + } + } + walk(body, false) +} diff --git a/mdl/executor/validate_commit_in_loop_test.go b/mdl/executor/validate_commit_in_loop_test.go new file mode 100644 index 0000000000..b307262645 --- /dev/null +++ b/mdl/executor/validate_commit_in_loop_test.go @@ -0,0 +1,112 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +func commitInLoopViolations(t *testing.T, body []ast.MicroflowStatement) []string { + t.Helper() + v := µflowValidator{} + v.checkCommitInLoop(body) + var msgs []string + for _, viol := range v.violations { + if viol.RuleID == "MDL-PERF01" { + msgs = append(msgs, viol.Message) + } + } + return msgs +} + +func commit(v string) ast.MicroflowStatement { return &ast.MfCommitStmt{Variable: v} } + +func loopOver(list string, body ...ast.MicroflowStatement) ast.MicroflowStatement { + return &ast.LoopStmt{ListVariable: list, LoopVariable: "Item", Body: body} +} + +// The reported case: one commit per iteration, one database round trip per +// iteration. `lint` has known it as CONV011 all along, but only after the write +// and only project-wide (upstream mendixlabs/mxcli#1186). +func TestCommitInLoopIsReported(t *testing.T) { + got := commitInLoopViolations(t, []ast.MicroflowStatement{ + loopOver("Items", commit("Item")), + }) + if len(got) != 1 { + t.Fatalf("got %d findings, want 1: %v", len(got), got) + } + if !strings.Contains(got[0], "CONV011") { + t.Errorf("the message does not name CONV011, so a reader cannot tell this is "+ + "the same finding lint reports:\n%s", got[0]) + } +} + +// CONTROL: a commit AFTER the loop is the fix this rule suggests, and must not +// be flagged — otherwise following the suggestion does not clear the warning. +func TestCommitAfterLoopIsNotReported(t *testing.T) { + got := commitInLoopViolations(t, []ast.MicroflowStatement{ + loopOver("Items", &ast.ChangeObjectStmt{}), + commit("Batch"), + }) + if len(got) != 0 { + t.Errorf("commit after the loop was flagged: %v", got) + } +} + +// A commit nested in a branch inside the loop still runs per iteration. Walking +// only the loop's direct statements would miss the realistic shape. +func TestCommitInsideABranchInsideALoopIsReported(t *testing.T) { + got := commitInLoopViolations(t, []ast.MicroflowStatement{ + loopOver("Items", &ast.IfStmt{ + ThenBody: []ast.MicroflowStatement{commit("Item")}, + ElseBody: []ast.MicroflowStatement{&ast.EnumSplitStmt{ + Cases: []ast.EnumSplitCase{{Body: []ast.MicroflowStatement{commit("Other")}}}, + ElseBody: []ast.MicroflowStatement{commit("Third")}, + }}, + }), + }) + if len(got) != 3 { + t.Errorf("got %d findings, want 3 (if / enum case / enum else): %v", len(got), got) + } +} + +// THE ALIGNMENT CONTROL. A `while true` is built as an ExclusiveMerge back-edge, +// not a LoopedActivity, so CONV011 — which walks LoopedActivity — does not flag a +// commit inside one. Neither does this rule, deliberately. +// +// A commit there is arguably still N+1 at runtime, and that is exactly why the +// boundary is pinned: two rules for one concept that disagree on what counts is +// how a pair like this drifts. If the case is worth reporting it is worth +// reporting in both, and CONV011 is the one that sees the built flow. +func TestWhileTrueMatchesCONV011AndIsNotReported(t *testing.T) { + whileTrue := &ast.WhileStmt{ + Condition: &ast.LiteralExpr{Kind: ast.LiteralBoolean, Value: true}, + Body: []ast.MicroflowStatement{commit("Item")}, + } + if got := commitInLoopViolations(t, []ast.MicroflowStatement{whileTrue}); len(got) != 0 { + t.Errorf("a commit in `while true` was flagged, which CONV011 does not do: %v", got) + } + + // CONTROL on the control: a while with a REAL condition IS a loop object, so + // it must be flagged. Without this the exemption could be "never flag a + // while", which would silently drop the whole construct. + realWhile := &ast.WhileStmt{ + Condition: &ast.LiteralExpr{Kind: ast.LiteralBoolean, Value: false}, + Body: []ast.MicroflowStatement{commit("Item")}, + } + if got := commitInLoopViolations(t, []ast.MicroflowStatement{realWhile}); len(got) != 1 { + t.Errorf("a commit in a conditional while was not flagged: %v", got) + } +} + +// A loop with no commit is the ordinary case and must stay quiet. +func TestLoopWithoutCommitIsNotReported(t *testing.T) { + if got := commitInLoopViolations(t, []ast.MicroflowStatement{ + loopOver("Items", &ast.ChangeObjectStmt{}), + }); len(got) != 0 { + t.Errorf("a loop with no commit was flagged: %v", got) + } +} diff --git a/mdl/executor/validate_microflow.go b/mdl/executor/validate_microflow.go index 3c5c6880cf..981b8c93b4 100644 --- a/mdl/executor/validate_microflow.go +++ b/mdl/executor/validate_microflow.go @@ -157,6 +157,11 @@ func (v *microflowValidator) validate(body []ast.MicroflowStatement) { // #895: the commit default changed to match Studio Pro. One informational // note per microflow, not per statement — see validate_commit_events.go. v.checkBareCommitEvents(body) + + // upstream mendixlabs/mxcli#1186: `lint` has known commit-in-a-loop as + // CONV011 all along, but only after the write and only project-wide. See + // validate_commit_in_loop.go. + v.checkCommitInLoop(body) } // checkDuplicateLoopVariables flags a loop iterator name used by more than one From 2451d7ea00934ef710d6c824a61453d300f7c654 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 11:09:34 +0000 Subject: [PATCH 35/47] fix(check): ALTER STYLING names the current property for a renamed key MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `alter styling … set 'Spacing bottom' = 'Outer medium'` uses an Atlas Core 4.1.3 old name, which the theme keeps in design-properties.json `oldNames`. It was warned as undeclared ("no widget type … declares — mxbuild reports this as CE6083") and given a spelling near-miss. mxbuild reports it as CE6087 "Design properties have been renamed in your theme", measured on 11.13.0. ALTER STYLING now looks an undeclared key up among the theme's old names, as the authoring paths do since #679, and names the current property: - a flat replacement becomes one `set 'New' = 'Value'`; - a compound one (a Spacing side, a multi-select option) can't be written by ALTER STYLING's single flat value, so the suggestion points at the inline DesignProperties form. The type-blind policy is unchanged: a key currently declared on any widget type is still accepted. Real run on FeedbackModule.ShareFeedback: the old name gives the rename warning and, once exec'd, CE6087. The suggested inline form builds with 0 errors. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../alter-styling-renamed-design-property.mdl | 44 +++++++++++ mdl/executor/validate_alter_styling.go | 53 +++++++++++++ .../validate_alter_styling_renamed_test.go | 76 +++++++++++++++++++ 4 files changed, 174 insertions(+) create mode 100644 mdl-examples/bug-tests/alter-styling-renamed-design-property.mdl create mode 100644 mdl/executor/validate_alter_styling_renamed_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 97246e4d4e..41ec707ce2 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -705,3 +705,4 @@ {"area":"mdl/executor","date":"2026-09-25","symptom":"describe → exec of an EXCLUDED page (Feedback v4.0.2 FeedbackModule.ShareFeedback_Logo, 11.13.0) refused: `page '…' has reference errors: - nanoflow not found: FeedbackModule.DS_FeedbackForm …`, though the untouched project passes mx check at 0 errors (Mendix does not validate excluded documents). With --no-check the page builder refused the same names again (`failed to resolve nanoflow`).","cause":"Two refusals, not one: validate.go's CreatePageStmtV3/CreateSnippetStmtV3 cases ignored exclusion (microflow/nanoflow/rule had been exempt since #312, silently), and pageBuilder.resolveMicroflow/resolveNanoflowByName/resolvePageRef/resolveSnippetRef fail on a missing name though the writer only ever stores the qualified NAME (IDs are never serialized).","file":"mdl/executor/validate.go (relaxExcludedWidgetRefs, carriedExclusion, warnExcluded), mdl/executor/cmd_pages_builder.go (tolerateDanglingRefs/danglingRefOK), cmd/mxcli/cmd_exec.go + cmd_check.go (ValidateProgramWithWarnings)","insight":"Relaxing the check is NOT safe for a DATA SOURCE, and only a real run shows it: the source flow's return type is the entity in scope, describe prints the nested bindings as bare names (`Attribute: Subject`, `ImageUrlParams: [{1} = ImageB64]`), and writing them without the entity left a bare `ImageB64` AttributeRef that made mx unable to LOAD the project (ArgumentNullException setting 'Attribute') — excluded page or not, where the pre-fix refusal had been protecting it by accident. So dangling action targets/snippet calls are warnings, dangling data sources and entities still block with the reason. 'Excluded' must mean what exec WRITES — @excluded OR the #914 carry (every stored namesake excluded) — or check and exec disagree. A/B on 11.13.0: identical page with 3 dangling action targets, excluded → 0 errors; live → 3x CE1613. Follow-up not fixed: describe loses attribute qualification inside a container whose flow is unresolvable, so ShareFeedback_Logo itself still cannot round-trip.","refs":["mdl-examples/bug-tests/excluded-page-dangling-references.mdl","#312","#914"]} {"area": "mdl/executor", "symptom": "describe → exec of Feedback v4.0.2's EXCLUDED FeedbackModule.ShareFeedback_Logo refused `nanoflow not found: FeedbackModule.DS_FeedbackForm (data source)`; forcing it through left a project `mx check` could not LOAD (ArgumentNullException setting 'Attribute')", "cause": "The data view's flow is not in the project, so DESCRIBE had no context entity and printed every binding inside it bare (`Attribute: Subject`, `{1} = ImageB64`, `Visible: _showEmail in (…)`); exec had nothing to qualify them against, and a bare DomainModels$AttributeRef makes Mendix's loader throw. The stored model always had the full names", "file": "`mdl/executor/cmd_pages_describe_flowcontext.go` (`withQualifiedAttrs`, `describeAttr`), `mdl/executor/validate.go` (`unscopedBindings`), `mdl/executor/cmd_pages_builder_v3.go` (dangling DS flow kept by name), `mdl/backend/modelsdk/page_bare_attributeref.go` (`refuseBareAttributeRefs`)", "insight": "**The information was never lost — DESCRIBE threw it away**: every AttributeRef inside the unresolvable container still stored Module.Entity.Attr; shortening to the bare name is only safe where the reader can re-derive the entity, so key the shortening on whether the context resolved, not on habit. Measure the loader's tolerance before adding a write guard: 72 of 72 Studio Pro AttributeRefs in the project are qualified, so refusing a bare one refuses only writes that were already fatal — and turns a load-time stack trace into a statement-level error naming the attribute. Pair a blanket AST-level refusal with the exact failing slots (Attribute, CaptionAttribute, Visible-in, *Params) so the refusal names the widget, and keep the writer guard as the net for slots the walk does not know. Forced-fault run: hand-edit one qualified binding back to bare and confirm the refusal names it", "refs": ["ako/mxcli#675", "FeedbackModule.ShareFeedback_Logo"], "date": "2026-09-25"} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_New fails `mx check` with [CE0642] \"Property 'Caption' is required.\" at Combo box 'comboBox2' (Account_Edit comboBox4 too); check and exec report success", "cause": "The ComboBox caption is an EXPRESSION (optionsSourceAssociationCaptionType=expression, optionsSourceAssociationCaptionExpression='$currentObject/Description'); describe read only the attribute caption, so the widget was rewritten with none. The write side already worked via the explicit-property pass, but MDL-WIDGET06 claimed both keys 'will be dropped'", "file": "`mdl/executor/cmd_pages_describe_parse.go` (combobox branch), `cmd_pages_describe_output.go`, `validate_widget_explicit_writable.go` (`persistedByExplicitPass`), `validate_widgets.go` (MDL-WIDGET06)", "insight": "**Resolve pluggable-widget properties by key before theorising**: a 20-line script mapping each Property's TypePointer to its PropertyKey through the widget's own Type.ObjectType settled the cause in one run, where the ndsl dump shows only anonymous values. Then TEST THE WRITE SIDE before building one: adding the two storage keys to the describe output by hand persisted both and built clean, which shrank the fix to describe + a false warning — no new syntax. A validator rule that asserts 'not persisted' must be tied to what the write path handles (here: the explicit pass writes Expression/TextTemplate/Attribute and scalar types), or it goes stale when the writer grows; the #643 test pinned the stale claim. A dedicated extractor for a 'known' pluggable widget silently drops every property it does not map — generic widgets emit them, known ones do not", "refs": ["ako/mxcli#664", "ako/mxcli#643"], "ce": ["CE0642"], "rules": ["MDL-WIDGET06"], "date": "2026-09-25"} +{"area": "mdl/executor", "symptom": "`alter styling … set 'Spacing bottom' = 'Outer medium'` (an Atlas Core 4.1.3 old name) warned MDL-WIDGET11 \"which no widget type in this project's theme declares — mxbuild reports this as CE6083\"; mxbuild actually reports CE6087 \"Design properties have been renamed in your theme\"", "cause": "validateAlterStylingDesignProps checked only current property names across all widget groups; #679 taught the authoring paths the theme's oldNames but not ALTER STYLING, which then offered a spelling near-miss instead of the current property", "file": "`mdl/executor/validate_alter_styling.go` (`renamedAnywhere`, `renamedStylingSuggestion`)", "insight": "**Pick the real-run example by what the resolver can see**: ALTER STYLING knows only a widget NAME, so it asks the whole theme — and Atlas 4.1.3 declares a CURRENT 'Align content' on the Image widget while 'Align content' is an old name on DivContainer, so that example is (correctly) silent under the under-report policy; 'Spacing bottom' is declared nowhere under a current name and exercises the rename path. A renamed key whose replacement is a compound (Spacing side, multi-select option) cannot be written by ALTER STYLING's one flat value, so the suggestion must point at the inline DesignProperties form rather than a `set` it cannot execute. Real run: old name via exec → CE6087; the suggested inline form → 0 errors", "refs": ["ako/mxcli#679"], "rules": ["MDL-WIDGET11"], "date": "2026-09-25"} diff --git a/mdl-examples/bug-tests/alter-styling-renamed-design-property.mdl b/mdl-examples/bug-tests/alter-styling-renamed-design-property.mdl new file mode 100644 index 0000000000..df6f1812df --- /dev/null +++ b/mdl-examples/bug-tests/alter-styling-renamed-design-property.mdl @@ -0,0 +1,44 @@ +-- ============================================================================ +-- ALTER STYLING with a design-property key the theme RENAMED +-- ============================================================================ +-- +-- Symptom: `alter styling … set 'Spacing bottom' = 'Outer medium'` — an Atlas +-- Core 4.1.3 old name, kept in design-properties.json `oldNames` — was reported +-- MDL-WIDGET11 … which no widget type in this project's theme declares — +-- mxbuild reports this as CE6083 +-- but mxbuild reports it as CE6087 "Design properties have been renamed in your +-- theme and need to be updated" (measured, Mendix 11.13.0), and the fix is the +-- current property, not a spelling near-miss. +-- +-- Fix: ALTER STYLING looks the key up among the theme's old names (as the +-- authoring paths do since #679) and names the current property. A compound +-- replacement (a Spacing side, a multi-select option) cannot be written by ALTER +-- STYLING, so the suggestion points at the inline DesignProperties form. +-- +-- Verify: `mxcli check -p --references` shows the +-- "renamed to "Spacing" … CE6087" warning on the first statement, none on the +-- second, and `mxcli docker check` after exec is 0 errors (only the second +-- statement is meant to be applied). +-- ============================================================================ + +create or replace page MyFirstModule.StylingRenamed_Test +( Title: 'Styling', Layout: Atlas_Core.Atlas_Default ) +{ + container cntBody { + dynamictext txt (Content: 'Hello') + } +} +/ + +-- The old name: warned, with the current form to use instead. +-- alter styling on page MyFirstModule.StylingRenamed_Test widget cntBody set 'Spacing bottom' = 'Outer medium'; + +-- The current name, written the way the warning says: 0 errors. +alter page MyFirstModule.StylingRenamed_Test { + replace cntBody with { + container cntBody (DesignProperties: ['Spacing': ['margin-bottom': 'M']]) { + dynamictext txtSpaced (Content: 'Hello') + } + } +}; +/ diff --git a/mdl/executor/validate_alter_styling.go b/mdl/executor/validate_alter_styling.go index 919ddeea1d..1623f334c9 100644 --- a/mdl/executor/validate_alter_styling.go +++ b/mdl/executor/validate_alter_styling.go @@ -105,6 +105,21 @@ func validateAlterStylingDesignProps(prog *ast.Program, reg *ThemeRegistry) []li } continue } + // Declared after all — under an OLD name the theme renamed. mxbuild + // answers that with CE6087, not CE6083, and the fix is the current + // name, not a spelling near-miss. + if r := renamedAnywhere(reg, a.Property, a.Value); r != nil { + out = append(out, linter.Violation{ + RuleID: "MDL-WIDGET11", + Severity: linter.SeverityWarning, + Message: fmt.Sprintf("%s: sets design property %q on %q, which the theme renamed to %q "+ + "— mxbuild reports the old name as CE6087 (\"Design properties have been renamed "+ + "in your theme\")", label, a.Property, s.WidgetName, r.NewKey), + Location: linter.Location{DocumentType: "page", DocumentName: s.ContainerName.String()}, + Suggestion: renamedStylingSuggestion(r, a.Value), + }) + continue + } out = append(out, linter.Violation{ RuleID: "MDL-WIDGET11", Severity: linter.SeverityWarning, @@ -196,3 +211,41 @@ func firstOptionName(p *ThemeProperty, authored string) string { } return authored } + +// renamedAnywhere finds key among the old names of any widget type's design +// properties. ALTER STYLING carries only a widget NAME, so — as for the +// declared check above — it asks the whole theme rather than one type. Groups +// are visited in a fixed order so the answer does not depend on map order. +func renamedAnywhere(reg *ThemeRegistry, key, value string) *designPropRename { + groups := make([]string, 0, len(reg.WidgetProperties)) + for g := range reg.WidgetProperties { + groups = append(groups, g) + } + sort.Strings(groups) + for _, g := range groups { + if r := findRenamedThemeProp(reg.WidgetProperties[g], key, value); r != nil { + return r + } + } + return nil +} + +// renamedStylingSuggestion turns a rename into what to write INSTEAD in an +// ALTER STYLING script. A flat replacement ('Key': 'Option' or 'Key': on) is +// one assignment; a compound one — a Spacing side, or a multi-select option — +// is a value ALTER STYLING cannot write (one flat value per assignment, the +// MDL-WIDGET12 limit), so it points at the inline form. +func renamedStylingSuggestion(r *designPropRename, value string) string { + if r.Replacement == "" { + return renamedDesignPropSuggestion(r, value) + } + if strings.Contains(r.Replacement, "[") { + return fmt.Sprintf("ALTER STYLING cannot write its current form, which is a compound: set "+ + "`DesignProperties: [%s]` on the widget in CREATE PAGE, or in an ALTER PAGE REPLACE.", r.Replacement) + } + // 'Key': 'Value' → set 'Key' = 'Value' + if i := strings.Index(r.Replacement, "': "); i > 0 { + return fmt.Sprintf("Write it as `set %s' = %s`.", r.Replacement[:i], r.Replacement[i+3:]) + } + return renamedDesignPropSuggestion(r, value) +} diff --git a/mdl/executor/validate_alter_styling_renamed_test.go b/mdl/executor/validate_alter_styling_renamed_test.go new file mode 100644 index 0000000000..409ca92af8 --- /dev/null +++ b/mdl/executor/validate_alter_styling_renamed_test.go @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" +) + +// ALTER STYLING reported a design-property key the theme RENAMED as if it were +// unknown: "no widget type in this project's theme declares — mxbuild reports +// this as CE6083", and suggested a near-miss by spelling. The key is declared — +// as an old name — and mxbuild answers a stored old name with CE6087 "Design +// properties have been renamed in your theme and need to be updated" (measured +// on Mendix 11.13.0 with Atlas Core 4.1.3's "Align content" / "Spacing bottom", +// PR #679). The authoring paths already say "was renamed to X"; this is the +// same answer on the one statement that writes design properties by name. +func TestAlterStyling_RenamedKey_NamesTheCurrentProperty(t *testing.T) { + reg := renamedThemeRegistry(t) + cases := []struct { + name, set string + want []string // in message or suggestion + wantNot []string + }{ + { + name: "renamed property, value mapped through old option names", + set: `set 'Align content' = 'Left align as column'`, + want: []string{"renamed", "Align content (deprecated)", "CE6087", `set 'Align content (deprecated)' = 'Left align as a column'`}, + wantNot: []string{"CE6083"}, + }, + { + // A Spacing side became one side of a compound, which ALTER STYLING + // cannot write (one flat value — the MDL-WIDGET12 limit): point at + // the inline form instead. + name: "spacing side renamed into a compound", + set: `set 'Spacing bottom' = 'Outer medium'`, + want: []string{"renamed", "CE6087", "'Spacing': ['margin-bottom': 'M']", "DesignProperties"}, + wantNot: []string{"CE6083"}, + }, + { + name: "multi-select toggle renamed into an option", + set: `set 'Hide on phone' = on`, + want: []string{"renamed", "CE6087", "'Hide on': ['Phone': on]", "DesignProperties"}, + wantNot: []string{"CE6083"}, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + prog := parseMDL(t, "alter styling on page M.P widget c1 "+tc.set+";") + got := validateAlterStylingDesignProps(prog, reg) + if len(got) != 1 || got[0].RuleID != "MDL-WIDGET11" { + t.Fatalf("want one MDL-WIDGET11, got %#v", got) + } + text := got[0].Message + " | " + got[0].Suggestion + for _, w := range tc.want { + if !strings.Contains(text, w) { + t.Errorf("lacks %q:\n%s", w, text) + } + } + for _, w := range tc.wantNot { + if strings.Contains(text, w) { + t.Errorf("must not say %q for a renamed key:\n%s", w, text) + } + } + }) + } +} + +// CONTROL: a key the theme knows under no name is still undeclared, CE6083. +func TestAlterStyling_UnknownKeyStillCE6083(t *testing.T) { + prog := parseMDL(t, "alter styling on page M.P widget c1 set 'Spacing bottomx' = 'Outer medium';") + got := validateAlterStylingDesignProps(prog, renamedThemeRegistry(t)) + if len(got) != 1 || !strings.Contains(got[0].Message, "CE6083") || strings.Contains(got[0].Message, "renamed") { + t.Fatalf("an unknown key must still be reported as undeclared (CE6083); got %#v", got) + } +} From 60bd4fa878f32540c913916dbc986989b500b9ce Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 11:11:47 +0000 Subject: [PATCH 36/47] fix(check): resolve a `label`'s design properties against the theme's Label group The `label` keyword (Forms$Label) had no entry in mdlKeywordToDesignPropsKey, so it resolved to the theme key "label", which no design-properties.json defines. The validator skipped every label (FeedbackModule.ShareFeedback_Logo label1's renamed 'Spacing bottom' went unreported while its sibling containers warned MDL-WIDGET11), and the builder saw only the "Widget" base group, so a Label's ColorPicker "Style" colour was written as an Option value and mxbuild refused it with CE6085. Map `label` to "Label", the key bsonTypeToDesignPropsKey already uses for a stored Forms$Label. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../label-design-properties-check.mdl | 37 ++++++ mdl/executor/label_design_properties_test.go | 115 ++++++++++++++++++ mdl/executor/theme_reader.go | 16 ++- 4 files changed, 163 insertions(+), 6 deletions(-) create mode 100644 mdl-examples/bug-tests/label-design-properties-check.mdl create mode 100644 mdl/executor/label_design_properties_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 97246e4d4e..3717222e25 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -705,3 +705,4 @@ {"area":"mdl/executor","date":"2026-09-25","symptom":"describe → exec of an EXCLUDED page (Feedback v4.0.2 FeedbackModule.ShareFeedback_Logo, 11.13.0) refused: `page '…' has reference errors: - nanoflow not found: FeedbackModule.DS_FeedbackForm …`, though the untouched project passes mx check at 0 errors (Mendix does not validate excluded documents). With --no-check the page builder refused the same names again (`failed to resolve nanoflow`).","cause":"Two refusals, not one: validate.go's CreatePageStmtV3/CreateSnippetStmtV3 cases ignored exclusion (microflow/nanoflow/rule had been exempt since #312, silently), and pageBuilder.resolveMicroflow/resolveNanoflowByName/resolvePageRef/resolveSnippetRef fail on a missing name though the writer only ever stores the qualified NAME (IDs are never serialized).","file":"mdl/executor/validate.go (relaxExcludedWidgetRefs, carriedExclusion, warnExcluded), mdl/executor/cmd_pages_builder.go (tolerateDanglingRefs/danglingRefOK), cmd/mxcli/cmd_exec.go + cmd_check.go (ValidateProgramWithWarnings)","insight":"Relaxing the check is NOT safe for a DATA SOURCE, and only a real run shows it: the source flow's return type is the entity in scope, describe prints the nested bindings as bare names (`Attribute: Subject`, `ImageUrlParams: [{1} = ImageB64]`), and writing them without the entity left a bare `ImageB64` AttributeRef that made mx unable to LOAD the project (ArgumentNullException setting 'Attribute') — excluded page or not, where the pre-fix refusal had been protecting it by accident. So dangling action targets/snippet calls are warnings, dangling data sources and entities still block with the reason. 'Excluded' must mean what exec WRITES — @excluded OR the #914 carry (every stored namesake excluded) — or check and exec disagree. A/B on 11.13.0: identical page with 3 dangling action targets, excluded → 0 errors; live → 3x CE1613. Follow-up not fixed: describe loses attribute qualification inside a container whose flow is unresolvable, so ShareFeedback_Logo itself still cannot round-trip.","refs":["mdl-examples/bug-tests/excluded-page-dangling-references.mdl","#312","#914"]} {"area": "mdl/executor", "symptom": "describe → exec of Feedback v4.0.2's EXCLUDED FeedbackModule.ShareFeedback_Logo refused `nanoflow not found: FeedbackModule.DS_FeedbackForm (data source)`; forcing it through left a project `mx check` could not LOAD (ArgumentNullException setting 'Attribute')", "cause": "The data view's flow is not in the project, so DESCRIBE had no context entity and printed every binding inside it bare (`Attribute: Subject`, `{1} = ImageB64`, `Visible: _showEmail in (…)`); exec had nothing to qualify them against, and a bare DomainModels$AttributeRef makes Mendix's loader throw. The stored model always had the full names", "file": "`mdl/executor/cmd_pages_describe_flowcontext.go` (`withQualifiedAttrs`, `describeAttr`), `mdl/executor/validate.go` (`unscopedBindings`), `mdl/executor/cmd_pages_builder_v3.go` (dangling DS flow kept by name), `mdl/backend/modelsdk/page_bare_attributeref.go` (`refuseBareAttributeRefs`)", "insight": "**The information was never lost — DESCRIBE threw it away**: every AttributeRef inside the unresolvable container still stored Module.Entity.Attr; shortening to the bare name is only safe where the reader can re-derive the entity, so key the shortening on whether the context resolved, not on habit. Measure the loader's tolerance before adding a write guard: 72 of 72 Studio Pro AttributeRefs in the project are qualified, so refusing a bare one refuses only writes that were already fatal — and turns a load-time stack trace into a statement-level error naming the attribute. Pair a blanket AST-level refusal with the exact failing slots (Attribute, CaptionAttribute, Visible-in, *Params) so the refusal names the widget, and keep the writer guard as the net for slots the walk does not know. Forced-fault run: hand-edit one qualified binding back to bare and confirm the refusal names it", "refs": ["ako/mxcli#675", "FeedbackModule.ShareFeedback_Logo"], "date": "2026-09-25"} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_New fails `mx check` with [CE0642] \"Property 'Caption' is required.\" at Combo box 'comboBox2' (Account_Edit comboBox4 too); check and exec report success", "cause": "The ComboBox caption is an EXPRESSION (optionsSourceAssociationCaptionType=expression, optionsSourceAssociationCaptionExpression='$currentObject/Description'); describe read only the attribute caption, so the widget was rewritten with none. The write side already worked via the explicit-property pass, but MDL-WIDGET06 claimed both keys 'will be dropped'", "file": "`mdl/executor/cmd_pages_describe_parse.go` (combobox branch), `cmd_pages_describe_output.go`, `validate_widget_explicit_writable.go` (`persistedByExplicitPass`), `validate_widgets.go` (MDL-WIDGET06)", "insight": "**Resolve pluggable-widget properties by key before theorising**: a 20-line script mapping each Property's TypePointer to its PropertyKey through the widget's own Type.ObjectType settled the cause in one run, where the ndsl dump shows only anonymous values. Then TEST THE WRITE SIDE before building one: adding the two storage keys to the describe output by hand persisted both and built clean, which shrank the fix to describe + a false warning — no new syntax. A validator rule that asserts 'not persisted' must be tied to what the write path handles (here: the explicit pass writes Expression/TextTemplate/Attribute and scalar types), or it goes stale when the writer grows; the #643 test pinned the stale claim. A dedicated extractor for a 'known' pluggable widget silently drops every property it does not map — generic widgets emit them, known ones do not", "refs": ["ako/mxcli#664", "ako/mxcli#643"], "ce": ["CE0642"], "rules": ["MDL-WIDGET06"], "date": "2026-09-25"} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "A `label` widget's DesignProperties were never checked: describe → check --references of FeedbackModule.ShareFeedback_Logo (Feedback v4.0.2, Atlas Core 4.1.3, Mendix 11.13.0) warned MDL-WIDGET11 'renamed' on the containers but not on label1's 'Spacing bottom': 'Outer none' (CE6087 in mxbuild). Same gap on the write side: `label l (DesignProperties: ['Style': '#ff0000'])` checked clean, exec'd, and failed mx check with [CE6085] \"Unknown option #ff0000 for design property Style.\" at Label", "cause": "The `label` keyword (PR #670, writes Forms$Label) was never added to mdlKeywordToDesignPropsKey, so resolveDesignPropsKey returned 'label' — no design-properties.json key. validateWidgetDesignProps skips a widget whose key is absent (meant for unknown pluggables), and the builder's GetPropertiesForWidget returned only the 'Widget' base group, so the Label's own ColorPicker 'Style' was unknown and a free colour fell to the Option default. bsonTypeToDesignPropsKey already had Forms$Label → Label; only the keyword half was missing", "file": "mdl/executor/theme_reader.go", "insight": "Adding a native widget keyword has a third registration nobody asks for: mdlKeywordToDesignPropsKey. Missing it is silent in both directions — the validator's 'no theme key → skip' rule turns an unmapped keyword into approval, and the builder still gets the Widget base group, so Spacing/Hide on write correctly and only the type-specific properties (Label's ColorPicker Style) mis-type. Test a type-specific property with an off-list value; a Widget-base property passes either way. Quick audit: a keyword should resolve to the same key as its stored $Type — groupbox (GroupBox, which has a ColorPicker Style), tabcontainer, navigationtree, menubar, simplemenubar, row/column (LayoutGridRow/Column) are still unmapped. Also seen: MDL-WIDGET12 warns on a ColorPicker free colour that the builder writes as Custom and mxbuild accepts (0 errors) — a pre-existing false positive, not changed here", "refs": ["#670", "#679"], "rules": ["MDL-WIDGET11", "MDL-WIDGET12"], "ce": ["CE6085", "CE6087"]} diff --git a/mdl-examples/bug-tests/label-design-properties-check.mdl b/mdl-examples/bug-tests/label-design-properties-check.mdl new file mode 100644 index 0000000000..edb103395d --- /dev/null +++ b/mdl-examples/bug-tests/label-design-properties-check.mdl @@ -0,0 +1,37 @@ +-- ============================================================================ +-- A `label`'s design properties were neither checked nor typed from the theme +-- ============================================================================ +-- +-- Symptom: describe → `check --references` of FeedbackModule.ShareFeedback_Logo +-- (Feedback v4.0.2, Atlas Core 4.1.3, Mendix 11.13.0) warned MDL-WIDGET11 on the +-- containers' renamed 'Spacing bottom' but NOT on label1's +-- 'Spacing bottom': 'Outer none', which mxbuild reports as CE6087 all the same. +-- And `label l (DesignProperties: ['Style': '#ff0000'])` checked clean, exec'd, +-- and failed the build: +-- [CE6085] "Unknown option #ff0000 for design property Style." at Label +-- +-- Cause: MDL keyword `label` (Forms$Label) had no entry in +-- mdlKeywordToDesignPropsKey, so it resolved to the theme key "label", which no +-- design-properties.json defines. The validator skipped the widget; the builder +-- saw only the "Widget" base group, so the Label's own ColorPicker "Style" was +-- unknown and a free colour fell back to an Option value. +-- +-- Fix: `label` → "Label", the key bsonTypeToDesignPropsKey already used for a +-- stored Forms$Label. +-- +-- Verify: `mxcli check --references` — the page below is clean; changing +-- lblSpaced to 'Spacing bottom': 'Outer none' must warn MDL-WIDGET11 "renamed +-- to Spacing". exec, then `mxcli docker check` — 0 errors. +-- ============================================================================ + +create or replace page MyFirstModule.LabelDesignProps_Test +( Title: 'Label design properties', Layout: Atlas_Core.Atlas_Default ) +{ + container c1 { + label lblOpt (Content: 'Option', DesignProperties: ['Style': 'Brand Primary', 'Align self': 'Left']) + label lblSpaced (Content: 'Spaced', DesignProperties: ['Spacing': ['margin-bottom': 'M'], 'Hide on': ['Phone': on]]) + } +} +/ + +describe page MyFirstModule.LabelDesignProps_Test; diff --git a/mdl/executor/label_design_properties_test.go b/mdl/executor/label_design_properties_test.go new file mode 100644 index 0000000000..30ae1339bc --- /dev/null +++ b/mdl/executor/label_design_properties_test.go @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// labelThemeRegistry is renamedThemeJSON (the Atlas Core 4.1.3 "Widget" base +// group, with its renamed Spacing steps) plus Atlas's "Label" group: one +// ColorPicker, "Style". That is the whole set Studio Pro offers a Forms$Label. +func labelThemeRegistry(t *testing.T) *ThemeRegistry { + t.Helper() + reg := renamedThemeRegistry(t) + reg.WidgetProperties["Label"] = []ThemeProperty{{ + Name: "Style", Type: "ColorPicker", + Options: []ThemeOption{{Name: "Brand Primary"}, {Name: "Brand Secondary"}}, + }} + return reg +} + +// The MDL keyword `label` writes Forms$Label, whose design properties come from +// the theme's "Label" group plus the "Widget" base. With no keyword mapping the +// validator looked up a group literally named "label", found none, and skipped +// the widget — so Feedback's ShareFeedback_Logo label1, which stores the renamed +// 'Spacing bottom': 'Outer none', went unreported while the containers beside +// it got MDL-WIDGET11 (and mxbuild reports CE6087 on all of them). +func TestValidateDesignProperties_LabelIsChecked(t *testing.T) { + reg := labelThemeRegistry(t) + vs := allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + container c1 { + label lblRenamed (Content: 'Attachment', DesignProperties: ['Spacing bottom': 'Outer none']) + label lblUndefined (Content: 'X', DesignProperties: ['Nonexistent': 'x']) + } +}`, reg) + + renamed := violationFor(vs, "Spacing bottom") + if renamed == nil { + t.Fatalf("label with an old-name design property: no violation (%d total) — the label was not checked", len(vs)) + } + if renamed.RuleID != "MDL-WIDGET11" || !strings.Contains(renamed.Message, "renamed") { + t.Errorf("want an MDL-WIDGET11 rename, got %s: %s", renamed.RuleID, renamed.Message) + } + if !strings.Contains(renamed.Suggestion, "'Spacing': ['margin-bottom': 'None']") { + t.Errorf("suggestion should give the current spelling, got: %s", renamed.Suggestion) + } + + undefined := violationFor(vs, "Nonexistent") + if undefined == nil { + t.Fatalf("label with an undefined design property: no violation (%d total)", len(vs)) + } + if undefined.RuleID != "MDL-WIDGET11" || !strings.Contains(undefined.Message, "not defined") { + t.Errorf("want MDL-WIDGET11 not defined, got %s: %s", undefined.RuleID, undefined.Message) + } +} + +// CONTROL: what Studio Pro offers a Label — its own "Style" (swatch or free +// colour) and the "Widget" base (Spacing, Align self, Hide on) — must not warn. +func TestValidateDesignProperties_LabelValidPropsPass(t *testing.T) { + reg := labelThemeRegistry(t) + vs := allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + label l1 (Content: 'A', DesignProperties: ['Style': 'Brand Primary', 'Align self': 'Left', 'Spacing': ['margin-bottom': 'M'], 'Hide on': ['Phone': on]]) +}`, reg) + if len(vs) != 0 { + for _, v := range vs { + t.Errorf("unexpected %s: %s", v.RuleID, v.Message) + } + } +} + +// The write path resolves the same key to type each value. Without it a Label's +// "Style" was unknown to the builder, so a free colour fell through to the +// "option" default and mxbuild refused the page: +// +// [CE6085] "Unknown option #ff0000 for design property Style." +// +// Measured on a copy of PedApp (Mendix 11.13.0): exec succeeded, check was silent. +func TestApplyWidgetAppearance_LabelStyleColourIsCustom(t *testing.T) { + reg := labelThemeRegistry(t) + prog, errs := visitor.Build(`create page M.P (layout: Atlas_Core.Atlas_Default) { + label l1 (Content: 'A', DesignProperties: ['Style': '#ff0000']) + label l2 (Content: 'B', DesignProperties: ['Style': 'Brand Primary']) +}`) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + page := prog.Statements[0].(*ast.CreatePageStmtV3) + want := map[string]string{"l1": "custom", "l2": "option"} + for _, w := range page.Widgets { + lbl := &pages.Label{} + if err := applyWidgetAppearance(lbl, w, reg); err != nil { + t.Fatalf("%s: %v", w.Name, err) + } + if len(lbl.DesignProperties) != 1 { + t.Fatalf("%s: %d design properties written, want 1", w.Name, len(lbl.DesignProperties)) + } + if got := lbl.DesignProperties[0].ValueType; got != want[w.Name] { + t.Errorf("%s: Style written as %q, want %q", w.Name, got, want[w.Name]) + } + } +} + +func TestResolveDesignPropsKey_Label(t *testing.T) { + if got := resolveDesignPropsKey("label"); got != "Label" { + t.Errorf(`resolveDesignPropsKey("label") = %q, want "Label" (the key Forms$Label reads)`, got) + } + if got, want := resolveDesignPropsKey("LABEL"), bsonTypeToDesignPropsKey["Forms$Label"]; got != want { + t.Errorf("keyword and stored type disagree: %q vs %q", got, want) + } +} diff --git a/mdl/executor/theme_reader.go b/mdl/executor/theme_reader.go index 55bc93d466..fb0accb882 100644 --- a/mdl/executor/theme_reader.go +++ b/mdl/executor/theme_reader.go @@ -179,12 +179,16 @@ var mdlKeywordToDesignPropsKey = map[string]string{ "layoutgrid": "LayoutGrid", "dynamictext": "DynamicText", "statictext": "Label", - "staticimage": "StaticImageViewer", - "dynamicimage": "DynamicImageViewer", - "navigationlist": "NavigationList", - "snippetcall": "SnippetCall", - "header": "Header", - "footer": "Footer", + // `label` writes Forms$Label (bsonTypeToDesignPropsKey below reads the same + // key). Unmapped, the validator skipped every label and the builder typed a + // Label's "Style" colour as an option — CE6085 at build time. + "label": "Label", + "staticimage": "StaticImageViewer", + "dynamicimage": "DynamicImageViewer", + "navigationlist": "NavigationList", + "snippetcall": "SnippetCall", + "header": "Header", + "footer": "Footer", } // pluggableKeywordIDs maps an MDL keyword to the pluggable widget id it writes, From 33fb2a95ce1624778831ae8c033a00719c160a94 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 11:14:22 +0000 Subject: [PATCH 37/47] fix(catalog): tell same-named excluded/live documents apart (#1185) Two import mappings sharing a name, one excluded, showed in the catalog as two indistinguishable rows: an AUTOINCREMENT Id, no Excluded column, and identical CATALOG.SOURCE text with no @excluded prefix. - DESCRIBE of an excluded import/export mapping prints `@excluded`, and `@excluded create ... mapping` reads it back (visitor + MDL059 table), so describe -> exec keeps the exclusion. - import_mappings / export_mappings record the document Id and Excluded. - The source build pins each describe to the document's Id (new ExecContext.describeID, pickDescribed) and stores it as ElementId, so twins get their own rows. This also fixes microflow, nanoflow, rule and page twins, which had the same identical-rows defect. - The backend's by-name mapping lookup prefers the live twin (#914). - CatalogSchemaVersion 14 (13 was in the history but the constant stayed at "12" after a merge). Verified on an 11.12.3 project with a real excluded twin (mx check 0 errors): the pre-fix binary gives Id 2/3 and identical source; the fixed one gives document ids, Excluded 0/1 and an @excluded source row. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01AcTKqypAVg5A6gDE6vYGed --- .../skills/fix-issue/findings/mdl-other.jsonl | 1 + .../1185-excluded-mapping-catalog.mdl | 64 +++++++ mdl/ast/ast_import_export_mapping.go | 2 + mdl/backend/modelsdk/mapping_write.go | 28 ++- mdl/catalog/builder.go | 5 +- mdl/catalog/builder_mappings_excluded_test.go | 143 +++++++++++++++ mdl/catalog/builder_modules.go | 16 +- mdl/catalog/builder_source.go | 35 ++-- mdl/catalog/builder_source_test.go | 22 +-- mdl/catalog/tables.go | 21 ++- mdl/executor/cmd_catalog.go | 9 +- mdl/executor/cmd_export_mappings.go | 17 +- mdl/executor/cmd_import_mappings.go | 17 +- mdl/executor/cmd_microflows_show.go | 9 +- mdl/executor/cmd_pages_describe.go | 18 +- mdl/executor/excluded_docs.go | 41 +++++ mdl/executor/exec_context.go | 7 + .../issue1185_excluded_twin_source_test.go | 167 ++++++++++++++++++ mdl/executor/validate_document_annotations.go | 7 +- .../validate_document_annotations_test.go | 1 + mdl/visitor/visitor_import_export_mapping.go | 15 ++ mdl/visitor/visitor_mapping_excluded_test.go | 42 +++++ 22 files changed, 624 insertions(+), 63 deletions(-) create mode 100644 mdl-examples/bug-tests/1185-excluded-mapping-catalog.mdl create mode 100644 mdl/catalog/builder_mappings_excluded_test.go create mode 100644 mdl/executor/issue1185_excluded_twin_source_test.go create mode 100644 mdl/visitor/visitor_mapping_excluded_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-other.jsonl b/.claude/skills/fix-issue/findings/mdl-other.jsonl index a271382b25..1508de8eee 100644 --- a/.claude/skills/fix-issue/findings/mdl-other.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-other.jsonl @@ -70,3 +70,4 @@ {"area":"mdl/executor","date":"2026-09-18","symptom":"A page parameter passed as an argument to a nanoflow/microflow BUTTON action is not wired — Studio Pro reports CE1571 \"No argument has been selected for parameter 'X' and no default is available\" on opening the page, while `mx check`, `mxcli check --references` and `mxcli lint` are all clean. Reported as an asymmetry: of two arguments, the one matching the enclosing dataview's DataSource 'works' and the other does not","cause":"Mendix stores a flow argument in one of TWO slots of Forms$MicroflowParameterMapping / Forms$NanoflowParameterMapping: a reference to a page parameter, snippet parameter or page variable goes in `Variable` as a Forms$PageVariable; a literal or expression goes in `Expression`. mxcli only ever wrote `Expression: \"$Name\"`, which binds nothing. The read side was wrong in the mirror image — the three action describers and flowSourceArgs looked for a `Name` key on that sub-document, which Forms$PageVariable does not have","file":"`sdk/pages/pages_widgets_action.go` (VariableKind on both mapping types), `mdl/executor/cmd_pages_flow_args.go` (new: classifyFlowArgValue + pageVariableArgValue), `mdl/executor/cmd_pages_builder_v3.go` (3 of the 4 copies of the $-rule), `mdl/backend/modelsdk/widget_write.go` (bindParameterMappingValue), `mdl/executor/cmd_pages_describe_output.go` + `cmd_pages_describe_datasource.go` (read)","insight":"**The reported asymmetry is a red herring — both arguments were written identically and NEITHER was bound.** Studio Pro supplies a default for the one that is the dataview's object and reports the other; 'and no default is available' in CE1571 says exactly that. Time spent on why $Dto worked is wasted. **mxbuild is not a detector here**: `mx check` on the reported project is 0 errors before AND after the fix, so the usual two-copies-of-a-real-project run proves nothing and the reporter is right that it only shows in Studio Pro. **Get the reference from a Marketplace .mpk — it contains a whole Studio Pro-authored `project.mpr`**: `mxcli marketplace download --output x.mpk && unzip -o x.mpk project.mpr`, then `mxcli bson dump` it. A blank app is useless for this (every mapping list in it is empty); Workflow Commons 4.11.0 gave 101 flow parameter mappings, of which 95 bind through Variable and 6 through Expression — and all 6 of those are Boolean literals, so the $-prefixed Expression mxcli wrote occurs ZERO times. `marketplace install` refuses that package (javasource path guard), so extract rather than install. **The PageVariable slot follows what the name refers to** (PageParameter 20, SnippetParameter 58, Widget 17) — a snippet is the COMMON case, not the corner, and `paramScope` is the right oracle because it holds only entity-typed parameters, which is the same set Mendix binds this way. **Leave $currentObject alone**: no reference for the bare form was measured and show_page already depends on the context object being inferred (MDL-PAGEARG01), so changing it on a guess risks the case that works. **The read bug hid the write bug**: describe printed `Action: microflow M.F` with no arguments for Studio Pro content, so a round-trip looked lossless and the missing binding never showed up as a diff","refs":["mendixlabs/mxcli#1140","mendixlabs/mxcli#835"],"ce":["CE1571"]} {"area": "mdl/versions", "date": "2026-09-21", "symptom": "A version gate copied from the issue text (\"Workflow Groups are GA from Mendix 11.6\") is wrong by four minors", "cause": "Mendix's release notes date the FEATURE's general availability; the metamodel floor is when the type and its property were introduced, and that is what decides whether the document loads. `Settings$WorkflowGroup` and `WorkflowsProjectSettingsPart.groups` are both `introduced: \"11.2.0\"`", "file": "`sdk/versions/mendix-11.yaml` (`workflows.groups`)", "insight": "The arbiter for a metamodel floor is the Model SDK's own StructureVersionInfo: `npm pack mendixmodelsdk && tar xzf \u2026 && grep -n '' package/src/gen/.js`, then read BOTH the class's `versionInfo.introduced` and its `properties..introduced` \u2014 a property can arrive later than its type. Release notes, proposal text and a number already written down in this repo are all downstream of it (same trap as mendixlabs/mxcli#1121). Corroborate it against two real projects rather than trusting one source: `mxcli new` at a version either side of the floor and diff the document's keys \u2014 an 11.1.0 workflows settings part has no `Groups` key at all, an 11.13.0 one carries `Groups: [2]`, which also proves the refusal is right rather than over-cautious (writing the property below the floor would be inventing a key). mendixlabs/mxcli#272", "refs": ["mendixlabs/mxcli#272"]} {"area": "mdl/linter/rules", "date": "2026-09-23", "symptom": "CONV010 flagged an ACT_ nanoflow that delegated to a sub-flow \u2014 the very thing the rule demands. A real project patched its own copy of the rule and asked for the fix upstream. An ACT_ nanoflow could satisfy CONV010 in NO way: delegate and be flagged, or inline the logic and be flagged.", "cause": "ALLOWED_ACTIONS held `MicroflowCallAction` but not `NanoflowCallAction`. `microflows()` yields nanoflows too \u2014 the catalog's `microflows` table carries a MicroflowType column \u2014 so CONV010 lints ACT_ nanoflows, and a nanoflow delegates with a nanoflow call.", "file": "`.claude/lint-rules/conv010_act_microflow_content.star` (NanoflowCallAction added to ALLOWED_ACTIONS; cmd/mxcli/lint-rules/ is gitignored and regenerated by `make sync-lint-rules`), test `mdl/catalog/lint_rule_vocabulary_test.go` (added to the `permitted` list)", "insight": "Third time this one allowlist has been short, and the rule's own comments record the previous two: the wrong vocabulary entirely (storage names vs SDK names, matching nothing, 11 false positives of 13 findings) and a missing ExclusiveMerge that a permitted ExclusiveSplit necessarily creates (122 hits on one project). The recurring shape is an UNSATISFIABLE rule, and its cost is asymmetric: a rule that cannot be satisfied does not read as a broken rule, it reads as broken CODE, so users refactor around it or patch the rule locally and the defect never comes back upstream \u2014 which is exactly what happened here until someone wrote 'report upstream' in their findings. A vocabulary pin test (TestCONV010AllowsWhatTheCatalogCallsUIActions) already existed to stop this class and did not, because its `permitted` list is hand-maintained and was itself incomplete: pinning a rule to a hand-written list of what SHOULD be allowed only moves the completeness problem. Worth considering: enumerate the delegation actions from the type system rather than listing them.", "refs": ["ako/mxcli#644"]} +{"area": "mdl/catalog", "date": "2026-09-25", "symptom": "Two import mappings with the same name, one excluded and one live (valid: mx check 0 errors), show up in CATALOG.IMPORT_MAPPINGS and CATALOG.SOURCE as two indistinguishable rows — integer Ids 2 and 3, no Excluded column, and both source rows the same `create or modify import mapping …` text with no `@excluded` prefix (mendixlabs/mxcli#1185)", "cause": "Three gaps: describeImportMapping/describeExportMapping never printed @excluded; import_mappings_data/export_mappings_data used an AUTOINCREMENT Id and recorded no Excluded; and buildSource enumerates DOCUMENTS but called the describe callback with only a qualified name, so every describer resolved by name and rendered the preferred twin for both rows. The last one was not mapping-specific — microflow/nanoflow/rule/page twins had identical source rows too, masked because the microflows table itself carries Id + Excluded", "file": "`mdl/catalog/builder_source.go` (`sourceItem.id`, `ElementId`), `mdl/catalog/builder.go` (`DescribeFunc` gains id), `mdl/catalog/tables.go` + `builder_modules.go` (mapping Id/Excluded, schema 14), `mdl/executor/excluded_docs.go` (`pickDescribed`, `describedMapping`), `mdl/executor/cmd_catalog.go` (`ExecContext.describeID`), `mdl/executor/cmd_{import,export}_mappings.go`, `mdl/visitor/visitor_import_export_mapping.go`, `mdl/backend/modelsdk/mapping_write.go` (by-name lookup prefers the live twin)", "insight": "**A collector that walks documents must not hand a NAME to a callback that resolves names** — #914's pickLive is correct for an interactive DESCRIBE and exactly wrong for a per-document sweep, because it maps both twins onto one. Pin by ID (ExecContext.describeID) and fall back to pickLive. **Printing a new annotation in DESCRIBE obliges the write side**: `@excluded` on a mapping was MDL059 until the visitor read it and documentAnnotations listed it; skipping that turns describe->exec into a refusal. **Check CatalogSchemaVersion against its own history comment**: the history recorded 13 (entity event handlers) while the constant said \"12\" — parallel bumps, the merge kept the lower — so caches built at 12 never rebuilt; bumping to 14 carries both. **Making a twin for a real run**: MDL cannot (CREATE refuses a taken name, no RENAME for mappings); `@excluded create` under a second name, then rewrite that unit's BSON Name (bson.D round-trip of the .mxunit) — mx check stays 0 errors, which is the reporter's state. Pre-fix binary on that project: two rows Id 2/3 and identical source text; fixed: document Ids, Excluded 0/1, excluded row starts `@excluded`. Controls: disable the describeID pin (4 executor tests fail, microflow included), drop the @excluded print, pass \"\" as id in buildSource (1 source row, not 2), stash tables.go/builder_modules.go (no such column: Excluded)", "refs": ["mendixlabs/mxcli#1185", "#914"]} diff --git a/mdl-examples/bug-tests/1185-excluded-mapping-catalog.mdl b/mdl-examples/bug-tests/1185-excluded-mapping-catalog.mdl new file mode 100644 index 0000000000..81585810c9 --- /dev/null +++ b/mdl-examples/bug-tests/1185-excluded-mapping-catalog.mdl @@ -0,0 +1,64 @@ +-- Bug test for upstream issue #1185: "documents with same [name] not +-- discernable in catalog" (mxcli 0.24, mx 11.12.3). +-- +-- Reported with "two import mappings with the same name, one included and one +-- excluded": the catalog showed both, and "the excluded mapping source doesn't +-- follow the established @excluded prefix convention". +-- +-- Fixed: +-- * DESCRIBE of an excluded import/export mapping starts with `@excluded`, +-- and `@excluded create … mapping` reads it back (MDL059 no longer +-- rejects it), so describe -> exec keeps the exclusion. +-- * CATALOG.IMPORT_MAPPINGS / EXPORT_MAPPINGS carry the document's own Id and +-- an Excluded column. +-- * CATALOG.SOURCE describes each document by its Id (column ElementId), so +-- same-named twins get their own rows instead of the live one twice. +-- +-- MDL cannot create a same-named twin (a CREATE without OR MODIFY refuses an +-- existing name), so this script covers the annotation round trip. To see the +-- catalog side, rename IMM_1185_Old's stored Name to IMM_1185, then: +-- +-- refresh catalog full source; +-- select Id, Excluded from catalog.import_mappings where Name = 'IMM_1185'; +-- select ElementId, SourceText from catalog.source +-- where QualifiedName = 'MyFirstModule.IMM_1185'; +-- +-- Expected: two distinct Ids, Excluded 0 and 1; the excluded row's source +-- starts with `@excluded`. `mx check` reports 0 errors (an excluded document is +-- not built). + +create json structure MyFirstModule.JSON_1185 +snippet '{"id": 1, "name": "x"}'; + +create non-persistent entity MyFirstModule.Order1185 ( OrderId: integer, Name: string(200) ); +/ + +create import mapping MyFirstModule.IMM_1185 + with json structure MyFirstModule.JSON_1185 +{ + create MyFirstModule.Order1185 { + OrderId = id, + Name = name + } +}; + +@excluded +create import mapping MyFirstModule.IMM_1185_Old + with json structure MyFirstModule.JSON_1185 +{ + create MyFirstModule.Order1185 { + OrderId = id + } +}; + +@excluded +create export mapping MyFirstModule.EXM_1185_Old + with json structure MyFirstModule.JSON_1185 +{ + MyFirstModule.Order1185 { + id = OrderId + } +}; + +describe import mapping MyFirstModule.IMM_1185_Old; +describe export mapping MyFirstModule.EXM_1185_Old; diff --git a/mdl/ast/ast_import_export_mapping.go b/mdl/ast/ast_import_export_mapping.go index 976376b632..0f964b1d75 100644 --- a/mdl/ast/ast_import_export_mapping.go +++ b/mdl/ast/ast_import_export_mapping.go @@ -30,6 +30,7 @@ type CreateImportMappingStmt struct { Parameter QualifiedName RootElement *ImportMappingElementDef CreateOrModify bool // true for CREATE OR MODIFY / CREATE OR REPLACE + Excluded bool // @excluded — document excluded from project } func (s *CreateImportMappingStmt) isStatement() {} @@ -108,6 +109,7 @@ type CreateExportMappingStmt struct { NullValueOption string // "LeaveOutElement" or "SendAsNil" (default: "LeaveOutElement") RootElement *ExportMappingElementDef CreateOrModify bool // true for CREATE OR MODIFY / CREATE OR REPLACE + Excluded bool // @excluded — document excluded from project } func (s *CreateExportMappingStmt) isStatement() {} diff --git a/mdl/backend/modelsdk/mapping_write.go b/mdl/backend/modelsdk/mapping_write.go index df1fb81b49..f59d1e06f8 100644 --- a/mdl/backend/modelsdk/mapping_write.go +++ b/mdl/backend/modelsdk/mapping_write.go @@ -105,11 +105,23 @@ func (b *Backend) GetImportMappingByQualifiedName(moduleName, name string) (*mod if err != nil { return nil, err } + // A name is not a unique key: a module may hold an excluded twin (#914). + // Prefer the live one, so DESCRIBE shows what the app runs and CREATE OR + // MODIFY edits it; fall back to the first excluded match (#1185). + var excluded *model.ImportMapping for _, im := range all { if im.Name == name && b.moduleNameFor(im.ID) == moduleName { - return im, nil + if !im.Excluded { + return im, nil + } + if excluded == nil { + excluded = im + } } } + if excluded != nil { + return excluded, nil + } return nil, fmt.Errorf("import mapping not found: %s.%s", moduleName, name) } @@ -300,11 +312,23 @@ func (b *Backend) GetExportMappingByQualifiedName(moduleName, name string) (*mod if err != nil { return nil, err } + // A name is not a unique key: a module may hold an excluded twin (#914). + // Prefer the live one, so DESCRIBE shows what the app runs and CREATE OR + // MODIFY edits it; fall back to the first excluded match (#1185). + var excluded *model.ExportMapping for _, em := range all { if em.Name == name && b.moduleNameFor(em.ID) == moduleName { - return em, nil + if !em.Excluded { + return em, nil + } + if excluded == nil { + excluded = em + } } } + if excluded != nil { + return excluded, nil + } return nil, fmt.Errorf("export mapping not found: %s.%s", moduleName, name) } diff --git a/mdl/catalog/builder.go b/mdl/catalog/builder.go index b43601664e..7e1ee6f6f6 100644 --- a/mdl/catalog/builder.go +++ b/mdl/catalog/builder.go @@ -77,7 +77,10 @@ type CatalogReader interface { } // DescribeFunc generates MDL source for a given object type and qualified name. -type DescribeFunc func(objectType string, qualifiedName string) (string, error) +// id is the stored document's ID and pins the describe to it: two documents +// may share a qualified name when one is excluded (#914), and describing by +// name alone renders the live one for both (#1185). +type DescribeFunc func(objectType string, qualifiedName string, id string) (string, error) // Builder populates catalog tables from MPR data. type Builder struct { diff --git a/mdl/catalog/builder_mappings_excluded_test.go b/mdl/catalog/builder_mappings_excluded_test.go new file mode 100644 index 0000000000..26b7019703 --- /dev/null +++ b/mdl/catalog/builder_mappings_excluded_test.go @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: Apache-2.0 + +package catalog + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" +) + +// mendixlabs/mxcli#1185: "two import mappings with the same name, one included +// and one excluded" — the catalog listed both and nothing told them apart. +// import_mappings / export_mappings had no Excluded column and an AUTOINCREMENT +// Id rather than the document's, and CATALOG.SOURCE rendered both twins by name, +// i.e. as the same document. + +const twinModuleID = model.ID("mod-integration") + +func twinBuilder(t *testing.T, reader *mock.MockBackend, describe DescribeFunc) (*Builder, *Catalog) { + t.Helper() + cat, err := New() + if err != nil { + t.Fatalf("new catalog: %v", err) + } + t.Cleanup(func() { cat.Close() }) + tx, err := cat.db.Begin() + if err != nil { + t.Fatalf("begin: %v", err) + } + b := &Builder{ + catalog: cat, + reader: reader, + snapshot: &Snapshot{ID: "snap-1"}, + hierarchy: &hierarchy{ + moduleIDs: map[model.ID]bool{twinModuleID: true}, + moduleNames: map[model.ID]string{twinModuleID: "Integration"}, + containerParent: map[model.ID]model.ID{}, + folderNames: map[model.ID]string{}, + }, + tx: tx, + sourceMode: describe != nil, + describeFunc: describe, + } + return b, cat +} + +func twinMappingsReader() *mock.MockBackend { + return &mock.MockBackend{ + ListImportMappingsFunc: func() ([]*model.ImportMapping, error) { + return []*model.ImportMapping{ + {BaseElement: model.BaseElement{ID: "im-excluded"}, ContainerID: twinModuleID, Name: "IMM_Order", Excluded: true}, + {BaseElement: model.BaseElement{ID: "im-live"}, ContainerID: twinModuleID, Name: "IMM_Order"}, + }, nil + }, + ListExportMappingsFunc: func() ([]*model.ExportMapping, error) { + return []*model.ExportMapping{ + {BaseElement: model.BaseElement{ID: "em-excluded"}, ContainerID: twinModuleID, Name: "EXM_Order", Excluded: true}, + {BaseElement: model.BaseElement{ID: "em-live"}, ContainerID: twinModuleID, Name: "EXM_Order"}, + }, nil + }, + } +} + +func TestMappingTables_RecordDocumentIdAndExcluded(t *testing.T) { + b, cat := twinBuilder(t, twinMappingsReader(), nil) + if err := b.buildImportMappings(); err != nil { + t.Fatalf("buildImportMappings: %v", err) + } + if err := b.buildExportMappings(); err != nil { + t.Fatalf("buildExportMappings: %v", err) + } + if err := b.tx.Commit(); err != nil { + t.Fatalf("commit: %v", err) + } + + for _, tc := range []struct{ table, liveID, excludedID string }{ + {"import_mappings", "im-live", "im-excluded"}, + {"export_mappings", "em-live", "em-excluded"}, + } { + rows, err := cat.db.Query("SELECT Id, Excluded FROM " + tc.table + " ORDER BY Id") + if err != nil { + t.Fatalf("%s: %v", tc.table, err) + } + got := map[string]bool{} + for rows.Next() { + var id string + var excluded bool + if err := rows.Scan(&id, &excluded); err != nil { + t.Fatalf("%s scan: %v", tc.table, err) + } + got[id] = excluded + } + rows.Close() + if len(got) != 2 { + t.Fatalf("%s: got %d distinct ids %v, want the two document ids", tc.table, len(got), got) + } + if ex, ok := got[tc.excludedID]; !ok || !ex { + t.Errorf("%s: the excluded twin must be recorded under its document id with Excluded=1; got %v", tc.table, got) + } + if ex, ok := got[tc.liveID]; !ok || ex { + t.Errorf("%s: the live twin must be recorded under its document id with Excluded=0; got %v", tc.table, got) + } + } +} + +func TestBuildSource_PinsEachTwinByDocumentId(t *testing.T) { + var asked []string + describe := func(objType, qn, id string) (string, error) { + asked = append(asked, id) + if id == "im-excluded" || id == "em-excluded" { + return "@excluded\ncreate or modify " + qn, nil + } + return "create or modify " + qn, nil + } + b, cat := twinBuilder(t, twinMappingsReader(), describe) + if err := b.buildSource(); err != nil { + t.Fatalf("buildSource: %v", err) + } + if err := b.tx.Commit(); err != nil { + t.Fatalf("commit: %v", err) + } + + rows, err := cat.db.Query(`SELECT ElementId, SourceText FROM source WHERE ObjectType = ? ORDER BY ElementId`, SourceImportMapping) + if err != nil { + t.Fatalf("query source: %v", err) + } + defer rows.Close() + got := map[string]string{} + for rows.Next() { + var id, text string + if err := rows.Scan(&id, &text); err != nil { + t.Fatalf("scan: %v", err) + } + got[id] = text + } + if len(got) != 2 { + t.Fatalf("got %d source rows keyed by ElementId %v, want one per twin (describes asked for %v)", len(got), got, asked) + } + if got["im-excluded"] == got["im-live"] { + t.Errorf("the two twins' source rows are identical:\n%s", got["im-live"]) + } +} diff --git a/mdl/catalog/builder_modules.go b/mdl/catalog/builder_modules.go index a9b13d1107..b648c55228 100644 --- a/mdl/catalog/builder_modules.go +++ b/mdl/catalog/builder_modules.go @@ -866,10 +866,10 @@ func (b *Builder) buildImportMappings() error { } stmt, err := b.tx.Prepare(` - INSERT INTO import_mappings_data (Name, QualifiedName, ModuleName, - SchemaSource, ElementCount, Documentation, Folder, + INSERT INTO import_mappings_data (Id, Name, QualifiedName, ModuleName, + SchemaSource, ElementCount, Documentation, Folder, Excluded, ProjectId, SnapshotId) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) `) if err != nil { return err @@ -893,6 +893,7 @@ func (b *Builder) buildImportMappings() error { } _, err := stmt.Exec( + string(im.ID), im.Name, qualifiedName, moduleName, @@ -900,6 +901,7 @@ func (b *Builder) buildImportMappings() error { len(im.Elements), im.Documentation, folderPath, + im.Excluded, projectID, snapshotID, ) if err != nil { @@ -918,10 +920,10 @@ func (b *Builder) buildExportMappings() error { } stmt, err := b.tx.Prepare(` - INSERT INTO export_mappings_data (Name, QualifiedName, ModuleName, - SchemaSource, NullValueOption, ElementCount, Documentation, Folder, + INSERT INTO export_mappings_data (Id, Name, QualifiedName, ModuleName, + SchemaSource, NullValueOption, ElementCount, Documentation, Folder, Excluded, ProjectId, SnapshotId) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) `) if err != nil { return err @@ -945,6 +947,7 @@ func (b *Builder) buildExportMappings() error { } _, err := stmt.Exec( + string(em.ID), em.Name, qualifiedName, moduleName, @@ -953,6 +956,7 @@ func (b *Builder) buildExportMappings() error { len(em.Elements), em.Documentation, folderPath, + em.Excluded, projectID, snapshotID, ) if err != nil { diff --git a/mdl/catalog/builder_source.go b/mdl/catalog/builder_source.go index ba047dda37..1edbcbd0b3 100644 --- a/mdl/catalog/builder_source.go +++ b/mdl/catalog/builder_source.go @@ -51,6 +51,11 @@ type sourceItem struct { objType string qn string moduleName string + // id is the stored document's ID. A name is not a unique key — a module + // may hold an excluded twin of a live document — so the describe is pinned + // to it, and it is written beside the text so two same-named rows stay + // distinguishable (#1185). + id string } // sourceResult holds the output of a parallel describe call. @@ -89,7 +94,7 @@ func (b *Builder) buildSource() error { moduleID := b.hierarchy.findModuleID(dm.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) for _, ent := range dm.Entities { - items = append(items, sourceItem{SourceEntity, moduleName + "." + ent.Name, moduleName}) + items = append(items, sourceItem{SourceEntity, moduleName + "." + ent.Name, moduleName, string(ent.ID)}) } } } @@ -100,7 +105,7 @@ func (b *Builder) buildSource() error { for _, mf := range mfList { moduleID := b.hierarchy.findModuleID(mf.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourceMicroflow, moduleName + "." + mf.Name, moduleName}) + items = append(items, sourceItem{SourceMicroflow, moduleName + "." + mf.Name, moduleName, string(mf.ID)}) } } @@ -110,7 +115,7 @@ func (b *Builder) buildSource() error { for _, nf := range nfList { moduleID := b.hierarchy.findModuleID(nf.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourceNanoflow, moduleName + "." + nf.Name, moduleName}) + items = append(items, sourceItem{SourceNanoflow, moduleName + "." + nf.Name, moduleName, string(nf.ID)}) } } @@ -121,7 +126,7 @@ func (b *Builder) buildSource() error { for _, rule := range ruleList { moduleID := b.hierarchy.findModuleID(rule.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourceRule, moduleName + "." + rule.Name, moduleName}) + items = append(items, sourceItem{SourceRule, moduleName + "." + rule.Name, moduleName, string(rule.ID)}) } } @@ -131,7 +136,7 @@ func (b *Builder) buildSource() error { for _, pg := range pageList { moduleID := b.hierarchy.findModuleID(pg.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourcePage, moduleName + "." + pg.Name, moduleName}) + items = append(items, sourceItem{SourcePage, moduleName + "." + pg.Name, moduleName, string(pg.ID)}) } } @@ -140,7 +145,7 @@ func (b *Builder) buildSource() error { for _, sn := range snippetList { moduleID := b.hierarchy.findModuleID(sn.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourceSnippet, moduleName + "." + sn.Name, moduleName}) + items = append(items, sourceItem{SourceSnippet, moduleName + "." + sn.Name, moduleName, string(sn.ID)}) } // Workflows @@ -149,7 +154,7 @@ func (b *Builder) buildSource() error { for _, wf := range wfList { moduleID := b.hierarchy.findModuleID(wf.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourceWorkflow, moduleName + "." + wf.Name, moduleName}) + items = append(items, sourceItem{SourceWorkflow, moduleName + "." + wf.Name, moduleName, string(wf.ID)}) } } @@ -159,7 +164,7 @@ func (b *Builder) buildSource() error { for _, en := range enumList { moduleID := b.hierarchy.findModuleID(en.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourceEnumeration, moduleName + "." + en.Name, moduleName}) + items = append(items, sourceItem{SourceEnumeration, moduleName + "." + en.Name, moduleName, string(en.ID)}) } } @@ -172,7 +177,7 @@ func (b *Builder) buildSource() error { for _, js := range jsonList { moduleID := b.hierarchy.findModuleID(js.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourceJsonStructure, moduleName + "." + js.Name, moduleName}) + items = append(items, sourceItem{SourceJsonStructure, moduleName + "." + js.Name, moduleName, string(js.ID)}) } } @@ -181,7 +186,7 @@ func (b *Builder) buildSource() error { for _, im := range importList { moduleID := b.hierarchy.findModuleID(im.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourceImportMapping, moduleName + "." + im.Name, moduleName}) + items = append(items, sourceItem{SourceImportMapping, moduleName + "." + im.Name, moduleName, string(im.ID)}) } } @@ -190,7 +195,7 @@ func (b *Builder) buildSource() error { for _, em := range exportList { moduleID := b.hierarchy.findModuleID(em.ContainerID) moduleName := b.hierarchy.getModuleName(moduleID) - items = append(items, sourceItem{SourceExportMapping, moduleName + "." + em.Name, moduleName}) + items = append(items, sourceItem{SourceExportMapping, moduleName + "." + em.Name, moduleName, string(em.ID)}) } } @@ -227,8 +232,8 @@ func (b *Builder) buildSource() error { // Phase 3: Insert results into FTS5 table (serial — SQLite constraint) stmt, err := b.tx.Prepare(` - INSERT INTO source (QualifiedName, ObjectType, SourceText, ModuleName) - VALUES (?, ?, ?, ?) + INSERT INTO source (QualifiedName, ObjectType, SourceText, ModuleName, ElementId) + VALUES (?, ?, ?, ?, ?) `) if err != nil { return err @@ -240,7 +245,7 @@ func (b *Builder) buildSource() error { if res.text == "" { continue } - stmt.Exec(res.item.qn, res.item.objType, res.text, res.item.moduleName) + stmt.Exec(res.item.qn, res.item.objType, res.text, res.item.moduleName, res.item.id) count++ } @@ -300,7 +305,7 @@ func runDescribes(items []sourceItem, describe DescribeFunc, workers int, onProg wg.Go(func() { for idx := range work { item := items[idx] - text, err := describe(item.objType, item.qn) + text, err := describe(item.objType, item.qn, item.id) switch { case err != nil: failed[idx] = &describeFailure{item, err.Error()} diff --git a/mdl/catalog/builder_source_test.go b/mdl/catalog/builder_source_test.go index 010307044b..25c3b4c963 100644 --- a/mdl/catalog/builder_source_test.go +++ b/mdl/catalog/builder_source_test.go @@ -21,15 +21,15 @@ import ( // runDescribes must therefore return what failed, so the caller can report it. func TestRunDescribes_ReturnsFailuresInsteadOfSwallowingThem(t *testing.T) { items := []sourceItem{ - {SourceEntity, "Mod.Customer", "Mod"}, - {SourceNanoflow, "Mod.ACT_Save", "Mod"}, - {SourceRule, "Mod.RL_IsActive", "Mod"}, - {SourceMicroflow, "Mod.IVK_Save", "Mod"}, + {SourceEntity, "Mod.Customer", "Mod", ""}, + {SourceNanoflow, "Mod.ACT_Save", "Mod", ""}, + {SourceRule, "Mod.RL_IsActive", "Mod", ""}, + {SourceMicroflow, "Mod.IVK_Save", "Mod", ""}, } // Stands in for the executor dispatch before the fix: nanoflows and rules // were unreachable, everything else described fine. - describe := func(objType, qn string) (string, error) { + describe := func(objType, qn, _ string) (string, error) { switch objType { case SourceNanoflow: return "", errors.New("nanoflow not found: " + qn) @@ -74,9 +74,9 @@ func TestRunDescribes_ReturnsFailuresInsteadOfSwallowingThem(t *testing.T) { // A describe that returns no error but also no text still produces no row. That // is the same silent drop wearing a different mask, so it counts as a failure. func TestRunDescribes_EmptyOutputCountsAsFailure(t *testing.T) { - items := []sourceItem{{SourcePage, "Mod.Home", "Mod"}} + items := []sourceItem{{SourcePage, "Mod.Home", "Mod", ""}} - _, failures := runDescribes(items, func(string, string) (string, error) { + _, failures := runDescribes(items, func(string, string, string) (string, error) { return "", nil }, 1, nil) @@ -90,10 +90,10 @@ func TestRunDescribes_EmptyOutputCountsAsFailure(t *testing.T) { func TestRunDescribes_AllSucceeding(t *testing.T) { items := []sourceItem{ - {SourceEntity, "Mod.A", "Mod"}, - {SourceEntity, "Mod.B", "Mod"}, + {SourceEntity, "Mod.A", "Mod", ""}, + {SourceEntity, "Mod.B", "Mod", ""}, } - results, failures := runDescribes(items, func(_, qn string) (string, error) { + results, failures := runDescribes(items, func(_, qn, _ string) (string, error) { return "create entity " + qn + ";", nil }, 4, nil) @@ -108,7 +108,7 @@ func TestRunDescribes_AllSucceeding(t *testing.T) { } func TestRunDescribes_NoItems(t *testing.T) { - results, failures := runDescribes(nil, func(string, string) (string, error) { + results, failures := runDescribes(nil, func(string, string, string) (string, error) { t.Fatal("describe called with no items") return "", nil }, 4, nil) diff --git a/mdl/catalog/tables.go b/mdl/catalog/tables.go index 84aca3aee0..231eb9f5d7 100644 --- a/mdl/catalog/tables.go +++ b/mdl/catalog/tables.go @@ -7,6 +7,12 @@ package catalog // // History: // +// 14 — import_mappings_data / export_mappings_data: Id is the document's ID +// (was an AUTOINCREMENT integer) and Excluded is recorded; source gains +// ElementId. Two mappings may share a name when one is excluded, and the +// catalog could not tell them apart (mendixlabs/mxcli#1185). 13 and 12 +// were bumped on parallel branches and the merge kept "12", so caches +// built at 12 never rebuilt for 13; this bump carries both. // 13 — entity_event_handlers_data + view, and the `event` edge in refs // (ENTITY -> MICROFLOW). Same reason as 11: refs are only written by // REFRESH CATALOG FULL, so without the bump a cached catalog keeps @@ -40,7 +46,7 @@ package catalog // SnapshotSource / SourceId / SourceBranch / SourceRevision columns // from every row (issue #576). // 1 — initial flat schema with denormalized snapshot columns on every row. -const CatalogSchemaVersion = "12" +const CatalogSchemaVersion = "14" // MetaSchemaVersion is the catalog_meta key that records the schema version // the cache was built against. @@ -1074,7 +1080,7 @@ func (c *Catalog) createTables() error { // import_mappings `CREATE TABLE IF NOT EXISTS import_mappings_data ( - Id INTEGER PRIMARY KEY AUTOINCREMENT, + Id TEXT PRIMARY KEY, Name TEXT NOT NULL, QualifiedName TEXT NOT NULL, ModuleName TEXT NOT NULL, @@ -1082,6 +1088,7 @@ func (c *Catalog) createTables() error { ElementCount INTEGER DEFAULT 0, Documentation TEXT, Folder TEXT, + Excluded BOOLEAN DEFAULT 0, ProjectId TEXT, SnapshotId TEXT )`, @@ -1089,7 +1096,7 @@ func (c *Catalog) createTables() error { // export_mappings `CREATE TABLE IF NOT EXISTS export_mappings_data ( - Id INTEGER PRIMARY KEY AUTOINCREMENT, + Id TEXT PRIMARY KEY, Name TEXT NOT NULL, QualifiedName TEXT NOT NULL, ModuleName TEXT NOT NULL, @@ -1098,6 +1105,7 @@ func (c *Catalog) createTables() error { ElementCount INTEGER DEFAULT 0, Documentation TEXT, Folder TEXT, + Excluded BOOLEAN DEFAULT 0, ProjectId TEXT, SnapshotId TEXT )`, @@ -1267,11 +1275,11 @@ func (c *Catalog) createTables() error { ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource FROM json_structures UNION ALL - SELECT CAST(Id AS TEXT), 'IMPORT_MAPPING' as ObjectType, Name, QualifiedName, ModuleName, Folder, Documentation as Description, + SELECT Id, 'IMPORT_MAPPING' as ObjectType, Name, QualifiedName, ModuleName, Folder, Documentation as Description, ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource FROM import_mappings UNION ALL - SELECT CAST(Id AS TEXT), 'EXPORT_MAPPING' as ObjectType, Name, QualifiedName, ModuleName, Folder, Documentation as Description, + SELECT Id, 'EXPORT_MAPPING' as ObjectType, Name, QualifiedName, ModuleName, Folder, Documentation as Description, ProjectId, ProjectName, SnapshotId, SnapshotDate, SnapshotSource FROM export_mappings UNION ALL @@ -1294,7 +1302,8 @@ func (c *Catalog) createTables() error { QualifiedName, ObjectType, SourceText, - ModuleName + ModuleName, + ElementId UNINDEXED )`, // Indexes for common queries — target the underlying *_data tables. diff --git a/mdl/executor/cmd_catalog.go b/mdl/executor/cmd_catalog.go index ce14c16a28..6d9be12561 100644 --- a/mdl/executor/cmd_catalog.go +++ b/mdl/executor/cmd_catalog.go @@ -510,8 +510,8 @@ func buildCatalog(ctx *ExecContext, full, isSource, communities bool, resolution if isSource { builder.SetSourceMode(true) preWarmCache(ctx) - builder.SetDescribeFunc(func(objectType string, qualifiedName string) (string, error) { - return captureDescribeParallel(ctx, objectType, qualifiedName) + builder.SetDescribeFunc(func(objectType string, qualifiedName string, id string) (string, error) { + return captureDescribeParallel(ctx, objectType, qualifiedName, model.ID(id)) }) } // Supply built-in widget definitions (hand-crafted COMBOBOX, GALLERY, @@ -896,7 +896,7 @@ func captureDescribe(ctx *ExecContext, objectType string, qualifiedName string) // It creates a lightweight ExecContext clone per call with its own output buffer, // sharing the backend and pre-warmed cache. Call preWarmCache() before using // this from multiple goroutines. -func captureDescribeParallel(ctx *ExecContext, objectType string, qualifiedName string) (string, error) { +func captureDescribeParallel(ctx *ExecContext, objectType string, qualifiedName string, id model.ID) (string, error) { parts := strings.SplitN(qualifiedName, ".", 2) if len(parts) != 2 { return "", mdlerrors.NewValidationf("invalid qualified name: %s", qualifiedName) @@ -914,6 +914,9 @@ func captureDescribeParallel(ctx *ExecContext, objectType string, qualifiedName Backend: ctx.Backend, Cache: ctx.Cache, MprPath: ctx.MprPath, + // Pins the describe to this document rather than to whichever + // document of this name the lookup prefers (#1185). + describeID: id, } describe := describeDispatch(objectType) diff --git a/mdl/executor/cmd_export_mappings.go b/mdl/executor/cmd_export_mappings.go index a1e172fa9c..eebdd07363 100644 --- a/mdl/executor/cmd_export_mappings.go +++ b/mdl/executor/cmd_export_mappings.go @@ -84,7 +84,9 @@ func describeExportMapping(ctx *ExecContext, name ast.QualifiedName) error { return mdlerrors.NewNotConnected() } - em, err := ctx.Backend.GetExportMappingByQualifiedName(name.Module, name.Name) + em, err := describedMapping(ctx, name, + ctx.Backend.GetExportMappingByQualifiedName, ctx.Backend.ListExportMappings, + func(m *model.ExportMapping) model.ID { return m.ID }) if err != nil { if strings.Contains(err.Error(), "not found") { return mdlerrors.NewNotFound("export mapping", name.String()) @@ -95,6 +97,12 @@ func describeExportMapping(ctx *ExecContext, name ast.QualifiedName) error { if em.Documentation != "" { fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", strings.ReplaceAll(em.Documentation, "\n", "\n * ")) } + // The same prefix a microflow or page carries: without it an excluded + // mapping reads as live, and beside a live twin of the same name the two + // are indistinguishable (#1185). + if em.Excluded { + fmt.Fprintln(ctx.Output, "@excluded") + } h, err := getHierarchy(ctx) if err != nil { @@ -317,10 +325,13 @@ func execCreateExportMapping(ctx *ExecContext, s *ast.CreateExportMappingStmt) e Name: s.Name.Name, ExportLevel: "Hidden", NullValueOption: nullValueOption, + // @excluded, which DESCRIBE prints for an excluded mapping (#1185). + Excluded: s.Excluded, } if existing != nil { - // Excluded is model state, not script state (#914). - em.Excluded = existing.Excluded + // Excluded is model state, not script state: an absent @excluded must + // not clear a stored exclusion (#914). + em.Excluded = s.Excluded || existing.Excluded // MessageDefinition2 is version-introduced (11.10+) and CARRIED, never // invented: a document written before then does not have the key, and // adding one is the overlay-rule mistake — mxbuild tolerates it, Studio diff --git a/mdl/executor/cmd_import_mappings.go b/mdl/executor/cmd_import_mappings.go index 5644c7ebf9..9860d3da0b 100644 --- a/mdl/executor/cmd_import_mappings.go +++ b/mdl/executor/cmd_import_mappings.go @@ -84,7 +84,9 @@ func describeImportMapping(ctx *ExecContext, name ast.QualifiedName) error { return mdlerrors.NewNotConnected() } - im, err := ctx.Backend.GetImportMappingByQualifiedName(name.Module, name.Name) + im, err := describedMapping(ctx, name, + ctx.Backend.GetImportMappingByQualifiedName, ctx.Backend.ListImportMappings, + func(m *model.ImportMapping) model.ID { return m.ID }) if err != nil { if strings.Contains(err.Error(), "not found") { return mdlerrors.NewNotFound("import mapping", name.String()) @@ -95,6 +97,12 @@ func describeImportMapping(ctx *ExecContext, name ast.QualifiedName) error { if im.Documentation != "" { fmt.Fprintf(ctx.Output, "/**\n * %s\n */\n", strings.ReplaceAll(im.Documentation, "\n", "\n * ")) } + // The same prefix a microflow or page carries: without it an excluded + // mapping reads as live, and beside a live twin of the same name the two + // are indistinguishable (#1185). + if im.Excluded { + fmt.Fprintln(ctx.Output, "@excluded") + } h, err := getHierarchy(ctx) if err != nil { @@ -410,10 +418,13 @@ func execCreateImportMapping(ctx *ExecContext, s *ast.CreateImportMappingStmt) e ContainerID: containerID, Name: s.Name.Name, ExportLevel: "Hidden", + // @excluded, which DESCRIBE prints for an excluded mapping (#1185). + Excluded: s.Excluded, } if existing != nil { - // Excluded is model state, not script state (#914). - im.Excluded = existing.Excluded + // Excluded is model state, not script state: an absent @excluded must + // not clear a stored exclusion (#914). + im.Excluded = s.Excluded || existing.Excluded // MessageDefinition2 is version-introduced (11.10+) and CARRIED, never // invented: a document written before then does not have the key, and // adding one is the overlay-rule mistake — mxbuild tolerates it, Studio diff --git a/mdl/executor/cmd_microflows_show.go b/mdl/executor/cmd_microflows_show.go index b82ae6f0b8..bbbe13a01b 100644 --- a/mdl/executor/cmd_microflows_show.go +++ b/mdl/executor/cmd_microflows_show.go @@ -228,7 +228,8 @@ func describeMicroflowMode(ctx *ExecContext, name ast.QualifiedName, normalized // Describe the live microflow: a module may hold an excluded twin of this // name, and describing that one shows a body the app does not run (#914). - targetMf, _ := pickLive(allMicroflows, + targetMf, _ := pickDescribed(ctx, allMicroflows, + func(mf *microflows.Microflow) model.ID { return mf.ID }, func(mf *microflows.Microflow) bool { return h.GetModuleName(h.FindModuleID(mf.ContainerID)) == name.Module && mf.Name == name.Name }, @@ -397,7 +398,8 @@ func describeNanoflow(ctx *ExecContext, name ast.QualifiedName) error { } // Describe the live nanoflow, not an excluded twin of the same name (#914). - targetNf, _ := pickLive(allNanoflows, + targetNf, _ := pickDescribed(ctx, allNanoflows, + func(nf *microflows.Nanoflow) model.ID { return nf.ID }, func(nf *microflows.Nanoflow) bool { return h.GetModuleName(h.FindModuleID(nf.ContainerID)) == name.Module && nf.Name == name.Name }, @@ -1560,7 +1562,8 @@ func describeRule(ctx *ExecContext, name ast.QualifiedName) error { } // Describe the live rule, not an excluded twin of the same name (#914). - target, _ := pickLive(allRules, + target, _ := pickDescribed(ctx, allRules, + func(r *microflows.Rule) model.ID { return r.ID }, func(r *microflows.Rule) bool { return h.GetModuleName(h.FindModuleID(r.ContainerID)) == name.Module && r.Name == name.Name }, diff --git a/mdl/executor/cmd_pages_describe.go b/mdl/executor/cmd_pages_describe.go index e60d326897..05b519b0dd 100644 --- a/mdl/executor/cmd_pages_describe.go +++ b/mdl/executor/cmd_pages_describe.go @@ -33,15 +33,15 @@ func describePage(ctx *ExecContext, name ast.QualifiedName) error { return mdlerrors.NewBackend("list pages", err) } - var foundPage *pages.Page - for _, p := range allPages { - modID := h.FindModuleID(p.ContainerID) - modName := h.GetModuleName(modID) - if p.Name == name.Name && (name.Module == "" || modName == name.Module) { - foundPage = p - break - } - } + // Describe the live page, not an excluded twin of the same name (#914) — + // or, under the catalog's source build, the pinned document (#1185). + foundPage, _ := pickDescribed(ctx, allPages, + func(p *pages.Page) model.ID { return p.ID }, + func(p *pages.Page) bool { + return p.Name == name.Name && (name.Module == "" || h.GetModuleName(h.FindModuleID(p.ContainerID)) == name.Module) + }, + func(p *pages.Page) bool { return p.Excluded }, + ) if foundPage == nil { return mdlerrors.NewNotFound("page", name.String()) diff --git a/mdl/executor/excluded_docs.go b/mdl/executor/excluded_docs.go index 1c362d56ab..8c5caa7a41 100644 --- a/mdl/executor/excluded_docs.go +++ b/mdl/executor/excluded_docs.go @@ -2,6 +2,11 @@ package executor +import ( + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/model" +) + // Excluded documents ("Exclude from project" in Studio Pro) make a document // name non-unique. Mendix allows two documents in one module to share a name as // long as at most one of them is active: mxbuild reports CE0122 "Duplicate @@ -65,3 +70,39 @@ func pickLiveIndex[T any](items []T, matches func(T) bool, excluded func(T) bool } return first } + +// pickDescribed is pickLive for a DESCRIBE: when ctx pins a document by ID (the +// catalog's source build, which walks documents rather than names) it returns +// that document, so an excluded twin is described as itself — `@excluded` and +// its own body — rather than as a second copy of the live one (#1185). Without +// a pin, or when the pinned ID is not among the matches, it is pickLive. +func pickDescribed[T any](ctx *ExecContext, items []T, idOf func(T) model.ID, matches func(T) bool, excluded func(T) bool) (T, bool) { + if ctx != nil && ctx.describeID != "" { + for _, it := range items { + if matches(it) && idOf(it) == ctx.describeID { + return it, true + } + } + } + return pickLive(items, matches, excluded) +} + +// describedMapping resolves the mapping a DESCRIBE renders: the pinned document +// when the catalog's source build set one (#1185), otherwise the backend's +// by-name lookup, which prefers the live twin. +func describedMapping[T any](ctx *ExecContext, name ast.QualifiedName, + byName func(module, name string) (T, error), list func() ([]T, error), idOf func(T) model.ID) (T, error) { + if ctx.describeID != "" { + all, err := list() + if err != nil { + var zero T + return zero, err + } + for _, m := range all { + if idOf(m) == ctx.describeID { + return m, nil + } + } + } + return byName(name.Module, name.Name) +} diff --git a/mdl/executor/exec_context.go b/mdl/executor/exec_context.go index 9508a966c2..7a5e97a653 100644 --- a/mdl/executor/exec_context.go +++ b/mdl/executor/exec_context.go @@ -101,6 +101,13 @@ type ExecContext struct { // empty EndEvent in a value-returning microflow, where bare `return;` is invalid. DescribingMicroflowHasReturnValue bool + // describeID pins a describe to one stored document. A name is not a unique + // key — a module may hold an excluded twin (#914) — so the catalog's source + // build, which enumerates documents rather than names, sets it to describe + // each twin as itself instead of the live one twice (#1185). Empty means + // "by name", which is every interactive DESCRIBE. + describeID model.ID + // lastWriteStats is the storage write watermark as of the previous // ReportMutation call (or of this context's construction, i.e. the start of // the statement). Its only use is telling "Modified X" from "X was already diff --git a/mdl/executor/issue1185_excluded_twin_source_test.go b/mdl/executor/issue1185_excluded_twin_source_test.go new file mode 100644 index 0000000000..2831dc8486 --- /dev/null +++ b/mdl/executor/issue1185_excluded_twin_source_test.go @@ -0,0 +1,167 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/catalog" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/microflows" +) + +// mendixlabs/mxcli#1185, reported against an app with "two import mappings with +// the same name, one included and one excluded": the catalog showed both but +// nothing told them apart, and "the excluded mapping source doesn't follow the +// established @excluded prefix convention". +// +// Two defects, both pinned here. DESCRIBE of an import/export mapping never +// printed @excluded. And the source build describes every DOCUMENT but looked +// each one up by NAME, so both twins were rendered as whichever one the lookup +// preferred — the same text twice. + +// twinImportMappings returns a live and an excluded import mapping sharing one +// qualified name. The excluded one comes FIRST, which is the order a by-name +// lookup that takes the first match gets wrong. +func twinImportMappings(t *testing.T) (*ExecContext, *model.ImportMapping, *model.ImportMapping) { + t.Helper() + mod := mkModule("Integration") + excluded := &model.ImportMapping{ + BaseElement: model.BaseElement{ID: nextID("im")}, + ContainerID: mod.ID, + Name: "IMM_Order", + Excluded: true, + JsonStructure: "Integration.JSON_OrderOld", + } + live := &model.ImportMapping{ + BaseElement: model.BaseElement{ID: nextID("im")}, + ContainerID: mod.ID, + Name: "IMM_Order", + JsonStructure: "Integration.JSON_Order", + } + all := []*model.ImportMapping{excluded, live} + + h := mkHierarchy(mod) + withContainer(h, mod.ID, mod.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListImportMappingsFunc: func() ([]*model.ImportMapping, error) { return all, nil }, + // The backend's by-name lookup prefers the live twin. + GetImportMappingByQualifiedNameFunc: func(_, _ string) (*model.ImportMapping, error) { return live, nil }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + return ctx, excluded, live +} + +func TestDescribeImportMapping_ExcludedPrintsAnnotation(t *testing.T) { + ctx, excluded, _ := twinImportMappings(t) + + text, err := captureDescribeParallel(ctx, catalog.SourceImportMapping, "Integration.IMM_Order", excluded.ID) + assertNoError(t, err) + if !strings.HasPrefix(text, "@excluded\ncreate or modify import mapping Integration.IMM_Order") { + t.Errorf("an excluded mapping must describe with the @excluded prefix a microflow carries; got:\n%s", text) + } +} + +func TestSourceDescribe_ImportMappingTwinsAreDistinct(t *testing.T) { + ctx, excluded, live := twinImportMappings(t) + + exText, err := captureDescribeParallel(ctx, catalog.SourceImportMapping, "Integration.IMM_Order", excluded.ID) + assertNoError(t, err) + liveText, err := captureDescribeParallel(ctx, catalog.SourceImportMapping, "Integration.IMM_Order", live.ID) + assertNoError(t, err) + + if exText == liveText { + t.Fatalf("both twins describe identically — the source table cannot tell them apart:\n%s", exText) + } + assertContainsStr(t, exText, "JSON_OrderOld") + if strings.Contains(liveText, "@excluded") || !strings.Contains(liveText, "Integration.JSON_Order\n") { + t.Errorf("the live twin must describe as itself, unannotated; got:\n%s", liveText) + } + + // An interactive DESCRIBE pins nothing and shows the live one. + byName, err := captureDescribeParallel(ctx, catalog.SourceImportMapping, "Integration.IMM_Order", "") + assertNoError(t, err) + if byName != liveText { + t.Errorf("an unpinned describe must show the live twin; got:\n%s", byName) + } +} + +func TestSourceDescribe_ExportMappingTwinsAreDistinct(t *testing.T) { + mod := mkModule("Integration") + excluded := &model.ExportMapping{ + BaseElement: model.BaseElement{ID: nextID("em")}, + ContainerID: mod.ID, + Name: "EXM_Order", + Excluded: true, + JsonStructure: "Integration.JSON_OrderOld", + } + live := &model.ExportMapping{ + BaseElement: model.BaseElement{ID: nextID("em")}, + ContainerID: mod.ID, + Name: "EXM_Order", + JsonStructure: "Integration.JSON_Order", + } + h := mkHierarchy(mod) + withContainer(h, mod.ID, mod.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListExportMappingsFunc: func() ([]*model.ExportMapping, error) { + return []*model.ExportMapping{excluded, live}, nil + }, + GetExportMappingByQualifiedNameFunc: func(_, _ string) (*model.ExportMapping, error) { return live, nil }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + exText, err := captureDescribeParallel(ctx, catalog.SourceExportMapping, "Integration.EXM_Order", excluded.ID) + assertNoError(t, err) + liveText, err := captureDescribeParallel(ctx, catalog.SourceExportMapping, "Integration.EXM_Order", live.ID) + assertNoError(t, err) + + if !strings.HasPrefix(exText, "@excluded\n") { + t.Errorf("excluded export mapping must carry @excluded; got:\n%s", exText) + } + if strings.Contains(liveText, "@excluded") { + t.Errorf("live export mapping must not carry @excluded; got:\n%s", liveText) + } +} + +// Microflows had the same source defect: describeMicroflow always picked the +// live twin, so the excluded one's row was a copy of the live one's. +func TestSourceDescribe_MicroflowTwinsAreDistinct(t *testing.T) { + mod := mkModule("MyModule") + excluded := µflows.Microflow{ + BaseElement: model.BaseElement{ID: nextID("mf")}, + ContainerID: mod.ID, + Name: "ACT_Calc", + Excluded: true, + } + live := µflows.Microflow{ + BaseElement: model.BaseElement{ID: nextID("mf")}, + ContainerID: mod.ID, + Name: "ACT_Calc", + } + h := mkHierarchy(mod) + withContainer(h, mod.ID, mod.ID) + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListMicroflowsFunc: func() ([]*microflows.Microflow, error) { + return []*microflows.Microflow{excluded, live}, nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(h)) + + exText, err := captureDescribeParallel(ctx, catalog.SourceMicroflow, "MyModule.ACT_Calc", excluded.ID) + assertNoError(t, err) + liveText, err := captureDescribeParallel(ctx, catalog.SourceMicroflow, "MyModule.ACT_Calc", live.ID) + assertNoError(t, err) + + if !strings.HasPrefix(exText, "@excluded\n") { + t.Errorf("the excluded twin's source must carry @excluded; got:\n%s", exText) + } + if strings.Contains(liveText, "@excluded") { + t.Errorf("the live twin's source must not carry @excluded; got:\n%s", liveText) + } +} diff --git a/mdl/executor/validate_document_annotations.go b/mdl/executor/validate_document_annotations.go index 634d089548..44c891c467 100644 --- a/mdl/executor/validate_document_annotations.go +++ b/mdl/executor/validate_document_annotations.go @@ -48,6 +48,10 @@ var documentAnnotations = map[string]map[string]bool{ "rule": {"excluded": true, "applyentityaccess": true}, // visitor_page_v3.go "page": {"excluded": true}, + // visitor_import_export_mapping.go — DESCRIBE prints @excluded for an + // excluded mapping (#1185), so the statement reads it back. + "importmapping": {"excluded": true}, + "exportmapping": {"excluded": true}, } // ValidateDocumentAnnotations reports (MDL059) an annotation the document it is @@ -88,7 +92,8 @@ func documentAnnotationSuggestion(kind string) string { accepted := documentAnnotations[kind] if len(accepted) == 0 { return fmt.Sprintf("A %s reads no annotations at all. Annotations before CREATE "+ - "belong to entity (@position), association (@anchor), page/nanoflow (@excluded) "+ + "belong to entity (@position), association (@anchor), page/nanoflow/import mapping/"+ + "export mapping (@excluded) "+ "and microflow/rule (@excluded, @applyentityaccess); activity annotations "+ "(@position, @caption, @colour, …) go inside the flow body, on the statement "+ "they belong to.", kind) diff --git a/mdl/executor/validate_document_annotations_test.go b/mdl/executor/validate_document_annotations_test.go index 53b57f81fe..3814a8c7e4 100644 --- a/mdl/executor/validate_document_annotations_test.go +++ b/mdl/executor/validate_document_annotations_test.go @@ -21,6 +21,7 @@ import ( var visitorFilesWithDocumentAnnotations = []string{ "../visitor/visitor_association.go", "../visitor/visitor_entity.go", + "../visitor/visitor_import_export_mapping.go", "../visitor/visitor_microflow.go", "../visitor/visitor_page_v3.go", } diff --git a/mdl/visitor/visitor_import_export_mapping.go b/mdl/visitor/visitor_import_export_mapping.go index 17fbc04b56..ea51cc9ce4 100644 --- a/mdl/visitor/visitor_import_export_mapping.go +++ b/mdl/visitor/visitor_import_export_mapping.go @@ -55,6 +55,7 @@ func (b *Builder) ExitCreateImportMappingStatement(ctx *parser.CreateImportMappi if createStmt.OR() != nil && (createStmt.MODIFY() != nil || createStmt.REPLACE() != nil) { stmt.CreateOrModify = true } + stmt.Excluded = excludedAnnotation(createStmt) } b.statements = append(b.statements, stmt) } @@ -194,6 +195,7 @@ func (b *Builder) ExitCreateExportMappingStatement(ctx *parser.CreateExportMappi if createStmt.OR() != nil && (createStmt.MODIFY() != nil || createStmt.REPLACE() != nil) { stmt.CreateOrModify = true } + stmt.Excluded = excludedAnnotation(createStmt) } b.statements = append(b.statements, stmt) } @@ -469,3 +471,16 @@ func applyMappingHandlingBackup(elem *ast.ImportMappingElementDef, ctx parser.IM } elem.BackupOverridable = c.OVERRIDABLE() != nil } + +// excludedAnnotation reports an `@excluded` on the create statement. DESCRIBE +// prints one for an excluded mapping (#1185), so the statement has to read it +// back or a describe -> exec round trip would lose the exclusion on a create. +func excludedAnnotation(createStmt *parser.CreateStatementContext) bool { + for _, ann := range createStmt.AllAnnotation() { + annCtx := ann.(*parser.AnnotationContext) + if strings.EqualFold(annCtx.AnnotationName().GetText(), "excluded") { + return true + } + } + return false +} diff --git a/mdl/visitor/visitor_mapping_excluded_test.go b/mdl/visitor/visitor_mapping_excluded_test.go new file mode 100644 index 0000000000..e5382712b3 --- /dev/null +++ b/mdl/visitor/visitor_mapping_excluded_test.go @@ -0,0 +1,42 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// DESCRIBE prints @excluded for an excluded import/export mapping +// (mendixlabs/mxcli#1185), so the statement has to read it back — otherwise a +// describe -> exec round trip creates the mapping live. +func TestCreateMapping_ExcludedAnnotation(t *testing.T) { + input := `@excluded +create or modify import mapping MyModule.IMM_Pet with json structure MyModule.JSON_Pet { + create MyModule.Pet { Name = name } +}; +@excluded +create or modify export mapping MyModule.EXM_Pet with json structure MyModule.JSON_Pet { + MyModule.Pet { name = Name } +}; +create import mapping MyModule.IMM_Live with json structure MyModule.JSON_Pet { + create MyModule.Pet { Name = name } +};` + prog, errs := Build(input) + for _, e := range errs { + t.Fatalf("parse error: %v", e) + } + if len(prog.Statements) != 3 { + t.Fatalf("got %d statements, want 3", len(prog.Statements)) + } + if im, ok := prog.Statements[0].(*ast.CreateImportMappingStmt); !ok || !im.Excluded { + t.Errorf("import mapping: want Excluded=true, got %#v", prog.Statements[0]) + } + if em, ok := prog.Statements[1].(*ast.CreateExportMappingStmt); !ok || !em.Excluded { + t.Errorf("export mapping: want Excluded=true, got %#v", prog.Statements[1]) + } + if im, ok := prog.Statements[2].(*ast.CreateImportMappingStmt); !ok || im.Excluded { + t.Errorf("unannotated import mapping must not be excluded, got %#v", prog.Statements[2]) + } +} From cb6810e1d28b555e08d0dd118759dc45d4e76dc7 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 11:16:25 +0000 Subject: [PATCH 38/47] fix(pages): refuse a bare attribute reference on every write, ALTER included `alter page FeedbackModule.ShareFeedback_Logo { insert after textBox1 { image zzImg (ImageType: imageUrl, ImageUrl: '{1}', ImageUrlParams: [{1} = ImageB64]) } }` reported "Altered page" and left a project `mx check` could not LOAD (ArgumentNullException setting 'Attribute', 11.13.0). The data view's nanoflow is missing, so nothing qualifies `ImageB64`, and the pluggable widget's template-parameter builder writes the name as given. The refusal from #678 sat in encodePage/encodeSnippet. ALTER PAGE patches the stored BSON in the page mutator and saves it through UpdateRawUnit, so it never reached that check. - The check moves to modelsdk/canon (BareAttributeRefError). The writer's updateUnit and insertUnit call it next to DuplicateElementIDError, so every raw write is covered: ALTER, styling, widget sync, layouts, templates. - encodePage/encodeSnippet keep calling it first, so a CREATE refusal still names the page and not the unit id. - It refuses every bare reference, stored ones too. Across all 374 units of a stock 11.13 project, 73 of 73 AttributeRefs are qualified: 71 in pages, 1 in a snippet, 1 in a page template, none in any other unit type. Studio Pro cannot load a bare one either, so no project it saved can hold one. Real run on a copy of that project: the ALTER above is refused, naming "ImageB64" and zzImg's path, and no unit changes. The qualified form, `set Title`, a qualified textbox insert and a Class change all apply, and `mxcli docker check` reports 0 errors. Control: with the writer guard stubbed, both new tests fail with the reported symptom. Co-Authored-By: Claude Opus 5.5 --- .../skills/fix-issue/findings/modelsdk.jsonl | 1 + .../bug-patterns/unloadable-model-writes.md | 6 +- .../alter-page-bare-attributeref-refused.mdl | 57 +++++++++ .../modelsdk/page_bare_attributeref.go | 59 --------- .../modelsdk/page_bare_attributeref_test.go | 47 -------- .../page_mutator_bare_attributeref_test.go | 113 ++++++++++++++++++ mdl/backend/modelsdk/page_write.go | 7 +- mdl/backend/modelsdk/snippet_write.go | 5 +- mdl/executor/cmd_pages_builder.go | 4 +- modelsdk/canon/attributeref.go | 96 +++++++++++++++ modelsdk/canon/attributeref_test.go | 49 ++++++++ modelsdk/mpr/writer_bare_attributeref_test.go | 95 +++++++++++++++ modelsdk/mpr/writer_core.go | 9 ++ 13 files changed, 434 insertions(+), 114 deletions(-) create mode 100644 mdl-examples/bug-tests/alter-page-bare-attributeref-refused.mdl delete mode 100644 mdl/backend/modelsdk/page_bare_attributeref.go delete mode 100644 mdl/backend/modelsdk/page_bare_attributeref_test.go create mode 100644 mdl/backend/modelsdk/page_mutator_bare_attributeref_test.go create mode 100644 modelsdk/canon/attributeref.go create mode 100644 modelsdk/canon/attributeref_test.go create mode 100644 modelsdk/mpr/writer_bare_attributeref_test.go diff --git a/.claude/skills/fix-issue/findings/modelsdk.jsonl b/.claude/skills/fix-issue/findings/modelsdk.jsonl index 29e35108d3..62b23166c7 100644 --- a/.claude/skills/fix-issue/findings/modelsdk.jsonl +++ b/.claude/skills/fix-issue/findings/modelsdk.jsonl @@ -21,3 +21,4 @@ {"area": "docs / modelsdk/mpr", "date": "2026-09-22", "symptom": "The MPR reference pages' \"Unit Types\" tables mapped BSON `$Type` to document kinds, and 15 of the rows named a spelling no unit carries: `Pages$Page`/`Pages$Layout`/`Pages$Snippet`/`Pages$BuildingBlock` (real units say `Forms$*`), and docs/05-mdl-specification/10-bson-mapping.md lowercased eleven more (`microflows$microflow`, `pages$page`, `security$ProjectSecurity`…). It also listed `CustomWidgets$customwidget` as a document type. Found while fixing mendixlabs/mxcli#1072, filed and fixed separately.", "cause": "The tables were written from the TypeScript SDK's QUALIFIED names rather than the storage names Mendix writes — the same split CLAUDE.md documents for `ShowPageAction`/`ShowFormAction`, never applied here. `CustomWidgets$CustomWidget` is a widget element inside a page's tree (mdl/catalog/builder_widget_refs.go), never a unit, so that row was removed rather than corrected.", "file": "`docs-site/src/internals/mpr-format.md`, `docs/05-mdl-specification/10-bson-mapping.md`; test `modelsdk/mpr/docs_schema_test.go` (TestDocumentedUnitTypesUseStorageNames)", "insight": "Measuring the real set is one command and settles the whole table at once: decode every `mprcontents/*/*/*.mxunit` (and every v1 `Unit.Contents` blob) and count `$Type` — 28 distinct values across a blank 11.6.6 app and a 9.24.30 one. Do NOT try to verify rows one at a time against gen, which carries BOTH spellings: `model/types.go` defines `DocumentTypePage = \"Pages$Page\"` and mdl/catalog/builder_xpath.go defensively matches `Forms$Page` AND `Pages$Page`, so grepping the codebase 'confirms' the wrong name. The fixture is the arbiter; the codebase is not. The test rule that makes this checkable without a maintenance burden keys on the LOCAL name after the `$`, case-insensitively: a fixture cannot prove a type ABSENT (a blank project has no business-event service), so demanding every documented type be present would fail correct rows — but when the fixture has a type with the same local name, the documented row must equal it exactly. That catches all four `Pages$` rows and all eleven lowercase ones with zero false positives. Its stated limit is real and cost a manual fix: a row whose local name appears nowhere in the fixtures is not checked at all, which is how `CustomWidgets$customwidget` slipped past and had to be removed by hand. One editing trap, not a Mendix one: anchoring a section replacement on `'---'` matches a markdown TABLE SEPARATOR (`|---|---|`) long before the horizontal rule you meant — the edit silently no-ops on the table you were replacing. Anchor on `'\\n---\\n'`.", "refs": ["mendixlabs/mxcli#1072"]} {"area": "modelsdk/canon", "date": "2026-09-23", "symptom": "The storage-GUID write guard (`canon.StorageGUIDChanges`) stopped refusing the MOVE ENTITY data loss it had exposed (ako/mxcli#503). With MoveEntity's carries removed, moving an association's TO side re-minted the in-place converted cross-association's GUID and the write went through silently, where the issue records a refusal.", "cause": "`sameMember` (added in 86927852 to stop the guard refusing transplant mis-pairings) required an equal `$Type` as well as an equal `Name`. MoveEntity converts `DomainModels$Association` to `DomainModels$CrossAssociation` IN PLACE, keeping `$ID` and `Name`, so the type clause made the guard skip the pair. The type clause excluded nothing the transplant can produce: `pairDoc` stops at a `$Type` mismatch (TestTransplantIgnoresMismatchedTypes).", "file": "`modelsdk/canon/storageguid.go` (`sameMember`, the note above `GUIDChange`)", "insight": "Before adding a clause to an identity test that sits on an approximate pairing, ask what error of THAT pairing the clause excludes. The transplant only mis-pairs same-type, different-name elements, so `$Type` excluded none of its errors. Its only effect was to exclude the one writer that keeps an `$ID` across a type change deliberately. Rule now: Name when both sides have one; `$Type` only when neither does; a pair with a name on one side only is not a match. How the gap was found: stub the three MoveEntity carries on main and run TestIssue503. The child-side case returned no error where the issue quotes a refusal. That mismatch between the recorded refusal and the observed silence was the tell. A guard's quiet is not evidence of a clean write, so a guard's comment must list every hole it leaves; this one listed only renames. Controls: (1) the new canon test fails on the old `sameMember` with `got 0 change(s)`; (2) with the MoveEntity carries stubbed the child-side move is refused again with the issue's exact message, and the parent-side move still goes through, because the moved element changes unit and pairs with nothing (a documented hole); (3) the 86927852 false positive does not return: `marketplace install --file mx-modules/BusinessEvents_3.12.0.mpk` into a copy of testdata/expr-checker, then `create or modify persistent entity BusinessEvents.PublishedBusinessEvent (EventId: long)` is accepted, while a build with an `$ID`-only rule refuses it (EventId paired with a removed attribute). The existing table case `DifferentType_NotAChange` pinned the wrong decision with the justification 'nothing authors this today', which was false the day it was written. Grep for the writers (`SetID(x.ID())` next to `New()`) before claiming nothing authors a shape.", "refs": ["ako/mxcli#503", "mendixlabs/mxcli#1119"]} {"area":"modelsdk/codec","date":"2026-09-25","symptom":"A compound design property (Atlas `Spacing` → `margin-bottom`, or a multiSelect toggle group) writes its `Forms$CompoundDesignPropertyValue.Properties` list with BSON array marker 3 where Studio Pro writes 2. `check`, `exec` and `mx check` all pass; a describe → exec round trip of FeedbackModule.ShareFeedback (Feedback v4.0.2, 11.13.0) turned every nested marker-2 list into 3","cause":"The codec picks a PartList's marker from the CHILD element's `$Type` only (`partListMarker` → `lookupListMarker`). The nested list and the enclosing `Forms$Appearance.DesignProperties` list (marker 3) both hold `Forms$DesignPropertyValue`, so no `RegisterListMarker` on the child type could tell them apart and both fell to the default 3","file":"`modelsdk/codec/defaults.go` (`RegisterPropertyListMarker`), `modelsdk/codec/encoder.go` (`propertyListMarker`), `mdl/backend/modelsdk/widget_write.go` (init)","insight":"When one child `$Type` sits in two lists with different markers, the marker belongs to the owner+key, not the child: `RegisterPropertyListMarker(owner, key, m)` is consulted first, for an empty list too and in the selective-rebuild path. Establish the marker by counting Studio Pro-authored BSON before changing anything: walking every mxunit gave Compound.Properties 373/373 marker 2 and Appearance.DesignProperties 1821/1821 marker 3 across pages, layouts, building blocks and page templates. Count per (owner $Type, key) — a flat grep of `Properties [marker=2]` in ndsl also matches unrelated lists. Pages, snippets and layouts share `newAppearance`, so one registration covers all. Test `TestAppearanceCompoundDesignPropertyMarkers`; bug-test `mdl-examples/bug-tests/compound-design-property-marker.mdl`. Same class, not fixed: the selective-rebuild branch of `encodeEntry` still hard-codes 3 for lists with no owner registration, ignoring a child-type `RegisterListMarker`","refs":["#668"]} +{"area":"modelsdk/mpr","date":"2026-09-25","symptom":"`alter page FeedbackModule.ShareFeedback_Logo { insert after textBox1 { image zzImg (ImageType: imageUrl, ImageUrl: '{1}', ImageUrlParams: [{1} = ImageB64]) } }` (data view over a nanoflow the project lacks, 11.13.0) reported \"Altered page\"; `mxcli docker check` then could not LOAD the project: ArgumentNullException setting 'Attribute' of an Attribute in a Page","cause":"The bare-AttributeRef refusal (#678) lived in encodePage/encodeSnippet only. ALTER PAGE patches the stored BSON in pagemutator and saves via UpdateRawUnit, never passing the encoder; with no entity in scope the pluggable-widget template-parameter builder (widgetobj) writes the name as given, so DomainModels$AttributeRef{Attribute:\"ImageB64\"} reached disk","file":"modelsdk/canon/attributeref.go; modelsdk/mpr/writer_core.go (updateUnit, insertUnit)","insight":"A guard placed in one encoder covers one write path; the page family has at least four (encodePage/Snippet, pagemutator Save, widget sync apply, layout/template raw writes). Put an unloadable-shape refusal at the writer beside DuplicateElementIDError, as that one already argued. Measured before refusing stored refs too: 73 of 73 AttributeRefs across all 374 units of a stock 11.13 project are qualified (71 page, 1 snippet, 1 page template, none elsewhere) — a stored bare one cannot have come from Studio Pro, so refusing ALL bare refs (not only new ones) blocks nothing legitimate. The textbox path does NOT reproduce it: attributeRefToGen nulls a bare name (a silent binding drop instead); the pluggable/column template builders are the ones that write it verbatim. The test goes through the real mutator + writer on the expr-checker fixture (InsertColumns with a bare CaptionParams ref).","refs":["#678"]} diff --git a/docs-wiki/bug-patterns/unloadable-model-writes.md b/docs-wiki/bug-patterns/unloadable-model-writes.md index be793e7122..4997197338 100644 --- a/docs-wiki/bug-patterns/unloadable-model-writes.md +++ b/docs-wiki/bug-patterns/unloadable-model-writes.md @@ -33,8 +33,10 @@ project rather than one page, and because the diagnostic is a stack trace. that points at the wrong thing. Mendix reconstructs each stored property into a typed identifier as it loads, and a value that cannot be parsed into that type takes the loader down. The shapes seen so far: a one-qualifier member name -written where an attribute reference is expected (an attribute is bare or -`Module.Entity.Attribute`, never `Module.Name`); an unqualified entity name in a +written where an attribute reference is expected (a stored +`DomainModels$AttributeRef` is `Module.Entity.Attribute` and nothing else — a +bare name fails to load as surely as `Module.Name`, and the writer now refuses +both for every unit, ALTER's raw patches included); an unqualified entity name in a generalization; a literal string where the property is a `ConstantIdentifier`; an empty `DestinationEntity`; an index column pointing at a GUID that no longer exists; a sequence flow dangling from a `break`; an association whose `ParentPointer` diff --git a/mdl-examples/bug-tests/alter-page-bare-attributeref-refused.mdl b/mdl-examples/bug-tests/alter-page-bare-attributeref-refused.mdl new file mode 100644 index 0000000000..1a44751049 --- /dev/null +++ b/mdl-examples/bug-tests/alter-page-bare-attributeref-refused.mdl @@ -0,0 +1,57 @@ +-- ============================================================================ +-- ALTER PAGE refuses to store a bare attribute reference +-- ============================================================================ +-- +-- Symptom: on a copy of a real 11.13.0 project, +-- alter page FeedbackModule.ShareFeedback_Logo { +-- insert after textBox1 { image zzImg (ImageType: imageUrl, ImageUrl: '{1}', +-- ImageUrlParams: [{1} = ImageB64]) } +-- } +-- reported "Altered page", and `mxcli docker check` then could not LOAD the +-- project: ArgumentNullException setting 'Attribute' of an Attribute in a Page. +-- The data view is sourced by a nanoflow the module does not ship, so there is +-- no entity to qualify `ImageB64` against, and the image's template parameter +-- was written as DomainModels$AttributeRef { Attribute: "ImageB64" }. +-- +-- Cause: CREATE PAGE refused a bare reference in its encoder (encodePage), but +-- ALTER PAGE patches the stored BSON tree in the page mutator and saves it with +-- UpdateRawUnit, which never passes that encoder. +-- +-- Fix: the refusal moved to the write choke point (modelsdk/mpr updateUnit and +-- insertUnit, canon.BareAttributeRefError), beside the duplicate-$ID guard, so +-- every raw write of any unit is covered. +-- +-- Verify: exec this script → "Altered page"; `mxcli docker check` → 0 errors. +-- Change the inserted image's parameter to `{1} = Subject` → the ALTER is +-- refused ("attribute reference not qualified as Module.Entity.Attribute … +-- "Subject" at …/imgQualified…"), and the stored page is unchanged. +-- ============================================================================ + +create entity MyFirstModule.AlterBareDraft ( + Subject: String(200), + PictureUrl: String(400) +); +/ + +@excluded +create or modify page MyFirstModule.AlterBareDraft_Example +( Title: 'Draft (example)', Layout: Atlas_Core.Atlas_Default ) +{ + dataview dv (DataSource: nanoflow MyFirstModule.DS_MissingAlterBareDraft) { + textbox txtSubject (Label: 'Subject', Attribute: MyFirstModule.AlterBareDraft.Subject) + } +} +/ + +-- Inside the data view there is no resolvable entity: only a qualified +-- reference can be stored. +alter page MyFirstModule.AlterBareDraft_Example { + insert after txtSubject { + image imgQualified ( + ImageType: imageUrl, + ImageUrl: '{1}', + ImageUrlParams: [{1} = MyFirstModule.AlterBareDraft.PictureUrl] + ) + } +}; +/ diff --git a/mdl/backend/modelsdk/page_bare_attributeref.go b/mdl/backend/modelsdk/page_bare_attributeref.go deleted file mode 100644 index ead2078258..0000000000 --- a/mdl/backend/modelsdk/page_bare_attributeref.go +++ /dev/null @@ -1,59 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package modelsdkbackend - -import ( - "fmt" - "strings" - - "go.mongodb.org/mongo-driver/bson" -) - -// refuseBareAttributeRefs refuses a page or snippet whose encoded form holds a -// DomainModels$AttributeRef that is not Module.Entity.Attribute. -// -// Mendix rebuilds each stored reference into a typed identifier as it loads, -// and an attribute that does not parse as one takes the loader down before any -// validation runs: a bare `ImageB64` image parameter left `mx check` unable to -// load the project (ArgumentNullException setting 'Attribute', 11.13.0) — the -// page was excluded, which does not help, as loading is not validating. Studio -// Pro qualifies every one (72 of 72 across a stock project's pages, snippets -// and layouts). A bare name reaches here when nothing could qualify it — inside -// a data container whose flow the project lacks — so this is the last line -// under the check that refuses it first (checkUnscopedBindings). -func refuseBareAttributeRefs(contents []byte) error { - var bad []string - var walk func(v bson.RawValue, path string) - walk = func(v bson.RawValue, path string) { - switch v.Type { - case bson.TypeEmbeddedDocument: - doc := v.Document() - if t, ok := doc.Lookup("$Type").StringValueOK(); ok && t == "DomainModels$AttributeRef" { - if a, ok := doc.Lookup("Attribute").StringValueOK(); ok && a != "" && strings.Count(a, ".") < 2 { - bad = append(bad, fmt.Sprintf("%q at %s", a, path)) - } - } - name, _ := doc.Lookup("Name").StringValueOK() - elems, _ := doc.Elements() - for _, e := range elems { - p := path + "/" + e.Key() - if name != "" { - p = path + "/" + name + "." + e.Key() - } - walk(e.Value(), p) - } - case bson.TypeArray: - vals, _ := v.Array().Values() - for _, x := range vals { - walk(x, path) - } - } - } - walk(bson.RawValue{Type: bson.TypeEmbeddedDocument, Value: contents}, "") - if len(bad) == 0 { - return nil - } - return fmt.Errorf("attribute reference not qualified as Module.Entity.Attribute — Mendix cannot load a "+ - "project holding one, so it is not written: %s. Qualify it in the script; inside a data container "+ - "whose flow the project lacks there is no entity to resolve a bare name against", strings.Join(bad, "; ")) -} diff --git a/mdl/backend/modelsdk/page_bare_attributeref_test.go b/mdl/backend/modelsdk/page_bare_attributeref_test.go deleted file mode 100644 index 83c2c2f001..0000000000 --- a/mdl/backend/modelsdk/page_bare_attributeref_test.go +++ /dev/null @@ -1,47 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 - -package modelsdkbackend - -import ( - "strings" - "testing" - - "go.mongodb.org/mongo-driver/bson" -) - -// A DomainModels$AttributeRef whose Attribute is not Module.Entity.Attr makes -// the project unloadable: an image URL parameter written as a bare `ImageB64` -// (Feedback v4.0.2's ShareFeedback_Logo, under a data view whose flow the -// project lacks) left `mx check` unable to LOAD the project — -// ArgumentNullException setting 'Attribute', Mendix 11.13.0 — while the page -// was excluded. Studio Pro qualifies every one: 72 of 72 AttributeRefs across -// that project's pages, snippets and layouts. So the writer refuses the bare -// form, naming it, instead of storing it. -func TestRefuseBareAttributeRefs(t *testing.T) { - attrRef := func(a string) bson.D { - return bson.D{{Key: "$Type", Value: "DomainModels$AttributeRef"}, {Key: "Attribute", Value: a}, {Key: "EntityRef", Value: nil}} - } - doc := func(a string) []byte { - b, err := bson.Marshal(bson.D{ - {Key: "$Type", Value: "Forms$Page"}, - {Key: "Widgets", Value: bson.A{int32(2), bson.D{ - {Key: "$Type", Value: "CustomWidgets$CustomWidget"}, - {Key: "Name", Value: "image1"}, - {Key: "Params", Value: bson.A{int32(2), bson.D{{Key: "AttributeRef", Value: attrRef(a)}}}}, - }}}, - }) - if err != nil { - t.Fatal(err) - } - return b - } - err := refuseBareAttributeRefs(doc("ImageB64")) - if err == nil || !strings.Contains(err.Error(), "ImageB64") { - t.Fatalf("a bare attribute reference must be refused, naming it; got %v", err) - } - for _, ok := range []string{"FeedbackModule.Feedback.ImageB64", ""} { - if err := refuseBareAttributeRefs(doc(ok)); err != nil { - t.Errorf("%q must be accepted: %v", ok, err) - } - } -} diff --git a/mdl/backend/modelsdk/page_mutator_bare_attributeref_test.go b/mdl/backend/modelsdk/page_mutator_bare_attributeref_test.go new file mode 100644 index 0000000000..5e2ecdb607 --- /dev/null +++ b/mdl/backend/modelsdk/page_mutator_bare_attributeref_test.go @@ -0,0 +1,113 @@ +// SPDX-License-Identifier: Apache-2.0 + +package modelsdkbackend + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// ALTER PAGE does not go through encodePage: the page mutator patches the stored +// BSON tree and saves the bytes with UpdateRawUnit. A bare attribute in a column +// template parameter (what an ALTER inside a data container with no resolvable +// entity produces — the column builder writes the name as given) reached disk +// that way, and Mendix cannot LOAD a project holding one (ArgumentNullException +// setting 'Attribute', 11.13.0). The same statement, run on a copy of a real +// project, left `mx check` unable to open it. +// +// So the refusal sits at the writer, and this test goes through the real +// mutator and the real writer — the wiring is what came undone. + +func openAccountOverviewMutator(t *testing.T) (*Backend, backend.PageMutator, model.ID) { + t.Helper() + proj := copyFixture(t) + b := New() + if err := b.Connect(proj); err != nil { + t.Fatalf("connect: %v", err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + ps, err := b.ListPages() + if err != nil { + t.Fatalf("ListPages: %v", err) + } + var id model.ID + for _, p := range ps { + if p.Name == "Account_Overview" { + id = p.ID + } + } + if id == "" { + t.Fatal("fixture has no Account_Overview page") + } + m, err := b.OpenPageForMutation(id) + if err != nil { + t.Fatalf("OpenPageForMutation: %v", err) + } + return b, m, id +} + +func captionColumn(attr string) *backend.DataGridColumnSpec { + return &backend.DataGridColumnSpec{ + Caption: "{1}", + CaptionParams: []*pages.ClientTemplateParameter{{ + BaseElement: model.BaseElement{ID: model.ID("0d4b8c1e-1111-4a1a-9a1a-111111111111")}, + AttributeRef: attr, + }}, + ShowContentAs: "dynamicText", + Content: "x", + } +} + +func TestAlterPageSaveRefusesABareAttributeRef(t *testing.T) { + b, m, id := openAccountOverviewMutator(t) + before, err := b.GetRawUnitBytes(id) + if err != nil { + t.Fatalf("read stored page: %v", err) + } + + if err := m.InsertColumns("dataGrid21", "WebServiceUser", backend.InsertPosition("after"), + []*backend.DataGridColumnSpec{captionColumn("FullName")}); err != nil { + t.Fatalf("InsertColumns: %v", err) + } + err = m.Save() + if err == nil { + t.Fatal("ALTER saved a page holding a bare attribute reference; Mendix cannot load that project") + } + for _, want := range []string{`"FullName"`, "Module.Entity.Attribute"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("refusal does not name %s: %v", want, err) + } + } + + after, err := b.GetRawUnitBytes(id) + if err != nil { + t.Fatalf("read page after refusal: %v", err) + } + if string(after) != string(before) { + t.Error("the refused ALTER still changed the stored page") + } +} + +func TestAlterPageSaveAcceptsAQualifiedAttributeRef(t *testing.T) { + // The control: the same column, qualified, is an ordinary ALTER. + b, m, id := openAccountOverviewMutator(t) + before, _ := b.GetRawUnitBytes(id) + if err := m.InsertColumns("dataGrid21", "WebServiceUser", backend.InsertPosition("after"), + []*backend.DataGridColumnSpec{captionColumn("Administration.Account.FullName")}); err != nil { + t.Fatalf("InsertColumns: %v", err) + } + if err := m.Save(); err != nil { + t.Fatalf("a qualified attribute reference was refused: %v", err) + } + after, _ := b.GetRawUnitBytes(id) + if string(after) == string(before) { + t.Fatal("control did not write: the accepted ALTER must reach the stored page") + } + if !strings.Contains(string(after), "Administration.Account.FullName") { + t.Error("the qualified reference is not in the stored page") + } +} diff --git a/mdl/backend/modelsdk/page_write.go b/mdl/backend/modelsdk/page_write.go index 6c15d5b803..ed2a15d413 100644 --- a/mdl/backend/modelsdk/page_write.go +++ b/mdl/backend/modelsdk/page_write.go @@ -11,6 +11,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/backend/bsonnav" "github.com/mendixlabs/mxcli/mdl/types" "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/canon" "github.com/mendixlabs/mxcli/modelsdk/codec" "github.com/mendixlabs/mxcli/modelsdk/element" genDT "github.com/mendixlabs/mxcli/modelsdk/gen/datatypes" @@ -260,8 +261,10 @@ func encodePage(page *pages.Page, pv *types.ProjectVersion, carry func(*genPg.Pa if err != nil { return nil, err } - if err := refuseBareAttributeRefs(contents); err != nil { - return nil, fmt.Errorf("page %q: %w", page.Name, err) + // Also refused at the writer (canon/attributeref.go); checked here first so + // the refusal names the page rather than its unit id. + if err := canon.BareAttributeRefError(fmt.Sprintf("page %q", page.Name), contents); err != nil { + return nil, err } return contents, nil } diff --git a/mdl/backend/modelsdk/snippet_write.go b/mdl/backend/modelsdk/snippet_write.go index 2921ad0025..3545aecbe4 100644 --- a/mdl/backend/modelsdk/snippet_write.go +++ b/mdl/backend/modelsdk/snippet_write.go @@ -7,6 +7,7 @@ import ( "github.com/mendixlabs/mxcli/mdl/types" "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/canon" "github.com/mendixlabs/mxcli/modelsdk/codec" "github.com/mendixlabs/mxcli/modelsdk/element" genPg "github.com/mendixlabs/mxcli/modelsdk/gen/pages" @@ -37,8 +38,8 @@ func encodeSnippet(snippet *pages.Snippet, pv *types.ProjectVersion) ([]byte, er if err != nil { return nil, err } - if err := refuseBareAttributeRefs(contents); err != nil { // see page_bare_attributeref.go - return nil, fmt.Errorf("snippet %q: %w", snippet.Name, err) + if err := canon.BareAttributeRefError(fmt.Sprintf("snippet %q", snippet.Name), contents); err != nil { // see encodePage + return nil, err } return contents, nil } diff --git a/mdl/executor/cmd_pages_builder.go b/mdl/executor/cmd_pages_builder.go index 77e70dc33a..35b102b570 100644 --- a/mdl/executor/cmd_pages_builder.go +++ b/mdl/executor/cmd_pages_builder.go @@ -86,8 +86,8 @@ type pageBuilder struct { // be qualified, and one written bare made `mx check` fail to LOAD the // project (ArgumentNullException setting 'Attribute', Mendix 11.13.0). // DESCRIBE writes those bindings qualified there, the check refuses a bare - // one (checkUnscopedBindings), and the page writer refuses any bare - // attribute reference as a last line (refuseBareAttributeRefs). + // one (unscopedBindings), and the writer refuses any bare attribute + // reference as a last line, ALTER included (canon.BareAttributeRefError). tolerateDanglingRefs bool // Local page/snippet variables (Variables: { $name: Type = 'default' }). diff --git a/modelsdk/canon/attributeref.go b/modelsdk/canon/attributeref.go new file mode 100644 index 0000000000..2d3a7e9907 --- /dev/null +++ b/modelsdk/canon/attributeref.go @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: Apache-2.0 + +package canon + +import ( + "fmt" + "strings" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// A DomainModels$AttributeRef whose Attribute is not Module.Entity.Attribute +// cannot be LOADED. Mendix rebuilds each stored reference into a typed +// identifier as it reads the unit, and a name that does not parse as one takes +// the loader down before any validation runs: +// +// InvalidOperationException: An error occurred when trying to set the +// 'Attribute' property of a Attribute in a Page with ID ... +// ---> ArgumentNullException: Value cannot be null. (Parameter 'value') +// +// (`mx check`, Mendix 11.13.0). Excluding the document does not help — loading +// is not validating. Studio Pro qualifies every one: 73 of 73 across all 374 +// units of a stock 11.13 project (71 in pages, one each in a snippet and a +// page template; no other unit type holds one). +// +// This sits at the write choke point for the same reason DuplicateElementIDError +// does. The page and snippet encoders refused a bare reference, but ALTER PAGE +// patches the stored tree and saves it through UpdateRawUnit, and a pluggable +// widget's template parameter inside a data container with no resolvable entity +// reached disk bare that way. Any raw write can. +// +// It refuses every bare reference in the unit, stored or new. A stored one +// cannot have come from Studio Pro, which cannot load it either; writing it back +// keeps the project unloadable, and the message names it so the statement that +// rewrites the unit can drop or qualify it. + +// BareAttributeRefs names every DomainModels$AttributeRef in raw whose +// Attribute is non-empty and not Module.Entity.Attribute, with where it sits +// (the nearest named element's Name, then the property path). An empty +// Attribute is an unbound slot and is not reported. A document that cannot be +// read yields nothing, as DuplicateElementIDs does. +func BareAttributeRefs(raw []byte) []string { + var bad []string + var walk func(v bson.RawValue, path string) + walk = func(v bson.RawValue, path string) { + switch v.Type { + case bson.TypeEmbeddedDocument: + doc, ok := v.DocumentOK() + if !ok { + return + } + if t, ok := doc.Lookup("$Type").StringValueOK(); ok && t == "DomainModels$AttributeRef" { + if a, ok := doc.Lookup("Attribute").StringValueOK(); ok && a != "" && strings.Count(a, ".") < 2 { + bad = append(bad, fmt.Sprintf("%q at %s", a, path)) + } + } + name, _ := doc.Lookup("Name").StringValueOK() + elems, _ := doc.Elements() + for _, e := range elems { + p := path + "/" + e.Key() + if name != "" { + p = path + "/" + name + "." + e.Key() + } + walk(e.Value(), p) + } + case bson.TypeArray: + arr, ok := v.ArrayOK() + if !ok { + return + } + vals, _ := arr.Values() + for _, x := range vals { + walk(x, path) + } + } + } + if err := bson.Raw(raw).Validate(); err != nil { + return nil + } + walk(bson.RawValue{Type: bson.TypeEmbeddedDocument, Value: raw}, "") + return bad +} + +// BareAttributeRefError returns the error a write should fail with, or nil. +// unitLabel names the unit: an id is enough, a qualified name is better. +func BareAttributeRefError(unitLabel string, raw []byte) error { + bad := BareAttributeRefs(raw) + if len(bad) == 0 { + return nil + } + return fmt.Errorf("refusing to write unit %s: attribute reference not qualified as "+ + "Module.Entity.Attribute — Mendix cannot load a project holding one: %s. Qualify it in the "+ + "script; inside a data container whose entity cannot be resolved (e.g. its data-source flow "+ + "is missing) there is nothing to qualify a bare name against", + unitLabel, strings.Join(bad, "; ")) +} diff --git a/modelsdk/canon/attributeref_test.go b/modelsdk/canon/attributeref_test.go new file mode 100644 index 0000000000..1f5fee2224 --- /dev/null +++ b/modelsdk/canon/attributeref_test.go @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: Apache-2.0 + +package canon + +import ( + "strings" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// Moved from mdl/backend/modelsdk (page_bare_attributeref_test.go) when the +// guard moved to the write choke point. The shape is Feedback v4.0.2's +// ShareFeedback_Logo: an image URL parameter written as a bare `ImageB64` +// under a data view whose flow the project lacks, which left `mx check` unable +// to LOAD the project (ArgumentNullException setting 'Attribute', 11.13.0). +func TestBareAttributeRefError(t *testing.T) { + attrRef := func(a string) bson.D { + return bson.D{{Key: "$Type", Value: "DomainModels$AttributeRef"}, {Key: "Attribute", Value: a}, {Key: "EntityRef", Value: nil}} + } + doc := func(a string) []byte { + b, err := bson.Marshal(bson.D{ + {Key: "$Type", Value: "Forms$Page"}, + {Key: "Widgets", Value: bson.A{int32(2), bson.D{ + {Key: "$Type", Value: "CustomWidgets$CustomWidget"}, + {Key: "Name", Value: "image1"}, + {Key: "Params", Value: bson.A{int32(2), bson.D{{Key: "AttributeRef", Value: attrRef(a)}}}}, + }}}, + }) + if err != nil { + t.Fatal(err) + } + return b + } + for _, bare := range []string{"ImageB64", "Feedback.ImageB64"} { + err := BareAttributeRefError("page X", doc(bare)) + if err == nil || !strings.Contains(err.Error(), `"`+bare+`"`) || !strings.Contains(err.Error(), "image1") { + t.Fatalf("a bare attribute reference must be refused, naming it and its widget; got %v", err) + } + } + for _, ok := range []string{"FeedbackModule.Feedback.ImageB64", ""} { + if err := BareAttributeRefError("page X", doc(ok)); err != nil { + t.Errorf("%q must be accepted: %v", ok, err) + } + } + if got := BareAttributeRefs([]byte{1, 2, 3}); got != nil { + t.Errorf("unreadable bytes must yield nothing, got %v", got) + } +} diff --git a/modelsdk/mpr/writer_bare_attributeref_test.go b/modelsdk/mpr/writer_bare_attributeref_test.go new file mode 100644 index 0000000000..bd4dc52c8f --- /dev/null +++ b/modelsdk/mpr/writer_bare_attributeref_test.go @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: Apache-2.0 + +package mpr + +import ( + "os" + "strings" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// A DomainModels$AttributeRef whose Attribute is not Module.Entity.Attribute +// makes the PROJECT unloadable — `mx check` dies in the loader with +// ArgumentNullException setting 'Attribute' (Mendix 11.13.0) before any +// validation runs, and an excluded page does not help. The page and snippet +// encoders refused one, but ALTER PAGE patches the stored tree and saves it +// through UpdateRawUnit, so `alter page … insert … { image … (ImageUrlParams: +// [{1} = ImageB64]) }` inside a data view with no resolvable entity wrote one. +// The guard sits here, next to the duplicate-$ID one, and these tests go +// through the Writer because the wiring is what can come undone. + +func pageWithAttributeRef(t *testing.T, attr string) []byte { + t.Helper() + bin := func(id string) bson.Binary { return bson.Binary{Subtype: 0x00, Data: uuidToBlob(id)} } + b, err := bson.Marshal(bson.D{ + {Key: "$Type", Value: "Forms$Page"}, + {Key: "$ID", Value: bin("22222222-2222-2222-2222-222222222222")}, + {Key: "Name", Value: "P"}, + {Key: "Widget", Value: bson.D{ + {Key: "$Type", Value: "Forms$TextBox"}, + {Key: "$ID", Value: bin("33333333-3333-3333-3333-333333333333")}, + {Key: "Name", Value: "textBox1"}, + {Key: "AttributeRef", Value: bson.D{ + {Key: "$Type", Value: "DomainModels$AttributeRef"}, + {Key: "$ID", Value: bin("44444444-4444-4444-4444-444444444444")}, + {Key: "Attribute", Value: attr}, + {Key: "EntityRef", Value: nil}, + }}, + }}, + }) + if err != nil { + t.Fatalf("marshal: %v", err) + } + return b +} + +func TestUpdateUnitRefusesABareAttributeRef(t *testing.T) { + const unitID = "55555555-5555-5555-5555-555555555555" + w, unitPath := newV2WriterForCommitTest(t, unitID, pageWithAttributeRef(t, "MyModule.Customer.Name")) + before, err := os.ReadFile(unitPath) + if err != nil { + t.Fatalf("read seeded unit: %v", err) + } + + for _, bare := range []string{"Name", "Customer.Name"} { + err := w.UpdateRawUnit(unitID, pageWithAttributeRef(t, bare)) + if err == nil { + t.Fatalf("write accepted a unit holding the bare attribute reference %q", bare) + } + for _, want := range []string{unitID, `"` + bare + `"`, "textBox1", "Module.Entity.Attribute"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("message missing %q: %v", want, err) + } + } + } + + after, err := os.ReadFile(unitPath) + if err != nil { + t.Fatalf("read unit after refusal: %v", err) + } + if string(after) != string(before) { + t.Error("the refused write still changed the stored unit") + } +} + +func TestUpdateUnitAcceptsQualifiedAndEmptyAttributeRefs(t *testing.T) { + // The control. A qualified reference is what Studio Pro stores (73 of 73 + // across a real project's units), and an EMPTY Attribute is a slot the + // author has not bound yet — neither may be refused. + const unitID = "66666666-6666-6666-6666-666666666666" + w, unitPath := newV2WriterForCommitTest(t, unitID, pageWithAttributeRef(t, "MyModule.Customer.Name")) + for _, ok := range []string{"MyModule.Customer.Email", ""} { + if err := w.UpdateRawUnit(unitID, pageWithAttributeRef(t, ok)); err != nil { + t.Fatalf("Attribute %q refused: %v", ok, err) + } + } + after, err := os.ReadFile(unitPath) + if err != nil { + t.Fatalf("read unit: %v", err) + } + if strings.Contains(string(after), "MyModule.Customer.Name") { + t.Error("the accepted writes did not reach the stored unit") + } +} diff --git a/modelsdk/mpr/writer_core.go b/modelsdk/mpr/writer_core.go index 01e090bfcd..9d9e0b23cb 100644 --- a/modelsdk/mpr/writer_core.go +++ b/modelsdk/mpr/writer_core.go @@ -526,6 +526,9 @@ func (w *Writer) insertUnit(unitID, containerID, containmentName, unitType strin if err := canon.DuplicateElementIDError(unitID, contents); err != nil { return err } + if err := canon.BareAttributeRefError(unitID, contents); err != nil { + return err + } // Convert UUID strings to 16-byte blobs for database unitIDBlob := uuidToBlob(unitID) @@ -610,6 +613,12 @@ func (w *Writer) updateUnit(unitID string, contents []byte, opts ...canon.Option if err := canon.DuplicateElementIDError(unitID, contents); err != nil { return err } + // Same reasoning, same place: a bare DomainModels$AttributeRef makes the + // project unloadable, and ALTER PAGE's patches reach here without passing + // the page encoder that also refuses one (canon/attributeref.go). + if err := canon.BareAttributeRefError(unitID, contents); err != nil { + return err + } // Session mode: divert to in-memory buffer and skip all disk/SQLite work. // The caller (e.g. ImportProject) is responsible for flushing later. From 67e31d29110fa32012151a20a1a57c535b9e7ed5 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 14:00:22 +0000 Subject: [PATCH 39/47] fix(visitor): store text expressions with their whitespace MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four places stored an expression as TEXT via ANTLR's ctx.GetText(), which joins the tokens without the whitespace between them. Literals, `+` and a lone $currentObject survive that, so the common cases looked right; a keyword operator fuses with its neighbours, and the fused text was written into the model: action: microflow GT.ACT(Flag: true and false) -> "trueandfalse" action: microflow GT.ACT(Mode: if true then 'a' else 'b') -> "iftruethen'a'else'b'" send rest request ... with ($OrderId = if $x then $id else 'none') -> "if$xthen$idelse'none'" Measured by decoding the units on a copy of ako/TestApp (11.14.0); check -p --references passed on all of it. Also affected: contentparams values and a dynamic `execute database query`. expressionSourceText takes the author's text - whitespace kept, MDL comments stripped - the same extraction the microflow expression sites moved to for the comment-leak fix, which missed these four because they live in page and REST code. ruleSourceText (the OData and widget expression slots) now shares it, so a comment inside those expressions is stripped too. Tests failed first with the fused values (`$aand$b`, `if$xthen'a'else'b'`, `if$xthen1else2`); the single-token control `$currentObject` passed throughout. After the fix the same TestApp run stores "true and false", "if true then 'a' else 'b'" and "if $x then $id else 'none'". Found while resolving PROPOSAL_first_class_expressions.md §5.1. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01P65SqmwwvbWdJVRwhYiMQw --- .../fix-issue/findings/mdl-visitor.jsonl | 1 + CHANGELOG.md | 1 + .../PROPOSAL_first_class_expressions.md | 6 +- .../expression-text-keyword-operators.mdl | 28 ++++ .../visitor_expression_source_text_test.go | 128 ++++++++++++++++++ mdl/visitor/visitor_helpers.go | 18 +++ mdl/visitor/visitor_microflow_actions.go | 4 +- mdl/visitor/visitor_odata_expression.go | 7 +- mdl/visitor/visitor_page_v3.go | 4 +- 9 files changed, 187 insertions(+), 10 deletions(-) create mode 100644 mdl-examples/bug-tests/expression-text-keyword-operators.mdl create mode 100644 mdl/visitor/visitor_expression_source_text_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-visitor.jsonl b/.claude/skills/fix-issue/findings/mdl-visitor.jsonl index aec27566e8..7fd1ec2b81 100644 --- a/.claude/skills/fix-issue/findings/mdl-visitor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-visitor.jsonl @@ -34,3 +34,4 @@ {"date": "2026-09-24", "area": "mdl-visitor", "symptom": "upstream #1174: DESCRIBE printed a view whose join is `AS ROLE`, and exec'ing that output failed — `line 7:62 mismatched input 'ROLE' expecting IDENTIFIER` then a knock-on `mismatched input ')' expecting {SELECT, HAVING}`. The model was valid (mx check 0 errors on 11.12.1)", "cause": "`tableReference` / `joinClause` took the source alias as `AS? IDENTIFIER`, and `associationPath`'s leading alias as `IDENTIFIER`, so any MDL keyword (ROLE, STATUS, VALUE…) was refused although OQL does not reserve it. The select alias already accepted `keyword`; the source alias had never been given the same", "file": "`mdl/grammar/domains/MDLCatalog.g4` (new `oqlSourceAlias`: `AS (IDENTIFIER | keyword) | IDENTIFIER`; `associationPath` leading `(IDENTIFIER | keyword)`); `mdl/visitor/visitor_entity.go` (`oqlSourceAliasText`); tests `mdl/visitor/oql_keyword_alias_test.go`; example `mdl-examples/bug-tests/1174-oql-keyword-source-alias.mdl`", "insight": "**Accept the keyword only after an explicit AS.** Copying `selectAlias`'s `IDENTIFIER | keyword` into `AS? …` would let `from M.Sale s left join …` read LEFT as the alias — the bare form has to stay IDENTIFIER-only, and the test keeps a control for exactly that. MDL's keyword list is not OQL's: rule 4 of `mxcli syntax domain-model.view-entity.oql` (rename a reserved alias) is about OQL's own words (Month, Year) and does not apply to an MDL-only keyword, so do not send the user to rename. Measured end-to-end on 11.12.1: exec + `mx check` 0 errors, DESCRIBE prints `AS ROLE`, and the described text re-parses and execs. Two gaps met on the way, both independent of the keyword and filed separately: an uppercase `AS` on an association-path join leaves that alias unresolved in `extractAliasMap` (case-sensitive `TrimSuffix(path, \"as\")`), so its columns get no type check — first misread as 'System entity lengths are not checked' until bisecting the query text against a fresh project (ako/mxcli#652); and describe -> exec adds 2 spaces to OQL lines 2..n on EVERY cycle — re-running the same described file said Unchanged and hid it, only describing again between runs shows the drift (ako/mxcli#653)", "refs": ["mendixlabs/mxcli#1174", "ako/mxcli#652", "ako/mxcli#653"]} {"area": "mdl/visitor", "date": "2026-09-24", "symptom": "`datagrid dg (DataSource: Mod.Car, …)` — the bare-entity shorthand — passed `check` (no project), `exec --no-check` printed \"Created page\", `describe page` showed `datagrid dg (onClick: …)` with the source gone, and mxbuild 11.14.0 reported CE0488 \"No entity configured for the data source of this widgets container\" + CE1571 + two column-attribute errors on that one grid. With `-p`, the reference pass refused it instead, but with the misleading \"Attribute 'Name' is bound but there is no enclosing data container providing entity context\" on a grid whose source the script did name.", "cause": "Every dataSourceExprV3 alternative starts with a keyword (DATABASE/MICROFLOW/…) or a VARIABLE, so `DataSource: Mod.Car` matched none and fell through to the generic `keyword COLON propertyValueV3` branch at the end of widgetPropertyV3 (DATASOURCE is in `keyword`). The visitor stored the string \"Mod.Car\"; GetDataSource() only type-asserts *ast.DataSourceV3, so nothing downstream saw it.", "file": "`mdl/visitor/visitor_page_v3.go` (bareEntityDataSource, bareEntityWidgets)", "insight": "Resolve the shorthand in the VISITOR, keyed on widget type (datagrid/listview/gallery/dataview), not as a grammar alternative. The first cut added `DATASOURCE COLON qualifiedName` to widgetPropertyV3 and broke the Barcode Scanner: a pluggable widget's generic keys are its own .mpk keys, and its `datasource: Module.Entity.Code` binds an ATTRIBUTE (Image's `datasource` is an enum) — only the widget type says what the key means. Grep `\"propertyKey\": \"datasource\"` in modelsdk/widgets/definitions before giving a common word a meaning. The tell for the class is a `map[string]any` property whose readers type-assert: a value of the wrong Go type is invisible rather than wrong. Side effects worth knowing: a data view with the shorthand (maint2-editable-never-create-page.mdl had one, unbound and unnoticed) is now refused as MDL-WIDGET09 instead of written unbound; `-p` reference checking on main already refused the grid case but blamed the column ('no enclosing data container'), a downstream symptom reading as user error; ALTER `set DataSource = M.E` was never silent (refuses 'must be a datasource expression'). Control: pre-fix binary + `exec --no-check` on a fresh 11.14.0 app reproduced the issue's four mxbuild errors verbatim; fixed binary, same script, 0 errors.", "refs": ["ako/mxcli#576", "ako/mxcli#552"], "ce": ["CE0488", "CE1571"]} {"area":"mdl-visitor","date":"2026-09-24","refs":["#653"],"symptom":"describe entity on a view entity, exec'd back, was never idempotent: \"Each cycle reports `Modified view entity` and stores the query with every line after the first indented two spaces further.\" Re-exec of the SAME described file reported Unchanged, so it looked stable until you described again","cause":"The two directions did not mirror: describe (cmd_entities_describe.go, and cmd_diff_mdl.go) prefixes two spaces to every stored OQL line; the visitor stored extractOriginalText(oqlCtx), which starts at the query's first token, so line 1 lost its indentation and lines 2…n kept all of it — +2 per cycle","file":"`mdl/visitor/visitor_entity.go` (dedentOQL, leadingLineWhitespace); tests `mdl/visitor/visitor_view_entity_oql_indent_test.go`, `mdl/executor/view_entity_oql_roundtrip_test.go`; bug-test `mdl-examples/bug-tests/653-view-entity-oql-indent-drift.mdl`","insight":"**Any verbatim-source capture that starts at the first token is asymmetric**: line 1 is dedented for free, the continuation lines are not. Fix it on the way IN (exec), not by making describe emit less: strip the common leading-whitespace prefix of the non-blank lines, counting line 1 at its column when only whitespace precedes it (read it from the input stream, start.GetStart()-GetColumn()). Compare prefixes byte-wise, not by width, so a Studio Pro query indented with tabs comes back byte-identical under describe's two spaces. A round-trip test must run describe → exec at least twice AND start from stored text mxcli did not write (flat, tabs, blank lines, comments): one pass from a script is exactly how this went unnoticed. Control: stubbing dedentOQL to return raw fails every round-trip case with lines 2…n two spaces deeper; a real 11.12.1 project with the old binary printed Modified ×3 with growing indent, the fixed one Unchanged ×3. Separate, not fixed here: a comment AFTER the query's last token is outside the captured span and is dropped on exec"} +{"area": "mdl/visitor", "date": "2026-09-25", "symptom": "A page action's microflow argument `Flag: true and false` was stored as the expression \"trueandfalse\", `Mode: if true then 'a' else 'b'` as \"iftruethen'a'else'b'\", and a REST call parameter `$OrderId = if $x then $id else 'none'` as \"if$xthen$idelse'none'\" (measured by decoding the units on a copy of ako/TestApp). `check -p --references` passed; describe printed the fused text back", "cause": "Four sites stored an expression as text via ANTLR's ctx.GetText(), which concatenates tokens without the hidden-channel whitespace: microflowArgV3 values (page/nanoflow call arguments), contentparams values, send-rest-request WITH parameters, and a dynamic `execute database query`. Literals, `+` and a lone $currentObject are single tokens or need no spacing, so every common case looked correct", "file": "`mdl/visitor/visitor_helpers.go` (`expressionSourceText`), `mdl/visitor/visitor_page_v3.go` (`buildMicroflowArgV3`, `buildParamAssignmentV3`), `mdl/visitor/visitor_microflow_actions.go` (dynamic query, send rest params)", "insight": "**GetText() on an expression context is always a bug** — grep `Expression().*GetText()` / `expr.GetText()` in mdl/visitor; each hit either builds the AST or must use expressionSourceText (whitespace kept, MDL comments stripped). The earlier comment-leak fix moved the six microflow sites to extractExpressionText and missed these four because they lived in page/REST code, not microflow statements: fix a text-extraction defect by searching for the call, not the feature. The tell that hid it: tests used `$currentObject` and `'a' + 'b'`, both immune; a probe with `and` / `if…then` exposed all four at once. Found while writing PROPOSAL_first_class_expressions.md §5.1. Tests `visitor_expression_source_text_test.go`", "refs": ["mendixlabs/mxcli#750"]} diff --git a/CHANGELOG.md b/CHANGELOG.md index 3ac4a90451..5382dde80a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed +- **Keyword operators were fused in some stored expressions** — a page action's microflow argument `Flag: $a and $b` was stored as `$aand$b`, `if $x then 'a' else 'b'` as `if$xthen'a'else'b'`; the same in `contentparams` values, `send rest request … with (…)` parameters and a dynamic `execute database query`. Those four places took the expression's text without its whitespace; literals, `+` and `$currentObject` were unaffected, which is why it went unnoticed, and `check --references` passed. Measured by decoding the stored units on a Mendix 11.14.0 project. The expression is now stored as written, with MDL comments removed, as the microflow expression sites already did. - **An expression property written in brackets was silently dropped** (mendixlabs/mxcli#750) — `dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ]`, the spelling #750 proposes, parsed as a list that no writer reads: `check` was clean, `exec` said `Created page`, and the widget was stored with no dynamic class. `alter page … set DynamicClasses = [ … ]` said `Altered page` and changed nothing, and a column's `DynamicCellClass` stored the list's text — tokens fused, `[if$x/Ythen'a'else'b']` — as its expression. Measured on a copy of a Mendix 11.14.0 project with the pre-fix binary. `mxcli check` now reports **MDL-WIDGET32** for `DynamicClasses` and `DynamicCellClass` written as a list (no project needed), and ALTER refuses it, so `check -p` reports that too. Write the expression quoted. - **`describe odata client` lost a quote level on a literal credential** — Studio Pro stores a literal user name as the expression `'abc'`, quotes included. `describe` printed `HttpUsername: 'abc'`, and re-executing that output stored `abc`, an identifier. `ClientCertificate`, header keys, `Version`, `MetadataUrl` and `Folder` were printed unescaped and did not re-parse when they held a quote. Every value is now quoted so a re-exec stores exactly what was read; measured against a Studio Pro-authored client decoded before and after a round trip. - **An OData client's proxy constant written `@Module.Const` was stored with the `@`** — `ProxyHost` / `ProxyPort` / `ProxyUsername` / `ProxyPassword` are by-name references to a constant, and Studio Pro stores the bare name (with `ProxyType: Override`). `"@Module.Const"` named no constant, so the proxy resolved to nothing. `create`, `create or modify` and `alter` now store the bare name for the bare, `@` and quoted-`@` spellings. The constant may be a String or an Integer. diff --git a/docs/11-proposals/PROPOSAL_first_class_expressions.md b/docs/11-proposals/PROPOSAL_first_class_expressions.md index 220c2d1c4c..dd645a198a 100644 --- a/docs/11-proposals/PROPOSAL_first_class_expressions.md +++ b/docs/11-proposals/PROPOSAL_first_class_expressions.md @@ -211,8 +211,10 @@ proposals compose rather than compete. tokens *without* the hidden-channel whitespace — `if $x then 'a' else ''` becomes `if$xthen'a'else''`. Literals survive (one token each); keywords and operators fuse. Every new slot must go through `buildExpression` → - `expressionToString`, never `GetText()`. Whether the microflow-argument path - is live-broken for `if`-expressions is untested. + `expressionToString`, never `GetText()`. *(Resolved 2026-09-25: it was live — + page microflow arguments, contentparams, send-rest-request parameters and a + dynamic database query stored the fused text. All four now use + `expressionSourceText`.)* 2. **Does `expressionToString` round-trip Studio Pro's spelling?** A stored `if $currentObject/Featured then 'x' else ''` re-emitted through diff --git a/mdl-examples/bug-tests/expression-text-keyword-operators.mdl b/mdl-examples/bug-tests/expression-text-keyword-operators.mdl new file mode 100644 index 0000000000..57f2a7a104 --- /dev/null +++ b/mdl-examples/bug-tests/expression-text-keyword-operators.mdl @@ -0,0 +1,28 @@ +-- An expression stored as TEXT lost its whitespace, fusing keyword operators. +-- +-- Four places took an expression's text with ANTLR's GetText(), which joins the +-- tokens without the whitespace between them. Literals, `+` and a lone +-- $currentObject survived, so the common cases looked fine; a keyword operator +-- fused with its neighbours and that text was written into the model. Measured +-- on a copy of ako/TestApp, pre-fix: +-- +-- action: microflow GT.ACT(Flag: true and false, …) -> "trueandfalse" +-- action: microflow GT.ACT(Mode: if true then 'a' else 'b') +-- -> "iftruethen'a'else'b'" +-- send rest request … with ($OrderId = if $x then $id else 'none') +-- -> "if$xthen$idelse'none'" +-- +-- `check -p --references` passed on all of it. Fix: expressionSourceText, the +-- same whitespace-preserving, comment-stripping extraction the microflow +-- expression sites already use. Also: contentparams values and a dynamic +-- `execute database query`. + +create microflow BugExpr.ACT ($Flag: boolean, $Mode: string) begin +end; +/ + +create page BugExpr.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + actionbutton b (caption: 'Go', action: microflow BugExpr.ACT(Flag: true and false, Mode: if true then 'a' else 'b')) + dynamictext t (content: 'x {1}', contentparams: [{1} = if true then 'a' else 'b']) +} +/ diff --git a/mdl/visitor/visitor_expression_source_text_test.go b/mdl/visitor/visitor_expression_source_text_test.go new file mode 100644 index 0000000000..8ae15a3a31 --- /dev/null +++ b/mdl/visitor/visitor_expression_source_text_test.go @@ -0,0 +1,128 @@ +// SPDX-License-Identifier: Apache-2.0 + +package visitor + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// Four places took an expression's text with ANTLR's GetText(), which joins the +// tokens WITHOUT the whitespace between them. Literals survive (one token each) +// and so do operators like `+`, so `'a' + 'b'` and `$currentObject` looked fine +// — but a keyword operator fuses with its neighbours: `$a and $b` became +// `$aand$b`, `if $x then 'a' else 'b'` became `if$xthen'a'else'b'`. The fused +// text was written into the model as the expression (measured on a copy of +// ako/TestApp: a page action's microflow argument stored "trueandfalse", a REST +// call parameter "if$xthen$idelse'none'"), while check -p --references passed. +// +// Same class as the MDL-comment leak (stripMDLComments): those six microflow +// sites were moved to extractExpressionText, and these four were missed. + +func buildExprSrc(t *testing.T, src string) *ast.Program { + t.Helper() + prog, errs := Build(src) + if len(errs) > 0 { + t.Fatalf("parse %q: %v", src, errs) + } + return prog +} + +func TestExpressionSourceText_PageMicroflowArguments(t *testing.T) { + prog := buildExprSrc(t, `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + actionbutton b (caption: 'Go', action: microflow M.ACT(Flag: $a and $b, Mode: if $x then 'a' else 'b', Obj: $currentObject)) +}`) + btn := prog.Statements[0].(*ast.CreatePageStmtV3).Widgets[0] + act, ok := btn.Properties["Action"].(*ast.ActionV3) + if !ok { + t.Fatalf("no action on the button: %#v", btn.Properties) + } + want := map[string]string{"Flag": "$a and $b", "Mode": "if $x then 'a' else 'b'", "Obj": "$currentObject"} + for _, a := range act.Args { + if a.Value != want[a.Name] { + t.Errorf("argument %s stored %q, want %q", a.Name, a.Value, want[a.Name]) + } + } +} + +func TestExpressionSourceText_ContentParams(t *testing.T) { + prog := buildExprSrc(t, `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + dynamictext t (content: 'x {1}', contentparams: [{1} = if $x then 'a' else 'b']) +}`) + w := prog.Statements[0].(*ast.CreatePageStmtV3).Widgets[0] + params := w.GetContentParams() + if len(params) != 1 || params[0].Value != "if $x then 'a' else 'b'" { + t.Errorf("contentparams stored %#v, want the expression with its spacing", params) + } +} + +func TestExpressionSourceText_SendRestRequestParameters(t *testing.T) { + prog := buildExprSrc(t, `create microflow M.F ($x: boolean, $a: boolean, $b: boolean) begin + send rest request M.C.Op with ($Id = if $x then 1 else 2, $Flag = $a and $b); +end;`) + mf := prog.Statements[0].(*ast.CreateMicroflowStmt) + var got []ast.SendRestParamDef + for _, s := range mf.Body { + if r, ok := s.(*ast.SendRestRequestStmt); ok { + got = r.Parameters + } + } + want := map[string]string{"Id": "if $x then 1 else 2", "Flag": "$a and $b"} + if len(got) != 2 { + t.Fatalf("parameters = %#v", got) + } + for _, p := range got { + if p.Expression != want[p.Name] { + t.Errorf("parameter %s stored %q, want %q", p.Name, p.Expression, want[p.Name]) + } + } +} + +func TestExpressionSourceText_DynamicDatabaseQuery(t *testing.T) { + prog := buildExprSrc(t, `create microflow M.F ($x: boolean) begin + $r = execute database query M.C.Q dynamic if $x then 'select 1' else 'select 2'; +end;`) + mf := prog.Statements[0].(*ast.CreateMicroflowStmt) + for _, s := range mf.Body { + if q, ok := s.(*ast.ExecuteDatabaseQueryStmt); ok { + if q.DynamicQuery != "if $x then 'select 1' else 'select 2'" { + t.Errorf("dynamic query stored %q, want the expression with its spacing", q.DynamicQuery) + } + return + } + } + t.Fatal("no execute database query statement") +} + +// Control: a single-token value was never affected, and must stay exactly as is. +// And an MDL comment between operands must not end up inside the expression. +func TestExpressionSourceText_ControlsAndComments(t *testing.T) { + prog := buildExprSrc(t, `create page M.P (title: 'P', layout: Atlas_Core.Atlas_Default) { + actionbutton b (caption: 'Go', action: microflow M.ACT(Obj: $currentObject, N: 'a' + 'b', C: $a -- why + and $b)) +}`) + act := prog.Statements[0].(*ast.CreatePageStmtV3).Widgets[0].Properties["Action"].(*ast.ActionV3) + want := map[string]string{"Obj": "$currentObject", "N": "'a' + 'b'", "C": "$a \n and $b"} + for _, a := range act.Args { + if a.Name == "C" { + v, _ := a.Value.(string) + if v == "$aand$b" || v == "" || containsComment(v) { + t.Errorf("argument C stored %q: fused or carrying the comment", a.Value) + } + continue + } + if a.Value != want[a.Name] { + t.Errorf("argument %s stored %q, want %q", a.Name, a.Value, want[a.Name]) + } + } +} + +func containsComment(s string) bool { + for i := 0; i+1 < len(s); i++ { + if s[i] == '-' && s[i+1] == '-' { + return true + } + } + return false +} diff --git a/mdl/visitor/visitor_helpers.go b/mdl/visitor/visitor_helpers.go index e4c557eb03..00ae92abf8 100644 --- a/mdl/visitor/visitor_helpers.go +++ b/mdl/visitor/visitor_helpers.go @@ -755,3 +755,21 @@ func buildErrorMessage(ctx parser.IErrorMessageClauseContext) string { } return unquoteString(emc.STRING_LITERAL().GetText()) } + +// expressionSourceText is an expression as the author wrote it — whitespace kept, +// MDL comments removed — for the places that store an expression as TEXT rather +// than building its AST. Never use ctx.GetText() for this: it joins the tokens +// without the whitespace between them, so literals and `+` survive but a keyword +// operator fuses with its neighbours (`$a and $b` -> `$aand$b`, `if $x then 'a'` +// -> `if$xthen'a'`), and that fused text is what reaches the model. +func expressionSourceText(expr parser.IExpressionContext) string { + if expr == nil { + return "" + } + if prc, ok := expr.(antlr.ParserRuleContext); ok { + if source := strings.TrimSpace(extractExpressionText(prc)); source != "" { + return source + } + } + return expr.GetText() +} diff --git a/mdl/visitor/visitor_microflow_actions.go b/mdl/visitor/visitor_microflow_actions.go index d8b7813626..0df0fd391b 100644 --- a/mdl/visitor/visitor_microflow_actions.go +++ b/mdl/visitor/visitor_microflow_actions.go @@ -624,7 +624,7 @@ func buildExecuteDatabaseQueryStatement(ctx parser.IExecuteDatabaseQueryStatemen } else if ds := execCtx.DOLLAR_STRING(); ds != nil { stmt.DynamicQuery = unquoteDollarString(ds.GetText()) } else if expr := execCtx.Expression(); expr != nil { - stmt.DynamicQuery = expr.GetText() + stmt.DynamicQuery = expressionSourceText(expr) stmt.DynamicQueryIsExpression = true } } @@ -1661,7 +1661,7 @@ func buildSendRestRequestStatement(ctx parser.ISendRestRequestStatementContext) param.Name = strings.TrimPrefix(v.GetText(), "$") } if expr := pc.Expression(); expr != nil { - param.Expression = expr.GetText() + param.Expression = expressionSourceText(expr) } stmt.Parameters = append(stmt.Parameters, param) } diff --git a/mdl/visitor/visitor_odata_expression.go b/mdl/visitor/visitor_odata_expression.go index 51c41411ba..cc2bdea3be 100644 --- a/mdl/visitor/visitor_odata_expression.go +++ b/mdl/visitor/visitor_odata_expression.go @@ -34,11 +34,10 @@ func ruleSourceText(ctx antlr.ParserRuleContext) string { if ctx == nil { return "" } - start, stop := ctx.GetStart(), ctx.GetStop() - if start == nil || stop == nil || stop.GetStop() < start.GetStart() { - return ctx.GetText() + if source := strings.TrimSpace(extractExpressionText(ctx)); source != "" { + return source } - return start.GetInputStream().GetText(start.GetStart(), stop.GetStop()) + return ctx.GetText() } // odataExpressionValue returns the expression an OData client expression diff --git a/mdl/visitor/visitor_page_v3.go b/mdl/visitor/visitor_page_v3.go index 238b0bbc34..5dfddafe67 100644 --- a/mdl/visitor/visitor_page_v3.go +++ b/mdl/visitor/visitor_page_v3.go @@ -1198,7 +1198,7 @@ func buildMicroflowArgV3(ctx parser.IMicroflowArgV3Context) ast.FlowArgV3 { arg.Name = identifierOrKeywordText(iok) } if expr := argCtx.Expression(); expr != nil { - arg.Value = expr.GetText() + arg.Value = expressionSourceText(expr) } return arg @@ -1312,7 +1312,7 @@ func buildParamAssignmentV3(ctx parser.IParamAssignmentV3Context) ast.ParamAssig } } if expr := paCtx.Expression(); expr != nil { - param.Value = stripExpressionIdentifierQuotes(expr.GetText()) + param.Value = stripExpressionIdentifierQuotes(expressionSourceText(expr)) } if fmtCtx := paCtx.ParamFormatV3(); fmtCtx != nil { param.Format = buildParamFormatV3(fmtCtx) From 51b36dc04ffe2de2c73466392642091d5cb93cbf Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 14:02:47 +0000 Subject: [PATCH 40/47] fix(describe): name the datasource key on a widget declaring several (#1199) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DESCRIBE of a File Uploader page printed a generic `DataSource:` clause, which exec then refused: "widget `upFiles` (fileuploader) exposes 2 datasources, so a generic `datasource:` clause is ambiguous". DESCRIBE counted CONFIGURED datasources (files mode leaves associatedImages unset, so one), while the builder counts DECLARED datasource mappings (two). DESCRIBE now reads the declared, non-linked datasource properties from the stored schema and emits the named key (`associatedFiles: …`) when there are several. Widgets with an embedded, mode-based definition (ComboBox) keep the generic clause the builder accepts for them. The #956 File Uploader bug-test authored the same refused generic clause; it now uses associatedFiles / associatedImages. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01H79ZLyf5E4LuV48TmbDdq9 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + ...fileuploader-describe-named-datasource.mdl | 43 +++++++ .../956-fileuploader-six-action-slots.mdl | 4 +- mdl/executor/cmd_pages_describe_parse.go | 9 ++ mdl/executor/cmd_pages_describe_pluggable.go | 55 ++++++++ ..._pages_describe_schema_multisource_test.go | 117 ++++++++++++++++++ 6 files changed, 227 insertions(+), 2 deletions(-) create mode 100644 mdl-examples/bug-tests/1199-fileuploader-describe-named-datasource.mdl create mode 100644 mdl/executor/cmd_pages_describe_schema_multisource_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index ed70b75f46..932a800e71 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -706,3 +706,4 @@ {"area": "mdl/executor", "symptom": "describe → exec of Feedback v4.0.2's EXCLUDED FeedbackModule.ShareFeedback_Logo refused `nanoflow not found: FeedbackModule.DS_FeedbackForm (data source)`; forcing it through left a project `mx check` could not LOAD (ArgumentNullException setting 'Attribute')", "cause": "The data view's flow is not in the project, so DESCRIBE had no context entity and printed every binding inside it bare (`Attribute: Subject`, `{1} = ImageB64`, `Visible: _showEmail in (…)`); exec had nothing to qualify them against, and a bare DomainModels$AttributeRef makes Mendix's loader throw. The stored model always had the full names", "file": "`mdl/executor/cmd_pages_describe_flowcontext.go` (`withQualifiedAttrs`, `describeAttr`), `mdl/executor/validate.go` (`unscopedBindings`), `mdl/executor/cmd_pages_builder_v3.go` (dangling DS flow kept by name), `mdl/backend/modelsdk/page_bare_attributeref.go` (`refuseBareAttributeRefs`)", "insight": "**The information was never lost — DESCRIBE threw it away**: every AttributeRef inside the unresolvable container still stored Module.Entity.Attr; shortening to the bare name is only safe where the reader can re-derive the entity, so key the shortening on whether the context resolved, not on habit. Measure the loader's tolerance before adding a write guard: 72 of 72 Studio Pro AttributeRefs in the project are qualified, so refusing a bare one refuses only writes that were already fatal — and turns a load-time stack trace into a statement-level error naming the attribute. Pair a blanket AST-level refusal with the exact failing slots (Attribute, CaptionAttribute, Visible-in, *Params) so the refusal names the widget, and keep the writer guard as the net for slots the walk does not know. Forced-fault run: hand-edit one qualified binding back to bare and confirm the refusal names it", "refs": ["ako/mxcli#675", "FeedbackModule.ShareFeedback_Logo"], "date": "2026-09-25"} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_New fails `mx check` with [CE0642] \"Property 'Caption' is required.\" at Combo box 'comboBox2' (Account_Edit comboBox4 too); check and exec report success", "cause": "The ComboBox caption is an EXPRESSION (optionsSourceAssociationCaptionType=expression, optionsSourceAssociationCaptionExpression='$currentObject/Description'); describe read only the attribute caption, so the widget was rewritten with none. The write side already worked via the explicit-property pass, but MDL-WIDGET06 claimed both keys 'will be dropped'", "file": "`mdl/executor/cmd_pages_describe_parse.go` (combobox branch), `cmd_pages_describe_output.go`, `validate_widget_explicit_writable.go` (`persistedByExplicitPass`), `validate_widgets.go` (MDL-WIDGET06)", "insight": "**Resolve pluggable-widget properties by key before theorising**: a 20-line script mapping each Property's TypePointer to its PropertyKey through the widget's own Type.ObjectType settled the cause in one run, where the ndsl dump shows only anonymous values. Then TEST THE WRITE SIDE before building one: adding the two storage keys to the describe output by hand persisted both and built clean, which shrank the fix to describe + a false warning — no new syntax. A validator rule that asserts 'not persisted' must be tied to what the write path handles (here: the explicit pass writes Expression/TextTemplate/Attribute and scalar types), or it goes stale when the writer grows; the #643 test pinned the stale claim. A dedicated extractor for a 'known' pluggable widget silently drops every property it does not map — generic widgets emit them, known ones do not", "refs": ["ako/mxcli#664", "ako/mxcli#643"], "ce": ["CE0642"], "rules": ["MDL-WIDGET06"], "date": "2026-09-25"} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`mxcli check` passed a microflow with a commit inside a loop \u2014 one database round trip per iteration \u2014 that `mxcli lint` already flagged as CONV011. The defect surfaced only at project-wide lint time, long after the write.", "cause": "CONV011 reads the STORED model, so it cannot speak until `exec` has written the microflow; `check` reads the MDL and had no equivalent rule. The gap is temporal, not a missing capability on either side.", "file": "`mdl/executor/validate_commit_in_loop.go` (MDL-PERF01, hooked in `validate_microflow.go`), test `validate_commit_in_loop_test.go`, example `mdl-examples/bug-tests/1186-commit-in-loop.mdl`", "insight": "**When adding a check-time rule that anticipates an existing lint rule, pin the BOUNDARY to the lint rule's, not to the better one, and say why in the code.** A `while true` is built as an ExclusiveMerge back-edge rather than a LoopedActivity, so CONV011 (which walks LoopedActivity) does not flag a commit inside one. A commit there is arguably still N+1, and the tempting move is to be more correct \u2014 but two rules for one concept that disagree on what counts is precisely how a pair drifts, and this repo already has CONV010's three successive short allowlists as the worked example. If the case is worth reporting it is worth reporting in BOTH, and the stored-model rule is the one that sees the built flow. The test that pins this carries a control on the control: `while true` is exempt, `while ` is not, so the exemption cannot silently become 'never flag a while'. Name the sibling rule in the message (`lint reports this as CONV011`) so a reader hitting one recognises the other rather than filing it twice. Also worth reusing: `checkReturnInLoop` already had the depth-tracking walk over Loop/While/If/EnumSplit/InheritanceSplit \u2014 copying its shape got the nesting cases right for free.", "refs": ["ako/mxcli#681", "mendixlabs/mxcli#1186"], "rules": ["MDL-PERF01"]} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "`describe page` on a File Uploader (files mode) emits `DataSource: association …`, and exec of that output fails: \"widget `upFiles` (fileuploader) exposes 2 datasources, so a generic `datasource:` clause is ambiguous — name the one you mean: associatedFiles, associatedImages\"", "cause": "DESCRIBE chose generic vs named by counting CONFIGURED datasources (namedCustomWidgetDataSources drops unset ones, so files mode = 1), while the builder's refuseAmbiguousGenericDataSource counts DECLARED datasource mappings (a generated def maps every top-level datasource = 2). Two sides of one round trip deciding the same question from different evidence.", "file": "mdl/executor/cmd_pages_describe_parse.go", "insight": "When describe and build each decide 'is this ambiguous?', they must count the same set. Fix read the DECLARED count from the stored schema (PropertyTypes with ValueType.Type=DataSource, excluding IsLinked) and excluded widgets with an embedded .def.json — those are hand-written and pick one datasource mapping per mode, so a database-mode ComboBox (two declared) must keep the generic clause. The #956 bug-test script itself authored the refused generic clause on a File Uploader: a bug-test that only runs `mxcli check` without a project can't see an exec-time refusal, so grep bug-tests for the old spelling whenever a builder starts refusing one.", "refs": ["mendixlabs/mxcli#1199", "mendixlabs/mxcli#956", "mendixlabs/mxcli#1109"], "rules": []} diff --git a/mdl-examples/bug-tests/1199-fileuploader-describe-named-datasource.mdl b/mdl-examples/bug-tests/1199-fileuploader-describe-named-datasource.mdl new file mode 100644 index 0000000000..874d671142 --- /dev/null +++ b/mdl-examples/bug-tests/1199-fileuploader-describe-named-datasource.mdl @@ -0,0 +1,43 @@ +-- #1199: DESCRIBE of a File Uploader page did not re-execute. +-- +-- Reported (mxcli 0.24.0, Mendix 11.12.2, File Uploader 2.5.0): the output of +-- `describe page` on a File Uploader in files mode, fed back to `exec`, fails: +-- +-- widget `upFiles` (fileuploader) exposes 2 datasources, so a generic +-- `datasource:` clause is ambiguous — name the one you mean: +-- associatedFiles, associatedImages +-- +-- DESCRIBE chose the generic clause by counting CONFIGURED datasources (one: +-- `associatedImages` is unset in files mode); the builder refuses it by counting +-- DECLARED ones (two). DESCRIBE now counts declared, authorable (non-linked) +-- datasources too, so the widget below describes back as +-- `associatedFiles: association Uploads.UploadedFile_UploadRequest`. +-- +-- Round trip: exec this, then +-- mxcli -p app.mpr -c 'describe page Uploads.UploadRequest_NewEdit' > rt.mdl +-- mxcli exec rt.mdl -p app.mpr -- must succeed, not refuse as ambiguous + +create module Uploads; +create module role Uploads.User; + +create persistent entity Uploads.UploadRequest ( Title: string(100) ); +create persistent entity Uploads.UploadedFile extends System.FileDocument (); +create association Uploads.UploadedFile_UploadRequest from Uploads.UploadedFile to Uploads.UploadRequest; + +create nanoflow Uploads.NF_CreateFile ( $Request: Uploads.UploadRequest ) begin end + +create or replace page Uploads.UploadRequest_NewEdit (Title: 'Upload', Layout: Atlas_Core.Atlas_Default, + Params: { $Request: Uploads.UploadRequest }) +{ + DATAVIEW dv (DataSource: $Request) { + pluggablewidget 'com.mendix.widget.web.fileuploader.FileUploader' upFiles ( + uploadMode: 'files', + associatedFiles: association Uploads.UploadedFile_UploadRequest, + createFileAction: nanoflow Uploads.NF_CreateFile($Request = $Request) + ) + } +} + +grant view on page Uploads.UploadRequest_NewEdit to Uploads.User; + +describe page Uploads.UploadRequest_NewEdit; diff --git a/mdl-examples/bug-tests/956-fileuploader-six-action-slots.mdl b/mdl-examples/bug-tests/956-fileuploader-six-action-slots.mdl index b8c74a1030..7f21a423e0 100644 --- a/mdl-examples/bug-tests/956-fileuploader-six-action-slots.mdl +++ b/mdl-examples/bug-tests/956-fileuploader-six-action-slots.mdl @@ -78,7 +78,7 @@ create or replace page FU.P_Files (Title: 'Files', Layout: Atlas_Core.Atlas_Defa DATAVIEW dv (DataSource: $Ctx) { pluggablewidget 'com.mendix.widget.web.fileuploader.FileUploader' fu ( uploadMode: 'files', - datasource: association FU.MyFile_UploadCtx, + associatedFiles: association FU.MyFile_UploadCtx, createFileAction: nanoflow FU.NF_CreateFile($Ctx = $Ctx), onUploadSuccessFile: microflow FU.ACT_FileUploaded($Ctx = $Ctx), onUploadFailureFile: microflow FU.ACT_FileFailed @@ -93,7 +93,7 @@ create or replace page FU.P_Images (Title: 'Images', Layout: Atlas_Core.Atlas_De DATAVIEW dv (DataSource: $Ctx) { pluggablewidget 'com.mendix.widget.web.fileuploader.FileUploader' fu ( uploadMode: 'images', - datasource: association FU.MyImage_UploadCtx, + associatedImages: association FU.MyImage_UploadCtx, createImageAction: nanoflow FU.NF_CreateImage($Ctx = $Ctx), onUploadSuccessImage: nanoflow FU.NF_UploadedImage($Ctx = $Ctx), onUploadFailureImage: microflow FU.ACT_ImageFailed($Ctx = $Ctx) diff --git a/mdl/executor/cmd_pages_describe_parse.go b/mdl/executor/cmd_pages_describe_parse.go index 481d0256c7..6d9c19a56f 100644 --- a/mdl/executor/cmd_pages_describe_parse.go +++ b/mdl/executor/cmd_pages_describe_parse.go @@ -563,6 +563,15 @@ func parseRawWidget(ctx *ExecContext, w map[string]any, parentEntityContext ...s if widget.EntityContext == "" { widget.EntityContext = dataSourceEntityContext(ctx, named[0].DataSource) } + // One CONFIGURED is not one DECLARED. The builder refuses the + // generic clause on a widget whose definition maps several + // datasources, whatever is configured — a File Uploader in files + // mode still declares `associatedImages` — so the generic spelling + // made every such page's description unexecutable (#1199). Name the + // key whenever the schema declares more than one the author can set. + if named[0].Key != "" && declaresSeveralAuthorableDataSources(w) { + widget.NamedDataSources = named + } } return []rawWidget{widget} diff --git a/mdl/executor/cmd_pages_describe_pluggable.go b/mdl/executor/cmd_pages_describe_pluggable.go index 25f28226c8..78626d4354 100644 --- a/mdl/executor/cmd_pages_describe_pluggable.go +++ b/mdl/executor/cmd_pages_describe_pluggable.go @@ -5,6 +5,7 @@ package executor import ( "context" "strings" + "sync" ) // buildPropertyTypeKeyMap builds a map from PropertyType $ID to PropertyKey for a CustomWidget. @@ -1480,3 +1481,57 @@ func parseColumnSlotWidgets(ctx *ExecContext, value map[string]any, entityContex } return out } + +// declaresSeveralAuthorableDataSources reports whether a pluggable widget's +// stored schema declares more than one datasource property MDL can set — the +// same count the builder's refuseAmbiguousGenericDataSource makes, read from the +// document instead of the definition. +// +// The two agree because a generated definition (the only kind a widget like the +// File Uploader has) maps every top-level datasource the package declares. A +// LINKED one is not counted: the platform fills it from the containing widget +// and a definition may not map it. A widget with an EMBEDDED definition is not +// counted either: those are hand-written, choose one datasource mapping per +// mode, and so accept the generic clause — a database-mode ComboBox declares +// two datasources and must keep describing as it always has. +func declaresSeveralAuthorableDataSources(w map[string]any) bool { + widgetType, ok := w["Type"].(map[string]any) + if !ok { + return false + } + if id, _ := widgetType["WidgetId"].(string); embeddedDefinitionWidgetIDs()[id] { + return false + } + objType, ok := widgetType["ObjectType"].(map[string]any) + if !ok { + return false + } + n := 0 + for _, pt := range getBsonArrayElements(objType["PropertyTypes"]) { + ptMap, ok := pt.(map[string]any) + if !ok { + continue + } + vt, ok := ptMap["ValueType"].(map[string]any) + if !ok || extractString(vt["Type"]) != "DataSource" { + continue + } + if linked, _ := vt["IsLinked"].(bool); linked { + continue + } + n++ + } + return n > 1 +} + +var embeddedDefinitionWidgetIDs = sync.OnceValue(func() map[string]bool { + ids := map[string]bool{} + reg, err := NewWidgetRegistry() + if err != nil { + return ids + } + for _, def := range reg.All() { + ids[def.WidgetID] = true + } + return ids +}) diff --git a/mdl/executor/cmd_pages_describe_schema_multisource_test.go b/mdl/executor/cmd_pages_describe_schema_multisource_test.go new file mode 100644 index 0000000000..3b07f92afa --- /dev/null +++ b/mdl/executor/cmd_pages_describe_schema_multisource_test.go @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "bytes" + "context" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" +) + +// fileUploaderBSON is the shape a Studio Pro-authored File Uploader 2.5.0 is +// stored in: its schema declares TWO datasource properties, `associatedFiles` +// and `associatedImages`, and in files mode only the first is configured. +func fileUploaderBSON() map[string]any { + propType := func(id, key string) map[string]any { + return map[string]any{ + "$ID": id, + "PropertyKey": key, + "ValueType": map[string]any{"Type": "DataSource"}, + } + } + return map[string]any{ + "$Type": "CustomWidgets$CustomWidget", + "Name": "upFiles", + "Type": map[string]any{ + "WidgetId": "com.mendix.widget.web.fileuploader.FileUploader", + "ObjectType": map[string]any{"PropertyTypes": []any{ + propType("pt-files", "associatedFiles"), + propType("pt-images", "associatedImages"), + map[string]any{"$ID": "pt-mode", "PropertyKey": "uploadMode", "ValueType": map[string]any{"Type": "Enumeration"}}, + }}, + }, + "Object": map[string]any{"Properties": []any{ + map[string]any{"TypePointer": "pt-files", "Value": map[string]any{"DataSource": dbSource("Uploads.UploadedFile")}}, + map[string]any{"TypePointer": "pt-images", "Value": map[string]any{"DataSource": dsBSON(dsTypeDatabase)}}, + map[string]any{"TypePointer": "pt-mode", "Value": map[string]any{"PrimitiveValue": "files"}}, + }}, + } +} + +func describeRawWidget(t *testing.T, w map[string]any) string { + t.Helper() + var buf bytes.Buffer + ctx := (&Executor{}).newExecContext(context.Background()) + ctx.Output = &buf + raw := parseRawWidget(ctx, w) + if len(raw) != 1 { + t.Fatalf("parsed %d widgets, want 1", len(raw)) + } + outputWidgetMDLV3(ctx, raw[0], 1) + return buf.String() +} + +// #1199: a widget whose SCHEMA declares several datasources is described with +// the named key even when only one is configured. The builder refuses the +// generic clause on such a widget whatever is configured ("exposes 2 +// datasources, so a generic `datasource:` clause is ambiguous"), so the generic +// spelling made the describe output of every File Uploader page unexecutable. +func TestDescribe_SchemaMultiSourceWidgetUsesNamedKey(t *testing.T) { + got := describeRawWidget(t, fileUploaderBSON()) + if !strings.Contains(got, "associatedFiles: database from Uploads.UploadedFile") { + t.Errorf("describe did not name the configured datasource's key:\n%s", got) + } + if strings.Contains(strings.ToLower(got), "datasource:") { + t.Errorf("describe emitted a generic `DataSource:` clause the builder rejects as ambiguous:\n%s", got) + } +} + +// A linked datasource is filled by the platform and never authored, so it does +// not make a widget multi-source: one linked plus one authorable keeps the +// generic clause. +func TestDescribe_LinkedDataSourceDoesNotCountAsAuthorable(t *testing.T) { + w := fileUploaderBSON() + pts := w["Type"].(map[string]any)["ObjectType"].(map[string]any)["PropertyTypes"].([]any) + pts[1].(map[string]any)["ValueType"] = map[string]any{"Type": "DataSource", "IsLinked": true} + got := describeRawWidget(t, w) + if !strings.Contains(got, "DataSource: database from Uploads.UploadedFile") { + t.Errorf("single authorable datasource should keep the generic clause:\n%s", got) + } +} + +// A widget with a hand-written embedded definition picks its datasource +// mapping by mode, so the builder accepts the generic clause there and its +// output must not change: a database-mode ComboBox declares two datasources. +func TestDescribe_EmbeddedDefinitionWidgetKeepsGenericClause(t *testing.T) { + w := fileUploaderBSON() + w["Type"].(map[string]any)["WidgetId"] = "com.mendix.widget.web.combobox.Combobox" + if got := describeRawWidget(t, w); !strings.Contains(got, "DataSource: database from Uploads.UploadedFile") { + t.Errorf("embedded-definition widget should keep the generic clause:\n%s", got) + } +} + +// The other half of the #1199 round trip: what DESCRIBE now emits for a +// multi-source widget with ONE configured datasource — its named key, the other +// left unset — is what the builder accepts. Before, it was offered the generic +// clause and refused it as ambiguous. +func TestBuild_OneNamedDataSourceOnMultiSourceWidget(t *testing.T) { + e := multiSourceEngine(t) + w := &ast.WidgetV3{ + Name: "cb", + Type: "pluggablewidget", + Properties: map[string]any{ + "WidgetType": "com.mendix.widget.web.combobox.Combobox", + "optionsSourceDatabaseDataSource": &ast.DataSourceV3{Type: "database", Reference: "Sales.Customer"}, + }, + } + widget, err := e.Build(multiSourceDef(), w) + if err != nil { + t.Fatalf("Build: %v", err) + } + if got := renderedWidgetStrings(t, widget); !strings.Contains(got, "Sales.Customer") { + t.Error("named datasource was not applied") + } +} From 968dc4fec02a67634397c813253c012a6193b425 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 14:03:40 +0000 Subject: [PATCH 41/47] fix(check): refuse bare bindings ALTER PAGE inserts where no entity is in scope MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `alter page … { insert after textBox1 { image … ImageUrlParams: [{1} = ImageB64] } }` on Feedback's ShareFeedback_Logo, whose data view is sourced by a nanoflow the project lacks, passed `check --references`; exec then wrote a bare AttributeRef and `mx check` could not load the project. A widget inserted outside every data container fails the same way (a text box's Attribute is dropped instead: CE7005). ALTER's entity context lives in the stored document, so `check --references` now opens it (read-only, as the ALTER … SET dry run already does) and resolves the insertion point's entity through alterEntityContext — lifted out of the INSERT/REPLACE exec paths so check and exec share it. With no entity, the bare bindings of the inserted widgets are reported via bindingsWithoutScope, the walk lifted out of CREATE PAGE's unscopedBindings (#678). Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../alter-page-unscoped-insert-bindings.mdl | 76 ++++++++ mdl/executor/cmd_alter_page.go | 52 +++--- mdl/executor/validate.go | 44 +++-- mdl/executor/validate_alter_unscoped.go | 143 +++++++++++++++ mdl/executor/validate_alter_unscoped_test.go | 173 ++++++++++++++++++ 6 files changed, 452 insertions(+), 37 deletions(-) create mode 100644 mdl-examples/bug-tests/alter-page-unscoped-insert-bindings.mdl create mode 100644 mdl/executor/validate_alter_unscoped.go create mode 100644 mdl/executor/validate_alter_unscoped_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index ed70b75f46..7cd8b9254b 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -706,3 +706,4 @@ {"area": "mdl/executor", "symptom": "describe → exec of Feedback v4.0.2's EXCLUDED FeedbackModule.ShareFeedback_Logo refused `nanoflow not found: FeedbackModule.DS_FeedbackForm (data source)`; forcing it through left a project `mx check` could not LOAD (ArgumentNullException setting 'Attribute')", "cause": "The data view's flow is not in the project, so DESCRIBE had no context entity and printed every binding inside it bare (`Attribute: Subject`, `{1} = ImageB64`, `Visible: _showEmail in (…)`); exec had nothing to qualify them against, and a bare DomainModels$AttributeRef makes Mendix's loader throw. The stored model always had the full names", "file": "`mdl/executor/cmd_pages_describe_flowcontext.go` (`withQualifiedAttrs`, `describeAttr`), `mdl/executor/validate.go` (`unscopedBindings`), `mdl/executor/cmd_pages_builder_v3.go` (dangling DS flow kept by name), `mdl/backend/modelsdk/page_bare_attributeref.go` (`refuseBareAttributeRefs`)", "insight": "**The information was never lost — DESCRIBE threw it away**: every AttributeRef inside the unresolvable container still stored Module.Entity.Attr; shortening to the bare name is only safe where the reader can re-derive the entity, so key the shortening on whether the context resolved, not on habit. Measure the loader's tolerance before adding a write guard: 72 of 72 Studio Pro AttributeRefs in the project are qualified, so refusing a bare one refuses only writes that were already fatal — and turns a load-time stack trace into a statement-level error naming the attribute. Pair a blanket AST-level refusal with the exact failing slots (Attribute, CaptionAttribute, Visible-in, *Params) so the refusal names the widget, and keep the writer guard as the net for slots the walk does not know. Forced-fault run: hand-edit one qualified binding back to bare and confirm the refusal names it", "refs": ["ako/mxcli#675", "FeedbackModule.ShareFeedback_Logo"], "date": "2026-09-25"} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_New fails `mx check` with [CE0642] \"Property 'Caption' is required.\" at Combo box 'comboBox2' (Account_Edit comboBox4 too); check and exec report success", "cause": "The ComboBox caption is an EXPRESSION (optionsSourceAssociationCaptionType=expression, optionsSourceAssociationCaptionExpression='$currentObject/Description'); describe read only the attribute caption, so the widget was rewritten with none. The write side already worked via the explicit-property pass, but MDL-WIDGET06 claimed both keys 'will be dropped'", "file": "`mdl/executor/cmd_pages_describe_parse.go` (combobox branch), `cmd_pages_describe_output.go`, `validate_widget_explicit_writable.go` (`persistedByExplicitPass`), `validate_widgets.go` (MDL-WIDGET06)", "insight": "**Resolve pluggable-widget properties by key before theorising**: a 20-line script mapping each Property's TypePointer to its PropertyKey through the widget's own Type.ObjectType settled the cause in one run, where the ndsl dump shows only anonymous values. Then TEST THE WRITE SIDE before building one: adding the two storage keys to the describe output by hand persisted both and built clean, which shrank the fix to describe + a false warning — no new syntax. A validator rule that asserts 'not persisted' must be tied to what the write path handles (here: the explicit pass writes Expression/TextTemplate/Attribute and scalar types), or it goes stale when the writer grows; the #643 test pinned the stale claim. A dedicated extractor for a 'known' pluggable widget silently drops every property it does not map — generic widgets emit them, known ones do not", "refs": ["ako/mxcli#664", "ako/mxcli#643"], "ce": ["CE0642"], "rules": ["MDL-WIDGET06"], "date": "2026-09-25"} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`mxcli check` passed a microflow with a commit inside a loop \u2014 one database round trip per iteration \u2014 that `mxcli lint` already flagged as CONV011. The defect surfaced only at project-wide lint time, long after the write.", "cause": "CONV011 reads the STORED model, so it cannot speak until `exec` has written the microflow; `check` reads the MDL and had no equivalent rule. The gap is temporal, not a missing capability on either side.", "file": "`mdl/executor/validate_commit_in_loop.go` (MDL-PERF01, hooked in `validate_microflow.go`), test `validate_commit_in_loop_test.go`, example `mdl-examples/bug-tests/1186-commit-in-loop.mdl`", "insight": "**When adding a check-time rule that anticipates an existing lint rule, pin the BOUNDARY to the lint rule's, not to the better one, and say why in the code.** A `while true` is built as an ExclusiveMerge back-edge rather than a LoopedActivity, so CONV011 (which walks LoopedActivity) does not flag a commit inside one. A commit there is arguably still N+1, and the tempting move is to be more correct \u2014 but two rules for one concept that disagree on what counts is precisely how a pair drifts, and this repo already has CONV010's three successive short allowlists as the worked example. If the case is worth reporting it is worth reporting in BOTH, and the stored-model rule is the one that sees the built flow. The test that pins this carries a control on the control: `while true` is exempt, `while ` is not, so the exemption cannot silently become 'never flag a while'. Name the sibling rule in the message (`lint reports this as CONV011`) so a reader hitting one recognises the other rather than filing it twice. Also worth reusing: `checkReturnInLoop` already had the depth-tracking walk over Loop/While/If/EnumSplit/InheritanceSplit \u2014 copying its shape got the nesting cases right for free.", "refs": ["ako/mxcli#681", "mendixlabs/mxcli#1186"], "rules": ["MDL-PERF01"]} +{"area":"mdl/executor","date":"2026-09-25","symptom":"`alter page FeedbackModule.ShareFeedback_Logo { insert after textBox1 { image zzImg (ImageType: imageUrl, ImageUrl: '{1}', ImageUrlParams: [{1} = ImageB64]) } }` passed `check --references`; exec wrote a bare AttributeRef and `mx check` could not LOAD the project (ArgumentNullException setting 'Attribute'). Same at page top level outside any data container; a text box's `Attribute:` there is silently dropped (CE7005).","cause":"ALTER's entity context comes from the STORED document (nearest enclosing data source, or a flow source's return type via resolveDataSourceFlowEntity). With the flow missing (Feedback v4.0.2 ships no DS_FeedbackForm) or no container at all, entityContext is \"\" and resolveAttributePath returns the bare name. CREATE PAGE refused this at check time (relaxExcludedWidgetRefs/unscopedBindings, #678); ALTER's check never opened the document, so nothing could know the scope.","file":"mdl/executor/validate_alter_unscoped.go, mdl/executor/cmd_alter_page.go (alterEntityContext), mdl/executor/validate.go (bindingsWithoutScope)","insight":"For ALTER, scope is a property of the stored document, not the statement: a check-time question about it must open the document (OpenPageForMutation, never Save; validate_alter_set.go already does this) and ask through the SAME function exec uses, so the INSERT/REPLACE entity resolution was lifted into alterEntityContext rather than restated, and the binding walk lifted out of unscopedBindings (bindingsWithoutScope) rather than copied. Controls that keep it from blocking working scripts: skip a target the stored doc lacks (added earlier in the script), a flow the script declares with an entity return (sc.flowParams), DataGrid2 column and list-view-template paths, and documents the script creates. Reproduce with a Studio Pro-authored page whose flow is genuinely absent.","refs":["#678","#685"]} diff --git a/mdl-examples/bug-tests/alter-page-unscoped-insert-bindings.mdl b/mdl-examples/bug-tests/alter-page-unscoped-insert-bindings.mdl new file mode 100644 index 0000000000..24c0891e8d --- /dev/null +++ b/mdl-examples/bug-tests/alter-page-unscoped-insert-bindings.mdl @@ -0,0 +1,76 @@ +-- ============================================================================ +-- ALTER PAGE INSERT/REPLACE where no entity is in scope: bindings qualified +-- ============================================================================ +-- +-- Symptom: on FeedbackModule.ShareFeedback_Logo (Feedback v4.0.2, Mendix +-- 11.13.0), whose data view is sourced by a nanoflow the module does not ship, +-- alter page FeedbackModule.ShareFeedback_Logo { insert after textBox1 { +-- image zzImg (ImageType: imageUrl, ImageUrl: '{1}', ImageUrlParams: [{1} = ImageB64]) } } +-- passed `mxcli check --references`, and exec wrote the parameter as a BARE +-- attribute reference: `mx check` could no longer LOAD the project +-- (ArgumentNullException setting 'Attribute'). The same happens to a widget +-- inserted outside every data container; a text box's `Attribute:` there is +-- dropped instead (CE7005 "No value selection has been made"). +-- +-- Cause: ALTER takes its entity from the STORED document (the enclosing data +-- source, or a flow source's return type). With none, the builder has nothing +-- to qualify a bare name against. CREATE PAGE already refused this at check +-- time; ALTER's check never opened the document. +-- +-- Fix: `check --references` opens the stored page, resolves the insertion +-- point's entity through the same function exec uses (alterEntityContext), and +-- refuses bare bindings when there is none, naming widget and binding. +-- +-- Verify: exec with -p → both ALTERs apply, `mxcli docker check` → 0 errors. +-- Change the qualified binding in the first ALTER to `{1} = Subject`, or +-- insert `textbox t (Attribute: Subject)` after ctTop (no data container) → +-- `check --references` reports it. The bare name in the second ALTER (inside +-- dvEntity, entity in scope) stays unflagged. +-- ============================================================================ + +create module BugTestAlterScope; +/ + +create persistent entity BugTestAlterScope.Draft ( + Subject: String(200) +); +/ + +@excluded +create or modify page BugTestAlterScope.Draft_Example +( Title: 'Draft (example)', Layout: Atlas_Core.Atlas_Default, + Params: { $Draft: BugTestAlterScope.Draft } ) +{ + container ctTop { + dynamictext txtIntro (Content: 'Example') + } + layoutgrid lg { + row r { + column c (DesktopWidth: AutoFill) { + dataview dvFlow (DataSource: nanoflow BugTestAlterScope.DS_MissingForm) { + textbox txtSubject (Label: 'Subject', Attribute: BugTestAlterScope.Draft.Subject) + } + dataview dvEntity (DataSource: $Draft) { + textbox txtSubject2 (Label: 'Subject', Attribute: Subject) + } + } + } + } +} +/ + +-- Inside the data view whose flow is missing: the binding must be qualified. +alter page BugTestAlterScope.Draft_Example { + insert after txtSubject { + dynamictext txtEcho (Content: 'About: {1}', ContentParams: [{1} = BugTestAlterScope.Draft.Subject]) + } +}; +/ + +-- Inside a data view whose entity is known: a bare name is fine. +alter page BugTestAlterScope.Draft_Example { + insert after txtSubject2 { + dynamictext txtEcho2 (Content: 'About: {1}', ContentParams: [{1} = Subject]) + } +}; +/ diff --git a/mdl/executor/cmd_alter_page.go b/mdl/executor/cmd_alter_page.go index 2af79fd09e..6266ec845f 100644 --- a/mdl/executor/cmd_alter_page.go +++ b/mdl/executor/cmd_alter_page.go @@ -349,19 +349,7 @@ func applyInsertWidgetMutator(ctx *ExecContext, mutator backend.PageMutator, op // target IS the container, so the children take the target's own context (e.g. // a dataview's entity). into := strings.EqualFold(op.Position, "INTO") - entityCtx := mutator.EnclosingEntity(op.Target.Widget) - if into { - entityCtx = mutator.EnclosingEntityForChildren(op.Target.Widget) - } - // A microflow/nanoflow datasource contributes no entity to the BSON walk (its - // entity is the flow's RETURN type), so resolve it via the model — otherwise a - // widget inserted into a flow-sourced list binds nothing (CE0402/CE1613). (#55) - if entityCtx == "" { - mfQN, nfQN := mutator.EnclosingDataSourceFlow(op.Target.Widget, into) - if e := resolveDataSourceFlowEntity(ctx, moduleName, moduleID, mfQN, nfQN); e != "" { - entityCtx = e - } - } + entityCtx, _ := alterEntityContext(ctx, mutator, op.Target.Widget, into, moduleName, moduleID) // Build new widgets from AST widgets, err := buildWidgetsFromAST(ctx, op.Widgets, moduleName, moduleID, entityCtx, mutator) @@ -435,14 +423,7 @@ func applyReplaceWidgetMutator(ctx *ExecContext, mutator backend.PageMutator, op } // Find entity context from enclosing DataView/DataGrid/ListView for regular widget replace. - entityCtx := mutator.EnclosingEntity(op.Target.Widget) - // Resolve a microflow/nanoflow datasource's return entity (see the INSERT path). - if entityCtx == "" { - mfQN, nfQN := mutator.EnclosingDataSourceFlow(op.Target.Widget, false) - if e := resolveDataSourceFlowEntity(ctx, moduleName, moduleID, mfQN, nfQN); e != "" { - entityCtx = e - } - } + entityCtx, _ := alterEntityContext(ctx, mutator, op.Target.Widget, false, moduleName, moduleID) // Build new widgets from AST, excluding the target widget/column from the // duplicate-name scope so a same-name replacement is allowed. @@ -578,6 +559,35 @@ func buildColumnSpecsFromAST(ctx *ExecContext, widgets []*ast.WidgetV3, moduleNa // Widget building from AST (domain logic stays in executor) // ============================================================================ +// alterEntityContext is the entity an INSERT or REPLACE builds its widgets +// against, plus the flow that was meant to supply it when a flow data source is +// where the scope comes from. forChildren is INSERT INTO: the target IS the +// container, so its own data source decides; otherwise (INSERT BEFORE/AFTER, +// REPLACE) the target is a sibling and the nearest ENCLOSING source does. +// +// A microflow/nanoflow datasource contributes no entity to the BSON walk (its +// entity is the flow's RETURN type), so it is resolved via the model — otherwise +// a widget inserted into a flow-sourced list binds nothing (CE0402/CE1613, #55). +// +// Shared with the check-time pass (validate_alter_unscoped.go), so check and exec +// cannot disagree about which entity is in scope. +func alterEntityContext(ctx *ExecContext, mutator backend.PageMutator, widgetRef string, forChildren bool, moduleName string, moduleID model.ID) (entity, flow string) { + if forChildren { + entity = mutator.EnclosingEntityForChildren(widgetRef) + } else { + entity = mutator.EnclosingEntity(widgetRef) + } + if entity != "" { + return entity, "" + } + mfQN, nfQN := mutator.EnclosingDataSourceFlow(widgetRef, forChildren) + flow = mfQN + if flow == "" { + flow = nfQN + } + return resolveDataSourceFlowEntity(ctx, moduleName, moduleID, mfQN, nfQN), flow +} + // resolveDataSourceFlowEntity resolves the entity context contributed by a // microflow/nanoflow datasource — its RETURN entity — for ALTER PAGE widget // builds. A flow datasource stores no entity in its own BSON (the entity lives diff --git a/mdl/executor/validate.go b/mdl/executor/validate.go index f1ebeee7dc..01f9c5f1ab 100644 --- a/mdl/executor/validate.go +++ b/mdl/executor/validate.go @@ -367,6 +367,11 @@ func validateProgramWithWarnings(ctx *ExecContext, prog *ast.Program) ([]error, // widget that is already stored, so its property can only be resolved // against the document — which is why it passed check and failed exec. errors = append(errors, validateAlterSetProperties(ctx, prog, sc)...) + // The entity an ALTER's INSERT / REPLACE binds against is in the stored + // document, not the statement. Where it has none (a missing flow source, or + // no data container at all) a bare binding is written bare and the project + // no longer loads — CREATE PAGE already refused this at check time. + errors = append(errors, validateAlterUnscopedBindings(ctx, prog, sc)...) return errors, sc.warnings } @@ -940,21 +945,6 @@ func (sc *scriptContext) relaxExcludedWidgetRefs(kind, name string, widgets []*a // the page writer's refusal of a bare attribute reference. func unscopedBindings(widgets []*ast.WidgetV3, ref string) []string { var out []string - var inScope func(ws []*ast.WidgetV3) - inScope = func(ws []*ast.WidgetV3) { - for _, w := range ws { - if w == nil { - continue - } - if _, own := w.Properties["DataSource"].(*ast.DataSourceV3); own { - continue // its own data source decides its children's scope - } - for _, b := range bareBindingsOf(w) { - out = append(out, fmt.Sprintf("%s `%s` (%s)", strings.ToLower(w.Type), w.Name, b)) - } - inScope(w.Children) - } - } var find func(ws []*ast.WidgetV3) find = func(ws []*ast.WidgetV3) { for _, w := range ws { @@ -966,7 +956,7 @@ func unscopedBindings(widgets []*ast.WidgetV3, ref string) []string { for _, b := range bareBindingsOf(w) { // the container's own bindings, e.g. its visibility out = append(out, fmt.Sprintf("%s `%s` (%s)", strings.ToLower(w.Type), w.Name, b)) } - inScope(w.Children) + out = append(out, bindingsWithoutScope(w.Children)...) continue } find(w.Children) @@ -976,6 +966,28 @@ func unscopedBindings(widgets []*ast.WidgetV3, ref string) []string { return out } +// bindingsWithoutScope names the bare bindings of widgets placed where no +// entity is in scope — the children of a container whose data source flow is +// missing, or widgets an ALTER inserts outside any container that resolves to +// an entity. Descent stops at a widget with a data source of its own, which +// scopes its children. +func bindingsWithoutScope(ws []*ast.WidgetV3) []string { + var out []string + for _, w := range ws { + if w == nil { + continue + } + if _, own := w.Properties["DataSource"].(*ast.DataSourceV3); own { + continue // its own data source decides its children's scope + } + for _, b := range bareBindingsOf(w) { + out = append(out, fmt.Sprintf("%s `%s` (%s)", strings.ToLower(w.Type), w.Name, b)) + } + out = append(out, bindingsWithoutScope(w.Children)...) + } + return out +} + // bareBindingsOf lists a widget's attribute bindings that need an entity in // scope to resolve. func bareBindingsOf(w *ast.WidgetV3) []string { diff --git a/mdl/executor/validate_alter_unscoped.go b/mdl/executor/validate_alter_unscoped.go new file mode 100644 index 0000000000..68df997eb9 --- /dev/null +++ b/mdl/executor/validate_alter_unscoped.go @@ -0,0 +1,143 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/model" +) + +// A bare binding in a widget that ALTER PAGE / ALTER SNIPPET inserts where no +// entity is in scope passed `check --references` and was only caught — or not +// caught — at exec. +// +// The builder qualifies a bare `Attribute: X` or `{1} = X` with the entity of +// the insertion point, which exec reads from the STORED document: the nearest +// enclosing data source, or for a flow source the flow's return type. When there +// is none — the flow is missing, as FeedbackModule.DS_FeedbackForm is from +// Feedback v4.0.2's ShareFeedback_Logo, or the widget goes outside every data +// container — the binding is written bare. Measured on Mendix 11.13.0: an image +// URL parameter written that way left a project `mx check` could not LOAD +// (ArgumentNullException setting 'Attribute'); a text box's `Attribute:` was +// silently dropped and the build failed with CE7005 "No value selection has +// been made". +// +// CREATE PAGE catches the same shape at check time (relaxExcludedWidgetRefs, +// via unscopedBindings). ALTER could not, because its scope is not in the +// statement: it is in the document. So this pass opens the stored document and +// asks it the question exec asks, through the same function (alterEntityContext) +// — check and exec cannot disagree about which entity is in scope. + +// validateAlterUnscopedBindings reports the bare bindings of widgets an +// INSERT or REPLACE places where the stored document puts no entity in scope. +func validateAlterUnscopedBindings(ctx *ExecContext, prog *ast.Program, sc *scriptContext) []error { + if prog == nil || !ctx.Connected() { + return nil + } + h, err := getHierarchy(ctx) + if err != nil || h == nil { + return nil + } + opened := map[model.ID]backend.PageMutator{} + + var errs []error + for _, stmt := range prog.Statements { + s, ok := stmt.(*ast.AlterPageStmt) + if !ok || !hasWidgetBuildingOp(s) || alterTargetComesFromScript(sc, s) { + continue + } + unitID, containerID, containerType, err := resolveAlterPageUnit(ctx, s, h) + if err != nil { + continue // validateAlterTarget's finding + } + mutator, seen := opened[unitID] + if !seen { + // Read-only: the mutator is never saved. A document that will not + // open is silence, never a finding. + mutator, _ = ctx.Backend.OpenPageForMutation(unitID) + opened[unitID] = mutator + } + if mutator == nil { + continue + } + modName := h.GetModuleName(containerID) + label := fmt.Sprintf("alter %s %s", containerType, s.PageName.String()) + for _, op := range s.Operations { + var target ast.WidgetRef + var widgets []*ast.WidgetV3 + var into bool + switch o := op.(type) { + case *ast.InsertWidgetOp: + target, widgets, into = o.Target, o.Widgets, strings.EqualFold(o.Position, "INTO") + if allListViewTemplates(widgets) { + continue // built against the list view's own entity, on its own path + } + case *ast.ReplaceWidgetOp: + target, widgets = o.Target, o.NewWidgets + default: + continue + } + // DataGrid 2 columns take their own path, scoped by the grid. + if target.IsColumn() && allColumns(widgets) { + continue + } + // A target the stored document lacks is either added earlier in the + // script or refused by exec as not found; neither is this finding. + if !mutator.FindWidget(target.Widget) { + continue + } + if msg := unscopedInsertion(ctx, sc, mutator, target.Widget, into, modName, containerID, widgets); msg != "" { + errs = append(errs, mdlerrors.NewValidation(fmt.Sprintf("%s: %s", label, msg))) + } + } + } + return errs +} + +// unscopedInsertion returns the finding for one INSERT/REPLACE, or "". +func unscopedInsertion(ctx *ExecContext, sc *scriptContext, mutator backend.PageMutator, target string, into bool, + modName string, modID model.ID, widgets []*ast.WidgetV3) string { + entity, flow := alterEntityContext(ctx, mutator, target, into, modName, modID) + if entity != "" { + return "" + } + if flow != "" && sc != nil { + // A flow the script creates is not in the project yet; its declared + // return type is what exec will resolve against. + if sig, ok := sc.flowParams[strings.ToLower(flow)]; ok { + if sig.Returns != "" { + return "" + } + } + } + bare := bindingsWithoutScope(widgets) + if len(bare) == 0 { + return "" + } + where := fmt.Sprintf("the insertion point (`%s`) is in no data container", target) + if flow != "" { + where = fmt.Sprintf("the data container around `%s` is sourced by %s, which does not exist or returns no entity", target, flow) + } + return fmt.Sprintf("%s, so no entity is in scope and these bindings cannot be qualified: %s. "+ + "Written without an entity, a template parameter is stored bare (Mendix can no longer load the "+ + "project) and an attribute binding is dropped (CE7005). Qualify them (Module.Entity.Attribute), "+ + "or insert into a data container whose entity is known.", + where, strings.Join(bare, ", ")) +} + +// hasWidgetBuildingOp reports whether a statement carries an INSERT or REPLACE, +// so a script of pure SETs and DROPs never opens a document for this pass. +func hasWidgetBuildingOp(s *ast.AlterPageStmt) bool { + for _, op := range s.Operations { + switch op.(type) { + case *ast.InsertWidgetOp, *ast.ReplaceWidgetOp: + return true + } + } + return false +} diff --git a/mdl/executor/validate_alter_unscoped_test.go b/mdl/executor/validate_alter_unscoped_test.go new file mode 100644 index 0000000000..de9c9093c2 --- /dev/null +++ b/mdl/executor/validate_alter_unscoped_test.go @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "go.mongodb.org/mongo-driver/bson" + + "github.com/mendixlabs/mxcli/mdl/backend" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/backend/pagemutator" + "github.com/mendixlabs/mxcli/mdl/types" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// storedScopesPage is a page with the three insertion contexts that matter: +// +// - dvFlow: a data view sourced by a nanoflow the project does not contain — +// the shape of Feedback v4.0.2's ShareFeedback_Logo, whose +// FeedbackModule.DS_FeedbackForm is missing. Nothing puts an entity in scope. +// - dvEntity: a data view with a database source, so MyModule.Customer is in +// scope for its children. +// - tbTop: a widget at the top level, outside any data container. +func storedScopesPage() bson.D { + textbox := func(name string) bson.D { + return bson.D{{Key: "$Type", Value: "Forms$TextBox"}, {Key: "Name", Value: name}} + } + dvFlow := bson.D{ + {Key: "$Type", Value: "Forms$DataView"}, + {Key: "Name", Value: "dvFlow"}, + {Key: "DataSource", Value: bson.D{ + {Key: "$Type", Value: "Forms$NanoflowSource"}, + {Key: "Nanoflow", Value: "MyModule.DS_Missing"}, + }}, + {Key: "Widgets", Value: bson.A{int32(2), textbox("tbFlow")}}, + } + dvEntity := bson.D{ + {Key: "$Type", Value: "Forms$DataView"}, + {Key: "Name", Value: "dvEntity"}, + {Key: "DataSource", Value: bson.D{ + {Key: "$Type", Value: "Forms$DataViewSource"}, + {Key: "EntityRef", Value: bson.D{ + {Key: "$Type", Value: "DomainModels$DirectEntityRef"}, + {Key: "Entity", Value: "MyModule.Customer"}, + }}, + }}, + {Key: "Widgets", Value: bson.A{int32(2), textbox("tbEntity")}}, + } + return bson.D{ + {Key: "$Type", Value: "Forms$Page"}, + {Key: "FormCall", Value: bson.D{ + {Key: "Arguments", Value: bson.A{ + int32(2), + bson.D{{Key: "Widgets", Value: bson.A{int32(2), textbox("tbTop"), dvFlow, dvEntity}}}, + }}, + }}, + } +} + +func scopesPageCtx(t *testing.T) (*ExecContext, *countingDeps) { + t.Helper() + mod := mkModule("MyModule") + pg := mkPage(mod.ID, "P_Scopes") + deps := &countingDeps{} + mb := &mock.MockBackend{ + IsConnectedFunc: func() bool { return true }, + ListModulesFunc: func() ([]*model.Module, error) { return []*model.Module{mod}, nil }, + ListFoldersFunc: func() ([]*types.FolderInfo, error) { return nil, nil }, + ListPagesFunc: func() ([]*pages.Page, error) { return []*pages.Page{pg}, nil }, + OpenPageForMutationFunc: func(unitID model.ID) (backend.PageMutator, error) { + return pagemutator.New(storedScopesPage(), unitID, deps), nil + }, + } + ctx, _ := newMockCtx(t, withBackend(mb), withHierarchy(mkHierarchy(mod))) + return ctx, deps +} + +func checkAlterUnscoped(t *testing.T, ctx *ExecContext, src string) []error { + t.Helper() + prog := parseMDL(t, src) + sc := newScriptContext() + sc.collectDefinitions(prog) + return validateAlterUnscopedBindings(ctx, prog, sc) +} + +// TestAlterUnscoped_MissingFlowSource is the reported gap: an image inserted +// next to a widget inside a data view whose nanoflow is missing, with its URL +// parameter bound bare. check --references passed; exec wrote a bare +// AttributeRef and `mx check` could not LOAD the project. +func TestAlterUnscoped_MissingFlowSource(t *testing.T) { + ctx, deps := scopesPageCtx(t) + errs := checkAlterUnscoped(t, ctx, `alter page MyModule.P_Scopes { + insert after tbFlow { image zzImg (ImageType: imageUrl, ImageUrl: '{1}', ImageUrlParams: [{1} = ImageB64]) } + }`) + if len(errs) != 1 { + t.Fatalf("got %d errors, want 1: %v", len(errs), errs) + } + msg := errs[0].Error() + for _, want := range []string{"MyModule.P_Scopes", "zzImg", "ImageB64", "MyModule.DS_Missing", "Module.Entity.Attribute"} { + if !strings.Contains(msg, want) { + t.Errorf("error %q does not name %q", msg, want) + } + } + if deps.saves != 0 { + t.Errorf("validation wrote to storage %d times, want 0", deps.saves) + } +} + +// TestAlterUnscoped_ReplaceAndInto — REPLACE resolves the sibling context and +// INSERT INTO the target's own; both land in the flow-sourced data view. +func TestAlterUnscoped_ReplaceAndInto(t *testing.T) { + ctx, _ := scopesPageCtx(t) + for _, src := range []string{ + `alter page MyModule.P_Scopes { replace tbFlow with { textbox tbNew (Attribute: Subject) } }`, + `alter page MyModule.P_Scopes { insert into dvFlow { textbox tbNew (Attribute: Subject) } }`, + } { + errs := checkAlterUnscoped(t, ctx, src) + if len(errs) != 1 || !strings.Contains(errs[0].Error(), "Attribute: Subject") { + t.Errorf("%s:\n got %v, want one error naming Attribute: Subject", src, errs) + } + } +} + +// TestAlterUnscoped_TopLevel — a widget inserted outside every data container +// has no entity either. Measured on 11.13.0: the image parameter was written +// bare (project unloadable) and a text box's attribute was silently dropped. +func TestAlterUnscoped_TopLevel(t *testing.T) { + ctx, _ := scopesPageCtx(t) + errs := checkAlterUnscoped(t, ctx, `alter page MyModule.P_Scopes { + insert before tbTop { image zzTop (ImageType: imageUrl, ImageUrl: '{1}', ImageUrlParams: [{1} = FullName]) } + }`) + if len(errs) != 1 || !strings.Contains(errs[0].Error(), "FullName") || + !strings.Contains(errs[0].Error(), "no data container") { + t.Fatalf("got %v, want one error naming FullName and the missing container", errs) + } +} + +// --------------------------------------------------------------------------- +// Controls — a bare binding with an entity in scope is the ordinary case +// --------------------------------------------------------------------------- + +func TestAlterUnscoped_Controls(t *testing.T) { + cases := map[string]string{ + "entity in scope (sibling)": `alter page MyModule.P_Scopes { + insert after tbEntity { textbox t (Attribute: Name) } }`, + "entity in scope (into)": `alter page MyModule.P_Scopes { + insert into dvEntity { textbox t (Attribute: Name) } }`, + "qualified binding under the missing flow": `alter page MyModule.P_Scopes { + insert after tbFlow { image zzImg (ImageType: imageUrl, ImageUrl: '{1}', ImageUrlParams: [{1} = MyModule.Feedback.ImageB64]) } }`, + "nested container scopes its own children": `alter page MyModule.P_Scopes { + insert before tbTop { dataview dvNew (DataSource: database MyModule.Customer) { textbox t (Attribute: Name) } } }`, + "page parameter path": `alter page MyModule.P_Scopes { + insert after tbFlow { dynamictext d (Content: '{1}', ContentParams: [{1} = $Customer/Name]) } }`, + "unknown target is someone else's finding": `alter page MyModule.P_Scopes { + insert after noSuchWidget { textbox t (Attribute: Name) } }`, + // The flow is created by the script: its declared return type is what + // exec will resolve against, so the bindings are in scope. + "flow defined in the script": `create nanoflow MyModule.DS_Missing () returns MyModule.Customer begin + return empty; end; + alter page MyModule.P_Scopes { insert after tbFlow { textbox t (Attribute: Name) } }`, + } + for name, src := range cases { + t.Run(name, func(t *testing.T) { + ctx, _ := scopesPageCtx(t) + if errs := checkAlterUnscoped(t, ctx, src); len(errs) != 0 { + t.Errorf("got %v, want none", errs) + } + }) + } +} From 1ec267ffe0d99dc1528369e6fe920b446e49f5a7 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 14:14:35 +0000 Subject: [PATCH 42/47] fix(catalog): mark java action type-parameter types unambiguously The catalog stored a Java action's return and parameter types via TypeString(), which renders a type-parameter reference as its bare name. A type parameter named `String` (Studio Pro allows it) therefore read back as 'String', identical to the primitive, and every other type parameter showed whatever the modeler had called it ('TypeParameter', 'TypeParEntity', ...) with nothing marking it as a type parameter. Encode them as `TypeParameter:` (object of the bound entity) and `EntityTypeParameter:` (the entity-type selector), following the `Kind:Name` shape microflows_data already uses. DESCRIBE is unchanged: there the bare name is the MDL syntax. Fixes mendixlabs/mxcli#1183 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G2c72jsT9JsY1c3V2eViLQ --- .../skills/fix-issue/findings/mdl-other.jsonl | 1 + ...alog-java-action-type-parameter-return.mdl | 42 ++++++ mdl/catalog/builder_java_actions_test.go | 125 ++++++++++++++++++ mdl/catalog/builder_modules.go | 28 +++- mdl/catalog/tables.go | 6 +- 5 files changed, 199 insertions(+), 3 deletions(-) create mode 100644 mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl create mode 100644 mdl/catalog/builder_java_actions_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-other.jsonl b/.claude/skills/fix-issue/findings/mdl-other.jsonl index 1778b7b34e..755261ae14 100644 --- a/.claude/skills/fix-issue/findings/mdl-other.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-other.jsonl @@ -71,3 +71,4 @@ {"area": "mdl/versions", "date": "2026-09-21", "symptom": "A version gate copied from the issue text (\"Workflow Groups are GA from Mendix 11.6\") is wrong by four minors", "cause": "Mendix's release notes date the FEATURE's general availability; the metamodel floor is when the type and its property were introduced, and that is what decides whether the document loads. `Settings$WorkflowGroup` and `WorkflowsProjectSettingsPart.groups` are both `introduced: \"11.2.0\"`", "file": "`sdk/versions/mendix-11.yaml` (`workflows.groups`)", "insight": "The arbiter for a metamodel floor is the Model SDK's own StructureVersionInfo: `npm pack mendixmodelsdk && tar xzf \u2026 && grep -n '' package/src/gen/.js`, then read BOTH the class's `versionInfo.introduced` and its `properties..introduced` \u2014 a property can arrive later than its type. Release notes, proposal text and a number already written down in this repo are all downstream of it (same trap as mendixlabs/mxcli#1121). Corroborate it against two real projects rather than trusting one source: `mxcli new` at a version either side of the floor and diff the document's keys \u2014 an 11.1.0 workflows settings part has no `Groups` key at all, an 11.13.0 one carries `Groups: [2]`, which also proves the refusal is right rather than over-cautious (writing the property below the floor would be inventing a key). mendixlabs/mxcli#272", "refs": ["mendixlabs/mxcli#272"]} {"area": "mdl/linter/rules", "date": "2026-09-23", "symptom": "CONV010 flagged an ACT_ nanoflow that delegated to a sub-flow \u2014 the very thing the rule demands. A real project patched its own copy of the rule and asked for the fix upstream. An ACT_ nanoflow could satisfy CONV010 in NO way: delegate and be flagged, or inline the logic and be flagged.", "cause": "ALLOWED_ACTIONS held `MicroflowCallAction` but not `NanoflowCallAction`. `microflows()` yields nanoflows too \u2014 the catalog's `microflows` table carries a MicroflowType column \u2014 so CONV010 lints ACT_ nanoflows, and a nanoflow delegates with a nanoflow call.", "file": "`.claude/lint-rules/conv010_act_microflow_content.star` (NanoflowCallAction added to ALLOWED_ACTIONS; cmd/mxcli/lint-rules/ is gitignored and regenerated by `make sync-lint-rules`), test `mdl/catalog/lint_rule_vocabulary_test.go` (added to the `permitted` list)", "insight": "Third time this one allowlist has been short, and the rule's own comments record the previous two: the wrong vocabulary entirely (storage names vs SDK names, matching nothing, 11 false positives of 13 findings) and a missing ExclusiveMerge that a permitted ExclusiveSplit necessarily creates (122 hits on one project). The recurring shape is an UNSATISFIABLE rule, and its cost is asymmetric: a rule that cannot be satisfied does not read as a broken rule, it reads as broken CODE, so users refactor around it or patch the rule locally and the defect never comes back upstream \u2014 which is exactly what happened here until someone wrote 'report upstream' in their findings. A vocabulary pin test (TestCONV010AllowsWhatTheCatalogCallsUIActions) already existed to stop this class and did not, because its `permitted` list is hand-maintained and was itself incomplete: pinning a rule to a hand-written list of what SHOULD be allowed only moves the completeness problem. Worth considering: enumerate the delegation actions from the type system rather than listing them.", "refs": ["ako/mxcli#644"]} {"area": "mdl/linter", "date": "2026-09-25", "symptom": "No way to ask a structural question about ONE document: `lint` scopes by module and by rule but not by document, so validating a microflow after each `exec` meant a project-wide lint (~13 s measured by the reporter) plus a baseline diff to see which findings were new. At that price the gate gets batched to once per session \u2014 three CONV011 violations shipped under a clean mxbuild log, six across three microflows before anyone looked.", "cause": "Feature gap, but the shape of the fix is not obvious: every rule guards its expensive per-document read (`FullMicroflow`) with `IsExcluded(moduleName)`, so module scoping already skipped the costly part for other modules \u2014 the missing granularity was WITHIN a module.", "file": "`mdl/linter/context.go` (`SetIncludedDocuments`, `IsDocumentExcluded`, `documentFilterSQL`; `Microflows`/`Pages`/`Widgets` narrowed in SQL; new `includeActive` flag), `cmd/mxcli/cmd_lint.go` + `main.go` (`-d/--documents`), `cmd/mxcli/lint_document_filter.go` (violation post-filter), tests `mdl/linter/context_document_filter_test.go`", "insight": "**Scope in the ITERATOR, not in each rule** \u2014 one SQL predicate on `Microflows()` covers CONV011, MPR002, CONV010, QUAL003 and every other rule that walks it, with no rule edited and no second copy of the filter to drift. **Two traps, both found by tests I wrote and then tried to break.** (1) An empty inclusion map means 'no filter' to `IsExcluded`, so intersecting `--modules A` with `--documents B.C` \u2014 an empty set \u2014 made lint scan the WHOLE project instead of nothing; distinguishing 'no allowlist' from 'allowlist that matched nothing' needs an explicit bool, not `len(map) > 0`. (2) The narrowing test PASSED against code with the SQL filter reverted, because the shared fixture has one microflow per module and the module implication alone explained the result \u2014 a sibling document in the SAME module is the only fixture that isolates document narrowing from module narrowing. Prove-by-revert is what caught both; the second would otherwise have shipped a test that could never fail. Also: a rule that reports from project settings rather than a document iterator is untouched by SQL narrowing, so a scoped run needs a violation post-filter too \u2014 and it must accept BOTH spellings of Location.DocumentName, since CONV010 sets the qualified name and MPR011 the short one.", "refs": ["ako/mxcli#681", "mendixlabs/mxcli#1186"]} +{"area": "mdl/catalog", "date": "2026-09-25", "symptom": "java_actions.ReturnType showed a type-parameter return as the type parameter's own name: 'TypeParameter', 'TypeParEntity', 'FileTypeDocument' depending on what the modeler called it, and a type parameter named `String` (Studio Pro allows it) read back as 'String', identical to the primitive. java_action_parameters.ParameterType had the same ambiguity for both the object parameter and the entity-type selector.", "cause": "The catalog stored TypeString(), which is DESCRIBE's MDL rendering: a bare type-parameter reference IS its name in MDL syntax, so the value carried no marker that it was a type parameter at all.", "file": "mdl/catalog/builder_modules.go (catalogCodeActionType)", "insight": "The report reads like three inconsistent conventions ('TypeParameter' / 'TypeParEntity' / a name) but it is one: every value was the modeler's chosen name, and 'TypeParameter' is merely Studio Pro's default. Fix the encoding in the catalog only (`TypeParameter:`, `EntityTypeParameter:`, the `Kind:Name` shape microflows_data already uses; no primitive contains a colon) \u2014 changing TypeString() would change DESCRIBE output, where the bare name is the syntax. Test with the name colliding with a primitive and a primitive action as control: a unit test on the builder with MockBackend.ListJavaActionsFullFunc, plus an end-to-end catalog query on testdata/expr-checker (copy the whole dir; the .mpr alone is v2 without mprcontents/). MDL itself cannot declare a type parameter named String (`entity ` is a parse error), so the colliding case is only reachable via Studio Pro-authored models.", "refs": ["mendixlabs/mxcli#1183"]} diff --git a/mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl b/mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl new file mode 100644 index 0000000000..52dfc9c15d --- /dev/null +++ b/mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl @@ -0,0 +1,42 @@ +-- ============================================================================ +-- Bug #1183: catalog shows a type-parameter return type as a bare name +-- ============================================================================ +-- +-- Symptom (before fix): +-- "when I create a Java action with a TypeParameter named String, which is +-- allowed by Studio, and use that as return type, the catalog will show +-- 'String', similar to the primitive 'String'." +-- java_actions.ReturnType (and java_action_parameters.ParameterType) held the +-- type parameter's own name — 'TypeParameter', 'TypeParEntity', … — with +-- nothing marking it as a type parameter. +-- +-- After fix: +-- A type-parameter reference is encoded `TypeParameter:`, the entity-type +-- selector `EntityTypeParameter:`; primitives are unchanged. +-- +-- MDL cannot declare a type parameter named `String` (`entity ` is a +-- parse error), so this script uses Studio Pro's default name. The primitive- +-- named case is covered by TestJavaActionTypeParameterNamedAfterPrimitive. +-- +-- Usage: +-- mxcli exec mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl -p app.mpr +-- mxcli -p app.mpr -c "refresh catalog" +-- mxcli -p app.mpr -c "select QualifiedName, ReturnType from CATALOG.JAVA_ACTIONS where ModuleName = 'BugTest1183'" +-- Expected: JA_ReturnsTypeParam -> TypeParameter:TypeParameter +-- JA_ReturnsString -> String +-- ============================================================================ + +create module BugTest1183; + +create java action BugTest1183.JA_ReturnsTypeParam ( + EntityType: entity not null, + Input: TypeParameter +) returns TypeParameter as $$ + return Input; +$$; + +create java action BugTest1183.JA_ReturnsString ( + Text: string +) returns string as $$ + return Text; +$$; diff --git a/mdl/catalog/builder_java_actions_test.go b/mdl/catalog/builder_java_actions_test.go new file mode 100644 index 0000000000..90889942ca --- /dev/null +++ b/mdl/catalog/builder_java_actions_test.go @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: Apache-2.0 + +package catalog + +import ( + "testing" + + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/sdk/javaactions" +) + +// Studio Pro lets a type parameter take any name, including a primitive's. The +// catalog wrote a type-parameter reference as its bare name, so an action whose +// type parameter is called `String` and returns it read back as ReturnType +// 'String' — the same value as an action returning the primitive String — and +// the column's values for type parameters were whatever the modeler had named +// them ('TypeParameter', 'TypeParEntity', 'FileTypeDocument', …) with nothing to +// say they were type parameters at all (mendixlabs/mxcli#1183). +func TestJavaActionTypeParameterNamedAfterPrimitive(t *testing.T) { + const modID = model.ID("mod-util") + + tpDef := &javaactions.TypeParameterDef{BaseElement: model.BaseElement{ID: "tp-string"}, Name: "String"} + generic := &javaactions.JavaAction{ + BaseElement: model.BaseElement{ID: "ja-generic"}, + ContainerID: modID, + Name: "ReturnsTypeParam", + TypeParameters: []*javaactions.TypeParameterDef{tpDef}, + ReturnType: &javaactions.TypeParameter{TypeParameterID: "tp-string", TypeParameter: "String"}, + Parameters: []*javaactions.JavaActionParameter{ + { + BaseElement: model.BaseElement{ID: "p-selector"}, + Name: "EntityType", + ParameterType: &javaactions.EntityTypeParameterType{TypeParameterID: "tp-string", TypeParameterName: "String"}, + }, + { + BaseElement: model.BaseElement{ID: "p-object"}, + Name: "Input", + ParameterType: &javaactions.TypeParameter{TypeParameterID: "tp-string", TypeParameter: "String"}, + }, + }, + } + primitive := &javaactions.JavaAction{ + BaseElement: model.BaseElement{ID: "ja-primitive"}, + ContainerID: modID, + Name: "ReturnsString", + ReturnType: &javaactions.StringType{}, + Parameters: []*javaactions.JavaActionParameter{ + { + BaseElement: model.BaseElement{ID: "p-text"}, + Name: "Text", + ParameterType: &javaactions.StringType{}, + }, + }, + } + + cat, err := New() + if err != nil { + t.Fatal(err) + } + defer cat.Close() + tx, err := cat.CatalogDB().Begin() + if err != nil { + t.Fatal(err) + } + b := &Builder{ + catalog: cat, + reader: &mock.MockBackend{ + ListJavaActionsFullFunc: func() ([]*javaactions.JavaAction, error) { + return []*javaactions.JavaAction{generic, primitive}, nil + }, + }, + snapshot: &Snapshot{ID: "snap"}, + hierarchy: &hierarchy{ + moduleIDs: map[model.ID]bool{modID: true}, + moduleNames: map[model.ID]string{modID: "Util"}, + containerParent: map[model.ID]model.ID{}, + folderNames: map[model.ID]string{}, + }, + tx: tx, + } + if err := b.buildJavaActions(); err != nil { + t.Fatalf("buildJavaActions: %v", err) + } + if err := tx.Commit(); err != nil { + t.Fatal(err) + } + + returns := queryStrings(t, cat, `SELECT QualifiedName, ReturnType FROM java_actions_data`) + if got, want := returns["Util.ReturnsString"], "String"; got != want { + t.Errorf("primitive return = %q, want %q (control)", got, want) + } + if got, want := returns["Util.ReturnsTypeParam"], "TypeParameter:String"; got != want { + t.Errorf("type-parameter return = %q, want %q -- a type parameter named "+ + "after a primitive must not read back as that primitive", got, want) + } + + params := queryStrings(t, cat, `SELECT Name, ParameterType FROM java_action_parameters_data`) + want := map[string]string{ + "Text": "String", + "Input": "TypeParameter:String", + "EntityType": "EntityTypeParameter:String", + } + for name, w := range want { + if params[name] != w { + t.Errorf("parameter %s type = %q, want %q", name, params[name], w) + } + } +} + +// queryStrings runs a two-column query and indexes the second column by the first. +func queryStrings(t *testing.T, cat *Catalog, q string) map[string]string { + t.Helper() + res, err := cat.Query(q) + if err != nil { + t.Fatalf("query: %v", err) + } + out := map[string]string{} + for _, row := range res.Rows { + k, _ := row[0].(string) + v, _ := row[1].(string) + out[k] = v + } + return out +} diff --git a/mdl/catalog/builder_modules.go b/mdl/catalog/builder_modules.go index a9b13d1107..7beeab0c35 100644 --- a/mdl/catalog/builder_modules.go +++ b/mdl/catalog/builder_modules.go @@ -7,6 +7,7 @@ import ( "strings" "github.com/mendixlabs/mxcli/sdk/domainmodel" + "github.com/mendixlabs/mxcli/sdk/javaactions" ) func (b *Builder) buildModules() error { @@ -353,6 +354,29 @@ func (b *Builder) buildEnumerations() error { return nil } +// catalogCodeActionType encodes a Java action return or parameter type for the +// catalog. A type-parameter reference is prefixed rather than written as its +// bare name: Studio Pro accepts any name for a type parameter, including a +// primitive's, and a bare `String` made an action returning its type parameter +// called String indistinguishable from one returning the primitive +// (mendixlabs/mxcli#1183). The prefix follows the `Kind:Name` shape +// microflows_data.ReturnType already uses; no primitive contains a colon. +// +// TypeParameter:T — an object of the entity bound to T +// EntityTypeParameter:T — the entity-type selector that binds T +// +// DESCRIBE keeps the bare name, which is its MDL syntax; this is the catalog's +// encoding only. +func catalogCodeActionType(t interface{ TypeString() string }) string { + switch tp := t.(type) { + case *javaactions.TypeParameter: + return "TypeParameter:" + tp.TypeParameter + case *javaactions.EntityTypeParameterType: + return "EntityTypeParameter:" + tp.TypeParameterName + } + return t.TypeString() +} + func (b *Builder) buildJavaActions() error { actions, err := b.reader.ListJavaActionsFull() if err != nil { @@ -392,7 +416,7 @@ func (b *Builder) buildJavaActions() error { returnType := "" if ja.ReturnType != nil { - returnType = ja.ReturnType.TypeString() + returnType = catalogCodeActionType(ja.ReturnType) } _, err := stmt.Exec( @@ -417,7 +441,7 @@ func (b *Builder) buildJavaActions() error { } paramType := "" if p.ParameterType != nil { - paramType = p.ParameterType.TypeString() + paramType = catalogCodeActionType(p.ParameterType) } if _, err := paramStmt.Exec( string(p.ID), diff --git a/mdl/catalog/tables.go b/mdl/catalog/tables.go index 84aca3aee0..9e539c353f 100644 --- a/mdl/catalog/tables.go +++ b/mdl/catalog/tables.go @@ -335,7 +335,11 @@ func (c *Catalog) createTables() error { )`, viewWithFullSnapshot("enumeration_values"), - // java_actions + // java_actions. ReturnType (and java_action_parameters.ParameterType) + // encode a type-parameter reference as `TypeParameter:` and the + // entity-type selector as `EntityTypeParameter:` — a bare name + // cannot tell a type parameter called String from the primitive + // (mendixlabs/mxcli#1183). See catalogCodeActionType. `CREATE TABLE IF NOT EXISTS java_actions_data ( Id TEXT PRIMARY KEY, Name TEXT, From c1f8c899bde115e7a92011a1611b67d1d61fe497 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 14:15:25 +0000 Subject: [PATCH 43/47] fix(pages): refuse an attribute binding with no object to bind to (MDL-WIDGET34) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `textbox t (Attribute: FullName)` at the top of a page, snippet or plain container has no entity to qualify the name with; the writer stores anything shorter than Module.Entity.Attribute as `AttributeRef: null`. Plain `check` passed, `exec --no-check` and `alter page … insert` said success, and mxbuild 11.13.0 failed the page: CE0544 + CE7005 on text box, text area, date picker, check box, radio buttons and drop-down, CE0402 on a dynamic text, CE0642 on a combo box. A qualified attribute there is stored and fails as well (CE0544 / CE2421 / CE1365 / CE7247 + CE7006). `Attribute: $P/Name` and `$currentObject/Name` parse as a data-source expression no builder reads, so they were dropped even inside a data view. - check: MDL-WIDGET34 in the widget-tree walk (no project needed), using the MDL-PAGEARG01 three-state context — refuses bare and qualified bindings at a document root outside any data widget, and the `$x/Attr` spelling anywhere. ALTER's subtree walk (unknown context) stands down. - build: the six input builders, dynamic text and the pluggable engine's primary `Attribute:` mapping refuse with the widget named, so nothing is written; with an unknown context only a bare name with no entity is refused, so qualified bindings inside an unresolvable flow source (excluded ShareFeedback_Logo) keep building. Two unit tests built inputs with no entity in scope and one asserted the bare `Title` reference counted as bound; they now set an entity context. Verified on a copy of a Mendix 11.13.0 project: 22 mxbuild errors before, every case refused after with nothing written; controls (data view, list view, gallery, data grid, snippet data view, ALTER into a data view) build at 0 errors; describe -> exec round trip 17/17 pages, 4/4 snippets. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + CHANGELOG.md | 1 + .../input-binding-without-context.fail.mdl | 36 ++++ .../input-binding-without-context.mdl | 49 ++++++ .../cmd_pages_builder_onchange_test.go | 3 + mdl/executor/cmd_pages_builder_v3_widgets.go | 24 +++ .../cmd_pages_input_binding_context.go | 150 ++++++++++++++++ .../cmd_pages_input_binding_context_test.go | 163 ++++++++++++++++++ mdl/executor/cmd_pages_popup_test.go | 7 + mdl/executor/validate_widgets.go | 2 + mdl/executor/widget_engine.go | 8 + 11 files changed, 444 insertions(+) create mode 100644 mdl-examples/bug-tests/input-binding-without-context.fail.mdl create mode 100644 mdl-examples/bug-tests/input-binding-without-context.mdl create mode 100644 mdl/executor/cmd_pages_input_binding_context.go create mode 100644 mdl/executor/cmd_pages_input_binding_context_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index ed70b75f46..766f97e6b4 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -706,3 +706,4 @@ {"area": "mdl/executor", "symptom": "describe → exec of Feedback v4.0.2's EXCLUDED FeedbackModule.ShareFeedback_Logo refused `nanoflow not found: FeedbackModule.DS_FeedbackForm (data source)`; forcing it through left a project `mx check` could not LOAD (ArgumentNullException setting 'Attribute')", "cause": "The data view's flow is not in the project, so DESCRIBE had no context entity and printed every binding inside it bare (`Attribute: Subject`, `{1} = ImageB64`, `Visible: _showEmail in (…)`); exec had nothing to qualify them against, and a bare DomainModels$AttributeRef makes Mendix's loader throw. The stored model always had the full names", "file": "`mdl/executor/cmd_pages_describe_flowcontext.go` (`withQualifiedAttrs`, `describeAttr`), `mdl/executor/validate.go` (`unscopedBindings`), `mdl/executor/cmd_pages_builder_v3.go` (dangling DS flow kept by name), `mdl/backend/modelsdk/page_bare_attributeref.go` (`refuseBareAttributeRefs`)", "insight": "**The information was never lost — DESCRIBE threw it away**: every AttributeRef inside the unresolvable container still stored Module.Entity.Attr; shortening to the bare name is only safe where the reader can re-derive the entity, so key the shortening on whether the context resolved, not on habit. Measure the loader's tolerance before adding a write guard: 72 of 72 Studio Pro AttributeRefs in the project are qualified, so refusing a bare one refuses only writes that were already fatal — and turns a load-time stack trace into a statement-level error naming the attribute. Pair a blanket AST-level refusal with the exact failing slots (Attribute, CaptionAttribute, Visible-in, *Params) so the refusal names the widget, and keep the writer guard as the net for slots the walk does not know. Forced-fault run: hand-edit one qualified binding back to bare and confirm the refusal names it", "refs": ["ako/mxcli#675", "FeedbackModule.ShareFeedback_Logo"], "date": "2026-09-25"} {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_New fails `mx check` with [CE0642] \"Property 'Caption' is required.\" at Combo box 'comboBox2' (Account_Edit comboBox4 too); check and exec report success", "cause": "The ComboBox caption is an EXPRESSION (optionsSourceAssociationCaptionType=expression, optionsSourceAssociationCaptionExpression='$currentObject/Description'); describe read only the attribute caption, so the widget was rewritten with none. The write side already worked via the explicit-property pass, but MDL-WIDGET06 claimed both keys 'will be dropped'", "file": "`mdl/executor/cmd_pages_describe_parse.go` (combobox branch), `cmd_pages_describe_output.go`, `validate_widget_explicit_writable.go` (`persistedByExplicitPass`), `validate_widgets.go` (MDL-WIDGET06)", "insight": "**Resolve pluggable-widget properties by key before theorising**: a 20-line script mapping each Property's TypePointer to its PropertyKey through the widget's own Type.ObjectType settled the cause in one run, where the ndsl dump shows only anonymous values. Then TEST THE WRITE SIDE before building one: adding the two storage keys to the describe output by hand persisted both and built clean, which shrank the fix to describe + a false warning — no new syntax. A validator rule that asserts 'not persisted' must be tied to what the write path handles (here: the explicit pass writes Expression/TextTemplate/Attribute and scalar types), or it goes stale when the writer grows; the #643 test pinned the stale claim. A dedicated extractor for a 'known' pluggable widget silently drops every property it does not map — generic widgets emit them, known ones do not", "refs": ["ako/mxcli#664", "ako/mxcli#643"], "ce": ["CE0642"], "rules": ["MDL-WIDGET06"], "date": "2026-09-25"} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`mxcli check` passed a microflow with a commit inside a loop \u2014 one database round trip per iteration \u2014 that `mxcli lint` already flagged as CONV011. The defect surfaced only at project-wide lint time, long after the write.", "cause": "CONV011 reads the STORED model, so it cannot speak until `exec` has written the microflow; `check` reads the MDL and had no equivalent rule. The gap is temporal, not a missing capability on either side.", "file": "`mdl/executor/validate_commit_in_loop.go` (MDL-PERF01, hooked in `validate_microflow.go`), test `validate_commit_in_loop_test.go`, example `mdl-examples/bug-tests/1186-commit-in-loop.mdl`", "insight": "**When adding a check-time rule that anticipates an existing lint rule, pin the BOUNDARY to the lint rule's, not to the better one, and say why in the code.** A `while true` is built as an ExclusiveMerge back-edge rather than a LoopedActivity, so CONV011 (which walks LoopedActivity) does not flag a commit inside one. A commit there is arguably still N+1, and the tempting move is to be more correct \u2014 but two rules for one concept that disagree on what counts is precisely how a pair drifts, and this repo already has CONV010's three successive short allowlists as the worked example. If the case is worth reporting it is worth reporting in BOTH, and the stored-model rule is the one that sees the built flow. The test that pins this carries a control on the control: `while true` is exempt, `while ` is not, so the exemption cannot silently become 'never flag a while'. Name the sibling rule in the message (`lint reports this as CONV011`) so a reader hitting one recognises the other rather than filing it twice. Also worth reusing: `checkReturnInLoop` already had the depth-tracking walk over Loop/While/If/EnumSplit/InheritanceSplit \u2014 copying its shape got the nesting cases right for free.", "refs": ["ako/mxcli#681", "mendixlabs/mxcli#1186"], "rules": ["MDL-PERF01"]} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "`textbox t (Attribute: FullName)` at the top of a page (CREATE PAGE/SNIPPET, a plain container, or ALTER PAGE … INSERT at page level) passed plain `mxcli check`, `exec --no-check`/ALTER reported success, and `bson dump` showed `AttributeRef: null` — mxbuild 11.13.0: CE0544 \"This widget can only function inside a data context\" + CE7005 (textbox/textarea/datepicker/checkbox/radiobuttons/dropdown), CE0402 (dynamictext Attribute:), CE0642 (combobox). Qualified `Mod.Ent.Attr` there is stored and fails CE0544/CE2421/CE1365/CE7247 \"Move this widget into a data container\" + CE7006. `Attribute: $P/Attr` / `$currentObject/Attr` dropped even INSIDE a data view.", "cause": "resolveAttributePath returns the bare name when entityContext is \"\", and attributeRefToGen (and widgetobj setAttributeRefField) write nil for any path with < 2 dots, so the binding vanished between builder and writer; refuseBareAttributeRefs never sees it because no Attribute string is emitted. The only refusal (validatePageContextTree) runs in the --references phase for CREATE PAGE/SNIPPET, so plain check, --no-check and ALTER were unguarded. `$x/Attr` parses via the generic property rule as an *ast.DataSourceV3, so GetAttribute() returns \"\" and every builder skipped it.", "file": "mdl/executor/cmd_pages_input_binding_context.go (inputBindingProblem, checkInputBinding, validateInputBindingContext = MDL-WIDGET34), wired in cmd_pages_builder_v3_widgets.go (6 input builders + buildDynamicTextV3), widget_engine.go (primary Attribute mapping), validate_widgets.go (validateWidgetTreeIn); tests cmd_pages_input_binding_context_test.go; bug-tests input-binding-without-context{,.fail}.mdl", "insight": "Reuse the MDL-PAGEARG01 three-state context (pageArgContext known/present) rather than entityContext==\"\" as the 'outside a data container' signal: entityContext is also empty INSIDE a container whose flow cannot be resolved (excluded ShareFeedback_Logo), where DESCRIBE writes qualified names that must keep building — refusing qualified-on-empty-entity would have broken that round trip. So known-absent context refuses bare AND qualified; unknown context (ALTER) refuses only the bare name the writer provably nulls. Two existing unit tests (OnChangeSurvivesBuilder, DynamicTextV3_AttributeBinds) built inputs with NO entity and passed — the second asserted a bare `Title` AttributeRef counted as 'bound', i.e. it pinned the bug: when a fixture has no entity context, ask what the writer does with its output. The `$P/Attr` drop was found only by dumping the control page, not from the report — print the AST value type with a probe test before assuming a spelling reaches the builder. Evidence: 22 mxbuild errors before on the probe matrix; after, every case refused with nothing written, controls (dataview/listview/gallery/datagrid/snippet dataview/ALTER into dataview) 0 errors, 17/17 stock pages + 4/4 snippets describe→exec round trip.", "refs": ["MDL-WIDGET34"], "ce": ["CE0544", "CE7005", "CE0402", "CE0642", "CE2421", "CE1365", "CE7247", "CE7006"]} diff --git a/CHANGELOG.md b/CHANGELOG.md index 3ac4a90451..c82d830053 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed +- **An input bound to an attribute outside any data container was written with no binding** — `textbox t (Attribute: FullName)` at the top of a page (or of a snippet, or inside a plain container) has no entity to qualify the name with, and the writer stored it as `AttributeRef: null`. Plain `check` passed, `exec --no-check` and `alter page … insert` at page level said success, and mxbuild 11.13.0 failed the page with CE0544 "This widget can only function inside a data context" + CE7005 (text box, text area, date picker, check box, radio buttons, drop-down), CE0402 (dynamic text `Attribute:`) or CE0642 (combo box). A qualified attribute there is stored and fails the same way (CE0544 / CE2421 / CE1365 / CE7247 "Move this widget into a data container"). `Attribute: $P/Name` and `Attribute: $currentObject/Name` never parsed as an attribute path and were dropped even inside a data view. `check` now reports all three as **MDL-WIDGET34** (no project needed), and the page builder refuses them with the widget named, so nothing is written; `alter page` refuses a bare name it has no entity for. Place the widget in a data view, list view, gallery or data grid and bind the attribute by name. - **An expression property written in brackets was silently dropped** (mendixlabs/mxcli#750) — `dynamicclasses: [ if $currentObject/Featured then 'a' else 'b' ]`, the spelling #750 proposes, parsed as a list that no writer reads: `check` was clean, `exec` said `Created page`, and the widget was stored with no dynamic class. `alter page … set DynamicClasses = [ … ]` said `Altered page` and changed nothing, and a column's `DynamicCellClass` stored the list's text — tokens fused, `[if$x/Ythen'a'else'b']` — as its expression. Measured on a copy of a Mendix 11.14.0 project with the pre-fix binary. `mxcli check` now reports **MDL-WIDGET32** for `DynamicClasses` and `DynamicCellClass` written as a list (no project needed), and ALTER refuses it, so `check -p` reports that too. Write the expression quoted. - **`describe odata client` lost a quote level on a literal credential** — Studio Pro stores a literal user name as the expression `'abc'`, quotes included. `describe` printed `HttpUsername: 'abc'`, and re-executing that output stored `abc`, an identifier. `ClientCertificate`, header keys, `Version`, `MetadataUrl` and `Folder` were printed unescaped and did not re-parse when they held a quote. Every value is now quoted so a re-exec stores exactly what was read; measured against a Studio Pro-authored client decoded before and after a round trip. - **An OData client's proxy constant written `@Module.Const` was stored with the `@`** — `ProxyHost` / `ProxyPort` / `ProxyUsername` / `ProxyPassword` are by-name references to a constant, and Studio Pro stores the bare name (with `ProxyType: Override`). `"@Module.Const"` named no constant, so the proxy resolved to nothing. `create`, `create or modify` and `alter` now store the bare name for the bare, `@` and quoted-`@` spellings. The constant may be a String or an Integer. diff --git a/mdl-examples/bug-tests/input-binding-without-context.fail.mdl b/mdl-examples/bug-tests/input-binding-without-context.fail.mdl new file mode 100644 index 0000000000..16245c9d62 --- /dev/null +++ b/mdl-examples/bug-tests/input-binding-without-context.fail.mdl @@ -0,0 +1,36 @@ +-- An input widget bound to an attribute where nothing supplies an object. +-- +-- `textbox t (Attribute: FullName)` at the top of a page — outside any data +-- view, list view, gallery or data grid — has no entity to qualify `FullName` +-- with, and the writer stores an unqualified name as `AttributeRef: null`. Plain +-- `mxcli check` said "Check passed!", `exec --no-check` (and ALTER PAGE … INSERT +-- at page level, which `check --references` never saw) reported success, and +-- mxbuild 11.13.0 then failed the page: +-- +-- [CE0544] "This widget can only function inside a data context — like a data +-- view, list view, or a page with parameters or variables." +-- [CE7005] "No value selection has been made. Please select a value." +-- +-- The same drop, per kind: textarea / datepicker / checkbox / radiobuttons / +-- dropdown (CE0544 + CE7005), dynamictext `Attribute:` (CE0402 "No value +-- specified."), combobox (CE0642 "Property 'Attribute' is required."). A +-- QUALIFIED attribute there is stored and fails all the same (CE0544 / CE2421 / +-- CE1365 / CE7247 "Move this widget into a data container", with CE7006), and +-- `Attribute: $P/Name` never parsed as an attribute path, so it was dropped even +-- inside a data view. +-- +-- MDL-WIDGET34 now refuses each of these at check time, and the page builder +-- refuses them with the widget named, so nothing is written. This script must +-- FAIL check. The valid forms are in input-binding-without-context.mdl. +create module ProbeInBind; + +create persistent entity ProbeInBind.Person ( + FullName: String(100) +); + +create page ProbeInBind.Edit ( + title: 'Edit', + layout: Atlas_Core.Atlas_Default +) { + textbox t (Attribute: FullName) +}; diff --git a/mdl-examples/bug-tests/input-binding-without-context.mdl b/mdl-examples/bug-tests/input-binding-without-context.mdl new file mode 100644 index 0000000000..f819cdba67 --- /dev/null +++ b/mdl-examples/bug-tests/input-binding-without-context.mdl @@ -0,0 +1,49 @@ +-- What the MDL-WIDGET34 refusal must NOT touch: every binding below has an +-- object to bind to. Executed on a copy of a Mendix 11.13.0 project, the pages +-- build at 0 errors. This script must PASS check. +-- +-- The refused forms are in input-binding-without-context.fail.mdl. +create module ProbeInBindOk; + +create enumeration ProbeInBindOk.Color ( Red 'Red', Blue 'Blue' ); + +create persistent entity ProbeInBindOk.Person ( + FullName: String(100), + Fav: Enumeration(ProbeInBindOk.Color) +); + +create page ProbeInBindOk.Edit ( + params: { $P: ProbeInBindOk.Person }, + title: 'Edit', + layout: Atlas_Core.Atlas_Default +) { + layoutgrid lg { + row r1 { + column c1 (desktopwidth: autofill) { + -- The data view supplies the object; bare and qualified both bind. + dataview dv (DataSource: $P) { + textbox tName (Attribute: FullName) + textbox tQualified (Attribute: ProbeInBindOk.Person.FullName) + combobox cbFav (Attribute: Fav) + dynamictext dtName (Attribute: FullName) + } + } + } + } + -- A list widget's rows are the object. + listview lv (DataSource: database ProbeInBindOk.Person) { + dynamictext lvName (Attribute: FullName) + } + datagrid dg (DataSource: database ProbeInBindOk.Person) { + column cName (Attribute: FullName, Caption: 'Name') + } +}; + +-- A snippet binds through a data view over its parameter, exactly as a page does. +create snippet ProbeInBindOk.PersonFields ( + params: { $P: ProbeInBindOk.Person } +) { + dataview sdv (DataSource: $P) { + textbox sName (Attribute: FullName) + } +}; diff --git a/mdl/executor/cmd_pages_builder_onchange_test.go b/mdl/executor/cmd_pages_builder_onchange_test.go index a1c154607c..79b3aaffd5 100644 --- a/mdl/executor/cmd_pages_builder_onchange_test.go +++ b/mdl/executor/cmd_pages_builder_onchange_test.go @@ -62,6 +62,9 @@ func TestBuildWidgetV3_OnChangeSurvivesBuilder(t *testing.T) { h := mkHierarchy(mod) withContainer(h, mod.ID, mod.ID) pb := newPageBuilder(&mock.MockBackend{}, h, "Mod") + // Inside a data container: with no entity in scope the binding + // has nothing to resolve against and the widget is refused. + pb.entityContext = "Mod.Ent" w, err := pb.buildWidgetV3(mkOnChangeWidget(mdlType, "w1")) if err != nil { diff --git a/mdl/executor/cmd_pages_builder_v3_widgets.go b/mdl/executor/cmd_pages_builder_v3_widgets.go index 3a02960f8b..8e27eaf987 100644 --- a/mdl/executor/cmd_pages_builder_v3_widgets.go +++ b/mdl/executor/cmd_pages_builder_v3_widgets.go @@ -456,6 +456,9 @@ func (pb *pageBuilder) buildTextBoxV3(w *ast.WidgetV3) (*pages.TextBox, error) { if attr := w.GetAttribute(); attr != "" { tb.AttributePath, tb.AttributeRefSteps = pb.resolveInputAttribute(attr) } + if err := pb.checkInputBinding(w, pb.entityContext); err != nil { + return nil, err + } // Forms$TextBox.IsPasswordBox. The writer always carried it; nothing parsed // it, so a describe → exec round trip turned a password field into a @@ -516,6 +519,9 @@ func (pb *pageBuilder) buildTextAreaV3(w *ast.WidgetV3) (*pages.TextArea, error) if attr := w.GetAttribute(); attr != "" { ta.AttributePath, ta.AttributeRefSteps = pb.resolveInputAttribute(attr) } + if err := pb.checkInputBinding(w, pb.entityContext); err != nil { + return nil, err + } // Handle Label if label := w.GetLabel(); label != "" { @@ -549,6 +555,9 @@ func (pb *pageBuilder) buildDatePickerV3(w *ast.WidgetV3) (*pages.DatePicker, er if attr := w.GetAttribute(); attr != "" { dp.AttributePath, dp.AttributeRefSteps = pb.resolveInputAttribute(attr) } + if err := pb.checkInputBinding(w, pb.entityContext); err != nil { + return nil, err + } // Handle Label if label := w.GetLabel(); label != "" { @@ -582,6 +591,9 @@ func (pb *pageBuilder) buildDropdownV3(w *ast.WidgetV3) (*pages.DropDown, error) if attr := w.GetAttribute(); attr != "" { dd.AttributePath, dd.AttributeRefSteps = pb.resolveInputAttribute(attr) } + if err := pb.checkInputBinding(w, pb.entityContext); err != nil { + return nil, err + } // Handle Label if label := w.GetLabel(); label != "" { @@ -615,6 +627,9 @@ func (pb *pageBuilder) buildCheckBoxV3(w *ast.WidgetV3) (*pages.CheckBox, error) if attr := w.GetAttribute(); attr != "" { cb.AttributePath, cb.AttributeRefSteps = pb.resolveInputAttribute(attr) } + if err := pb.checkInputBinding(w, pb.entityContext); err != nil { + return nil, err + } // Handle Label if label := w.GetLabel(); label != "" { @@ -683,6 +698,9 @@ func (pb *pageBuilder) buildRadioButtonsV3(w *ast.WidgetV3) (*pages.RadioButtons if attr := w.GetAttribute(); attr != "" { rb.AttributePath, rb.AttributeRefSteps = pb.resolveInputAttribute(attr) } + if err := pb.checkInputBinding(w, pb.entityContext); err != nil { + return nil, err + } // Handle OnChange (the "On change" client action) if err := pb.applyOnChangeV3(w, &rb.OnChangeAction); err != nil { @@ -781,6 +799,12 @@ func (pb *pageBuilder) buildDynamicTextV3(w *ast.WidgetV3) (*pages.DynamicText, // to `ContentParams: [{1} = X]`. Without this the Attribute was dropped, leaving // an orphaned `{1}` template with no parameter — which Studio Pro can't open // (NullReferenceException in ClientTemplateFormPart.CollectControls). + // Outside a data container that parameter binds nothing (CE0402). + if explicitParams == nil && len(autoGeneratedParams) == 0 { + if err := pb.checkInputBinding(w, pb.entityContext); err != nil { + return nil, err + } + } if attr := w.GetAttribute(); attr != "" && explicitParams == nil && len(autoGeneratedParams) == 0 { autoGeneratedParams = append(autoGeneratedParams, attr) if content == "" { diff --git a/mdl/executor/cmd_pages_input_binding_context.go b/mdl/executor/cmd_pages_input_binding_context.go new file mode 100644 index 0000000000..b16b114b50 --- /dev/null +++ b/mdl/executor/cmd_pages_input_binding_context.go @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "fmt" + "strings" + + "github.com/mendixlabs/mxcli/mdl/ast" + mdlerrors "github.com/mendixlabs/mxcli/mdl/errors" + "github.com/mendixlabs/mxcli/mdl/linter" +) + +// An `Attribute:` binding is only storable when there is an object to bind to. +// +// The writer stores a binding as a DomainModels$AttributeRef holding +// Module.Entity.Attribute and writes a null reference for anything shorter +// (attributeRefToGen), so a bare name nothing could qualify — `textbox t +// (Attribute: FullName)` at the top of a page, where there is no entity in +// scope — went out as `AttributeRef: null`. `exec` said "Created page", and +// mxbuild 11.13.0 reported, per widget kind: +// +// textbox/textarea/datepicker/checkbox/radiobuttons/dropdown +// CE0544 "This widget can only function inside a data context" + CE7005 +// dynamictext CE0402 "No value specified." +// combobox CE0642 "Property 'Attribute' is required." +// +// Qualifying the name does not rescue it: the reference is stored, but outside a +// data container there is no object of that entity to edit, and mxbuild rejects +// that too — CE0544/CE2421 on a text box, CE1365 on a dynamic text, CE7247 on a +// combo box ("Move this widget into a data container"), each with CE7006. +// +// `Attribute: $P/Name` is a third shape of the same drop: it does not parse as +// an attribute path but as a data-source expression, which no input builder +// reads, so it was dropped INSIDE a data view as well as outside one. +// +// `mxcli check -p … --references` already refused the first two on CREATE +// PAGE/SNIPPET (validatePageContextTree), but plain `mxcli check`, `exec +// --no-check` and ALTER PAGE did not. The check-time rule below needs no +// project; the builder guard catches what reaches the writer by any route. + +// inputBindingProblem says why w's `Attribute:` binding cannot be stored, or "" +// when it can (or the widget has none). +// +// c is the context the widget sits in. noEntity says that nothing could qualify +// a bare name here — the builder knows that; the check-time walk has no project +// to ask, passes false, and lets the context alone decide. +func inputBindingProblem(w *ast.WidgetV3, c pageArgContext, noEntity bool) string { + raw, present := lookupPropCI(w, "Attribute") + if !present || raw == nil { + return "" + } + kind := strings.ToLower(w.Type) + attr, isString := raw.(string) + if !isString { + return fmt.Sprintf("%s `%s`: `Attribute: %s` is not an attribute binding MDL can store — the widget "+ + "would be written with no binding at all, inside a data view or outside one. Bind the attribute by "+ + "name inside a data container over that object: `dataview dv (DataSource: $Param) { %s %s "+ + "(Attribute: Name) }` ($currentObject/Name is written `Name`)", + kind, w.Name, nonStringAttributeText(raw), kind, w.Name) + } + if attr == "" { + return "" + } + qualified := !strings.Contains(attr, "/") && strings.Count(attr, ".") >= 2 + if c.known && !c.present { + if qualified { + return fmt.Sprintf("%s `%s`: attribute `%s` is qualified, but the widget is not inside a data view, "+ + "list view, gallery or data grid, so there is no object of that entity for it to show or edit — "+ + "mxbuild rejects it (CE0544 \"This widget can only function inside a data context\", or \"Move "+ + "this widget into a data container\"). Place it inside a data container over that entity", + kind, w.Name, attr) + } + return fmt.Sprintf("%s `%s`: attribute `%s` has no entity to bind against — the widget is not inside "+ + "a data view, list view, gallery or data grid, so it would be written with no binding "+ + "(AttributeRef: null; mxbuild: %s). "+ + "Place it inside a data container, e.g. `dataview dv (DataSource: $Param) { %s %s (Attribute: %s) }`", + kind, w.Name, attr, unboundBuildError(kind), kind, w.Name, attr) + } + if noEntity && !qualified { + return fmt.Sprintf("%s `%s`: attribute `%s` has no entity to bind against — no enclosing data source "+ + "with a resolvable entity was found, so it would be written with no binding (AttributeRef: null). "+ + "Place it inside a data container or qualify it (Module.Entity.Attribute)", + kind, w.Name, attr) + } + return "" +} + +// unboundBuildError is what mxbuild 11.13.0 reports for a widget of this kind +// written with a null binding at the top of a page — measured per kind. +func unboundBuildError(kind string) string { + switch kind { + case "dynamictext": + return `CE0402 "No value specified."` + case "combobox": + return `CE0642 "Property 'Attribute' is required."` + } + return `CE0544 "This widget can only function inside a data context"` +} + +// nonStringAttributeText renders an `Attribute:` value that did not parse as an +// attribute path back into the spelling the author wrote, for the message. +func nonStringAttributeText(v any) string { + ds, ok := v.(*ast.DataSourceV3) + if !ok { + return fmt.Sprintf("%v", v) + } + switch { + case ds.ContextVariable != "": + return "$" + ds.ContextVariable + "/" + ds.Reference + case strings.HasPrefix(ds.Reference, "$"): + return ds.Reference + } + return ds.Type + " " + ds.Reference +} + +// checkInputBinding is the builder's refusal: the widget being built must not +// reach the writer with a binding the writer will turn into nothing. +func (pb *pageBuilder) checkInputBinding(w *ast.WidgetV3, entity string) error { + if msg := inputBindingProblem(w, pb.argCtx, entity == ""); msg != "" { + return mdlerrors.NewValidation(msg) + } + return nil +} + +// validateInputBindingContext is the check-time mirror (MDL-WIDGET34). It runs +// in the widget-tree walk, which has no project and so no entity: only the +// document-root context — known, and empty — refuses a string binding. ALTER PAGE's subtree walk has an unknown +// context and is left to the builder. +// +// A widget with a data source of its own binds its attributes against that +// source, so it is judged in the context it creates, not the one it sits in. +func validateInputBindingContext(w *ast.WidgetV3, c pageArgContext, locationPrefix string) []linter.Violation { + if w == nil { + return nil + } + if own := argContextForOwnAction(w, c); own != c { + c = own + } + msg := inputBindingProblem(w, c, false) + if msg == "" { + return nil + } + return []linter.Violation{{ + RuleID: "MDL-WIDGET34", + Severity: linter.SeverityError, + Message: locationPrefix + ": " + msg, + Suggestion: "An input or dynamic text shows an attribute of the object a data view, list view, gallery or data grid supplies — wrap it in one.", + }} +} diff --git a/mdl/executor/cmd_pages_input_binding_context_test.go b/mdl/executor/cmd_pages_input_binding_context_test.go new file mode 100644 index 0000000000..26b3ecd285 --- /dev/null +++ b/mdl/executor/cmd_pages_input_binding_context_test.go @@ -0,0 +1,163 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/model" +) + +// An input widget bound to an attribute where nothing supplies an object — +// `textbox t (Attribute: FullName)` at the top of a page — was built with its +// binding resolved to the bare name, which the writer cannot store: the widget +// went out with `AttributeRef: null`. `exec` said "Created page"; mxbuild +// 11.13.0 then reported CE0544 "This widget can only function inside a data +// context" plus CE7005 "No value selection has been made", CE0402 on a dynamic +// text and CE0642 "Property 'Attribute' is required" on a combo box. A +// QUALIFIED attribute there is stored, and is just as wrong: CE0544 / CE2421 / +// CE1365 / CE7247 "Move this widget into a data container". + +func testInputPB(entity string, argCtx pageArgContext) *pageBuilder { + return &pageBuilder{entityContext: entity, argCtx: argCtx, widgetScope: map[string]model.ID{}} +} + +func inputWidget(kind, name string, attr any) *ast.WidgetV3 { + return &ast.WidgetV3{Type: kind, Name: name, Properties: map[string]any{"Attribute": attr}} +} + +// The builder is the last gate before the writer, and ALTER PAGE reaches it +// without passing the check-time walk. It must refuse, naming the widget, rather +// than hand the writer a binding it will turn into null. +func TestBuildInputWithoutDataContextIsRefused(t *testing.T) { + cases := []struct { + kind, attr string + want []string + }{ + {"textbox", "FullName", []string{"`t`", "FullName", "data container"}}, + {"textarea", "Notes", []string{"`t`", "Notes", "data container"}}, + {"datepicker", "Born", []string{"`t`", "data container"}}, + {"checkbox", "Active", []string{"`t`", "data container"}}, + {"radiobuttons", "Fav", []string{"`t`", "data container"}}, + {"dropdown", "Fav", []string{"`t`", "data container"}}, + {"dynamictext", "FullName", []string{"`t`", "data container"}}, + // Qualified is stored, and fails the build all the same. + {"textbox", "AN.Person.FullName", []string{"`t`", "AN.Person.FullName", "data container"}}, + } + for _, c := range cases { + t.Run(c.kind+"/"+c.attr, func(t *testing.T) { + pb := testInputPB("", atDocumentRoot()) + _, err := pb.buildWidgetV3(inputWidget(c.kind, "t", c.attr)) + if err == nil { + t.Fatalf("%s bound to %q at page level was built — a bare name is written AttributeRef: null, a qualified one fails CE0544", c.kind, c.attr) + } + for _, w := range c.want { + if !strings.Contains(err.Error(), w) { + t.Errorf("refusal does not mention %q: %v", w, err) + } + } + }) + } +} + +// ALTER PAGE builds with an UNKNOWN context (the stored page is never walked). +// A qualified binding may be legitimate there — inside a container whose flow +// the project lacks, it is how DESCRIBE writes it — so only the binding the +// writer provably cannot store, a bare name with no entity, is refused. +func TestBuildInputUnknownContext(t *testing.T) { + pb := testInputPB("", pageArgContext{}) + if _, err := pb.buildWidgetV3(inputWidget("textbox", "t", "FullName")); err == nil { + t.Fatal("bare attribute with no entity in scope was built — the writer stores AttributeRef: null") + } else if !strings.Contains(err.Error(), "Module.Entity.Attribute") { + t.Errorf("refusal should offer the qualified form: %v", err) + } + if _, err := pb.buildWidgetV3(inputWidget("textbox", "t", "AN.Person.FullName")); err != nil { + t.Errorf("qualified attribute in an unknown context was refused: %v", err) + } +} + +// The control: inside a data container the same widgets build. +func TestBuildInputInsideDataContext(t *testing.T) { + for _, kind := range []string{"textbox", "textarea", "datepicker", "checkbox", "radiobuttons", "dropdown", "dynamictext"} { + pb := testInputPB("AN.Person", pageArgContext{known: true, present: true}) + if _, err := pb.buildWidgetV3(inputWidget(kind, "t", "FullName")); err != nil { + t.Errorf("%s inside a data container was refused: %v", kind, err) + } + } +} + +// `Attribute: $P/FullName` does not parse as an attribute path at all — it lands +// as a data-source expression, which no input builder reads — so the binding +// was dropped INSIDE a data view as well as outside one. +func TestBuildInputVariableRootedAttributeIsRefused(t *testing.T) { + ds := &ast.DataSourceV3{Type: "parameter", Reference: "$P"} + pb := testInputPB("AN.Person", pageArgContext{known: true, present: true}) + _, err := pb.buildWidgetV3(inputWidget("textbox", "t", ds)) + if err == nil { + t.Fatal("`Attribute: $P/…` was built — no builder reads it, so the binding is dropped") + } + if !strings.Contains(err.Error(), "`t`") { + t.Errorf("refusal does not name the widget: %v", err) + } +} + +// Check time: the same refusal from the widget-tree walk, which runs with no +// project, so `mxcli check script.mdl` reports it. +func TestValidateInputBindingWithoutDataContext(t *testing.T) { + registry := LoadWidgetRegistry("") + if registry == nil { + t.Fatal("LoadWidgetRegistry returned nil") + } + hits := func(tree []*ast.WidgetV3, subtree bool) (n int, msg string) { + vs := validateWidgetTree(tree, registry, "page AN.P") + if subtree { + vs = validateWidgetSubtree(tree, registry, "alter AN.P") + } + for _, v := range vs { + if v.RuleID == "MDL-WIDGET34" { + n++ + msg = v.Message + } + } + return + } + + for _, kind := range []string{"textbox", "textarea", "datepicker", "checkbox", "radiobuttons", "dropdown", "dynamictext", "combobox"} { + if n, _ := hits([]*ast.WidgetV3{inputWidget(kind, "t", "FullName")}, false); n != 1 { + t.Errorf("%s at page level: MDL-WIDGET34 = %d, want 1", kind, n) + } + } + if n, msg := hits([]*ast.WidgetV3{inputWidget("textbox", "t", "AN.Person.FullName")}, false); n != 1 { + t.Errorf("qualified textbox at page level: MDL-WIDGET34 = %d, want 1", n) + } else if !strings.Contains(msg, "CE0544") { + t.Errorf("message should name the build error: %s", msg) + } + // A plain container supplies nothing. + nested := []*ast.WidgetV3{{Type: "container", Name: "c", Children: []*ast.WidgetV3{inputWidget("textbox", "t", "FullName")}}} + if n, _ := hits(nested, false); n != 1 { + t.Errorf("textbox in a plain container: MDL-WIDGET34 = %d, want 1", n) + } + // Controls: a data view supplies the object; ALTER cannot say. + dv := []*ast.WidgetV3{{ + Type: "dataview", Name: "dv", + Properties: map[string]any{"DataSource": &ast.DataSourceV3{Type: "parameter", Reference: "$P"}}, + Children: []*ast.WidgetV3{inputWidget("textbox", "t", "FullName")}, + }} + if n, msg := hits(dv, false); n != 0 { + t.Errorf("textbox inside a data view was flagged: %s", msg) + } + if n, msg := hits([]*ast.WidgetV3{inputWidget("textbox", "t", "FullName")}, true); n != 0 { + t.Errorf("ALTER PAGE insert was flagged at check time, where the enclosing context is unknown: %s", msg) + } + // `$P/Attr` is dropped wherever it is written. + varRooted := []*ast.WidgetV3{{ + Type: "dataview", Name: "dv", + Properties: map[string]any{"DataSource": &ast.DataSourceV3{Type: "parameter", Reference: "$P"}}, + Children: []*ast.WidgetV3{inputWidget("textbox", "t", &ast.DataSourceV3{Type: "parameter", Reference: "$P"})}, + }} + if n, _ := hits(varRooted, false); n != 1 { + t.Errorf("`Attribute: $P/…` inside a data view: MDL-WIDGET34 = %d, want 1", n) + } +} diff --git a/mdl/executor/cmd_pages_popup_test.go b/mdl/executor/cmd_pages_popup_test.go index 65d69ec79b..1ed0df64ca 100644 --- a/mdl/executor/cmd_pages_popup_test.go +++ b/mdl/executor/cmd_pages_popup_test.go @@ -56,6 +56,10 @@ func TestBuildPageV3_PopupDefaults(t *testing.T) { // ClientTemplateParameter. func TestBuildDynamicTextV3_AttributeBinds(t *testing.T) { pb := newPopupPageBuilder() + // Inside a data container over M.Item. Without one there is no entity to + // qualify `Title` with, and the parameter used to be "bound" to the bare + // name — which the writer stores as a null AttributeRef (CE0402). + pb.entityContext = "M.Item" w := &ast.WidgetV3{Type: "dynamictext", Name: "txt", Properties: map[string]any{"Attribute": "Title"}} dt, err := pb.buildDynamicTextV3(w) if err != nil { @@ -74,6 +78,9 @@ func TestBuildDynamicTextV3_AttributeBinds(t *testing.T) { if p.AttributeRef == "" && p.Expression == "" && p.SourceVariable == "" { t.Error("parameter has no binding (AttributeRef/Expression/SourceVariable all empty)") } + if p.AttributeRef != "M.Item.Title" { + t.Errorf("AttributeRef = %q, want M.Item.Title — anything shorter is written as null", p.AttributeRef) + } } // A content-less dynamictext with no binding is unchanged (no panic, no params). diff --git a/mdl/executor/validate_widgets.go b/mdl/executor/validate_widgets.go index 9ef2ee5c3a..32d5ab4d6a 100644 --- a/mdl/executor/validate_widgets.go +++ b/mdl/executor/validate_widgets.go @@ -211,6 +211,8 @@ func validateWidgetTreeIn(widgets []*ast.WidgetV3, registry *WidgetRegistry, loc // The widget's OWN action is judged in the context IT establishes, not the // one it sits in — a list widget's onClick is row-scoped (ako/mxcli#552). out = append(out, validateShowPageArguments(w, argContextForOwnAction(w, argCtx), locationPrefix)...) + // An `Attribute:` binding with no object to bind to is written empty. + out = append(out, validateInputBindingContext(w, argCtx, locationPrefix)...) // Unknown-property warning applies only to built-in widgets; pluggable // widgets get the stricter def.json check (MDL-WIDGET01) above, and // object-list items are validated by the object-list engine. diff --git a/mdl/executor/widget_engine.go b/mdl/executor/widget_engine.go index 5dd93bfb84..50f8fc2c80 100644 --- a/mdl/executor/widget_engine.go +++ b/mdl/executor/widget_engine.go @@ -1297,6 +1297,14 @@ func (e *PluggableWidgetEngine) resolveMapping(mapping PropertyMapping, w *ast.W // for those three and the hidden-property guard cannot catch it. // Not writing them in the first place does not depend on that data. attr = w.GetAttribute() + // Bound against the enclosing object rather than a source of the + // widget's own: with none, the binding is written empty (a combo + // box's CE0642 "Property 'Attribute' is required"). + if entity := e.entityContextFor(mapping.PropertyKey); entity == e.pageBuilder.entityContext { + if err := e.pageBuilder.checkInputBinding(w, entity); err != nil { + return nil, err + } + } } if attr != "" { // Against THIS property's datasource entity, which is the shared From aaa564570e588bac921f1fd14c59d5d9a9ceb44a Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 14:16:12 +0000 Subject: [PATCH 44/47] fix(lint): pin documented Starlark field values to what the API emits The rule-authoring skill documented microflow_type as "microflow"/"nanoflow" and example_microflow.star (shipped into every project by `mxcli init`) as "Microflow"/"Nanoflow". Microflows() returns the catalog value raw, so a rule written from either compares against a value that never occurs and silently reports nothing. Neither source mentioned "RULE", which microflows() also yields. The skill's entity table also omitted the four audit fields (has_created_date, has_changed_date, has_owner, has_changed_by). Fix the docs, and add the test that makes the class unrepeatable: run a Starlark rule over a fixture holding every stored kind to observe the field names and enum values the API actually hands a rule, then assert - the skill's entity/microflow tables list exactly those fields and values, - every `.entity_type`/`.microflow_type` header line in shipped rules does too, - every comparison of those fields in a shipped rule uses an emitted value (the #1164 shape). No behaviour change: normalizing MicroflowType would break user rules that already compare "MICROFLOW" correctly. Fixes mendixlabs/mxcli#1178 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011AvRv9GAQJbrHgrgMmfsBM --- .claude/lint-rules/example_microflow.star | 2 +- .claude/lint-rules/mccabe_complexity.star | 2 +- .../skills/fix-issue/findings/cmd-mxcli.jsonl | 1 + .../skills/mendix/write-lint-rules/SKILL.md | 6 +- mdl/linter/starlark_documented_values_test.go | 285 ++++++++++++++++++ 5 files changed, 293 insertions(+), 3 deletions(-) create mode 100644 mdl/linter/starlark_documented_values_test.go diff --git a/.claude/lint-rules/example_microflow.star b/.claude/lint-rules/example_microflow.star index 1a6d4d9a3f..6785430caf 100644 --- a/.claude/lint-rules/example_microflow.star +++ b/.claude/lint-rules/example_microflow.star @@ -14,7 +14,7 @@ # .name - Simple name (e.g., "ACT_ProcessOrder") # .qualified_name - Full name (e.g., "MyModule.ACT_ProcessOrder") # .module_name - Module name -# .microflow_type - "Microflow" or "Nanoflow" +# .microflow_type - "MICROFLOW", "NANOFLOW" or "RULE" # .description - Documentation # .return_type - Return type # .parameter_count - Number of parameters diff --git a/.claude/lint-rules/mccabe_complexity.star b/.claude/lint-rules/mccabe_complexity.star index 208bf8b65c..dea1199110 100644 --- a/.claude/lint-rules/mccabe_complexity.star +++ b/.claude/lint-rules/mccabe_complexity.star @@ -18,7 +18,7 @@ # .name - Simple name (e.g., "ProcessOrder") # .qualified_name - Full name (e.g., "MyModule.ProcessOrder") # .module_name - Module name -# .microflow_type - "MICROFLOW" or "NANOFLOW" +# .microflow_type - "MICROFLOW", "NANOFLOW" or "RULE" # .description - Documentation # .return_type - Return type # .parameter_count - Number of parameters diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl index 2350194b7c..b328c6d7c0 100644 --- a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl +++ b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl @@ -126,3 +126,4 @@ {"area": "cmd/mxcli/diag", "date": "2026-09-23", "symptom": "`mxcli diag loop-report` showed all 5 `test` runs as 'did not close' although every test passed, and inflated the `-c` (111) and `exec` (180) counts in the same log. Reported as 'the command apparently skips the summary record on success too' \u2014 which is not what happens: `test` returns normally, PersistentPostRun fires, and the session_end IS written.", "cause": "mxcli runs mxcli. Measured from a real `mxcli test` with MXCLI_LOG_DIR pointed at a scratch dir: one parent session_start (pid 1071) followed by THREE child session_starts \u2014 `-c DESCRIBE SETTINGS`, `-c SHOW MODULES`, and an `exec` of the generated runner \u2014 before a single test executes. `new`, `eval`, `tui` and the LSP self-spawn the same way (six os.Executable() sites). buildInvocations segmented on 'next session_end OR next session_start, whichever comes first', so a child's start closed the parent's invocation and the parent's own end landed on whatever was open by then. session_end carried no pid, so pairing by process was impossible.", "file": "`mdl/diaglog/diaglog.go` (pid on session_end; parentPIDEnv marker set once in Init and inherited by every child), `cmd/mxcli/diag_loop_report.go` (buildInvocations pairs by pid with the positional rule as fallback; spawned runs excluded from the table and wall time, counted on their own line), tests `cmd/mxcli/diag_loop_report_test.go`", "insight": "The segmentation rule documented itself as exact 'for sequential invocations, which is what an agent loop produces' \u2014 and the thing that breaks that assumption is the tool itself, not concurrency by the user. When a tool can invoke itself, EVERY per-process measurement over it needs a parent link, not just a pid: a pid alone fixes the pairing but still counts three phantom agent calls per test run. The marker belongs on the ENVIRONMENT, not at each spawn site: exec.Command inherits the parent's environment (explicitly via os.Environ(), implicitly when Cmd.Env is nil), so one os.Setenv in Init covers all six self-spawn sites and any added later \u2014 six edits that would each have to be remembered become zero. Second-order trap: spawned runs must be excluded from WALL TIME too, not just the count, because a child's seconds are already inside its parent's; the test asserts 10s for a parent with three 1s children, and the reverted code says 3. Prove-by-revert done on the measured record shape: the positional rule gives Invocations=4 (want 1), Unclosed=1 for a parent whose tests all passed, Wall=3 (want 10).", "refs": ["ako/mxcli#617", "ako/mxcli#629"]} {"area": "cmd/mxcli/syntax", "date": "2026-09-23", "symptom": "`mxcli syntax page datasource` documented `DataSource: MICROFLOW Module.MF($P)`. That form is a parse error: `dataview dv (datasource: microflow M.DS_X($State))` gives 'line 2:15 no viable alternative at input datasource'. Only the NAMED form `M.DS_X(State: $State)` parses. Hit in a real build, diagnosed from the error rather than the doc.", "cause": "The syntax entry was written from the intended shape rather than from something that had been run through the parser. Nothing checks it: the Syntax and Example fields are free text.", "file": "`cmd/mxcli/syntax/features_page.go` (page.datasource entry now shows `MICROFLOW Module.MF(Param: $P)` and states that the positional form is a parse error)", "insight": "CLAUDE.md deliberately points at `mxcli syntax` instead of restating syntax, so that it cannot go stale \u2014 which makes a wrong entry there worse than a wrong entry in prose, because it is the thing consulted INSTEAD of checking. The cost lands in the agent loop: read it, write it, fail to parse, diagnose, retry. Measured before reaching for the systemic guard: 42 of 164 syntax examples fail `mxcli check` today, but the large majority are fragments by design (a microflow body like `IF \u2026`, a widget snippet like `DATAGRID \u2026`, an OQL fragment) and are legitimately not standalone top-level MDL \u2014 so a blanket 'every example must parse' test would be mostly noise, and making it useful needs a way to mark which examples are standalone. Measuring that first is what stopped a plausible-sounding guard from being built wrong.", "refs": ["ako/mxcli#630"]} {"area": "cmd/mxcli/test", "date": "2026-09-23", "symptom": "Windows: `mxcli test tests/ -p MyApp.mpr --local` fails with `local runtime: starting mxbuild serve: mxbuild --serve did not become ready` after caching a Linux ELF in %USERPROFILE%\\.mxcli\\mxbuild, and there is \"no flag, environment variable, or mechanism to redirect mxcli to the Windows mxbuild.exe already present in the Studio Pro installation\"", "cause": "Two layers. The platform part (downloading/exec'ing the Linux binary, serving from the cache instead of the resolved binary) was already fixed by #916 and #1122, both after the reporter's v0.21.0. What remained on main: `test` never registered `--mxbuild-path` — `run` gained it in #1125, but `test --local` boots through the same `ResolveMxBuildForLocal` and prints the same 'pass --mxbuild-path' guidance while answering `unknown flag`. `RunOptions` had no field and `localAppOptions` never set `LocalAppOptions.MxBuildPath`, though StartLocalApp honoured it. No env override existed anywhere", "file": "`cmd/mxcli/main.go` + `cmd_test_run.go` (flag), `testrunner/runner.go` + `localapp_options.go` (plumbing), `docker/mxbuild_platform.go` (`MxBuildPathEnv`, read in `resolveMxBuildForLocalOn` after the flag)", "insight": "**The guard for #1125 asserted its invariant against one command** — `TestErrorGuidanceNamesAFlagThatExists` checked only `runCmd`, while the guidance it polices is emitted by a resolver two commands share. When a test pins 'the advertised flag exists', enumerate the callers of the code that ADVERTISES it, not the command the report named; it now iterates `run` and `test`. **Before fixing a platform report, date it against the fixes**: the reporter's first two suggestions ('download platform-correct binary', 'auto-discover Studio Pro') were already on main, and re-implementing them would have been churn — only the override was missing. Put the env var in the resolver, not the CLI, so `run --local` and `test --local` both get it from one line; a flag-level env read would have been one more per-command copy to drift. Control: the env test runs as goos=windows with an unmatched version, so without the override it fails fast with the 'Linux binary cannot run natively on windows' refusal instead of hitting the CDN; removing only the `MxBuildPath:` line in localAppOptions fails the plumbing test for both runners. **Unverified**: no Windows host; code-level with OS-injected tests", "refs": ["mendixlabs/mxcli#1086", "mendixlabs/mxcli#1125", "mendixlabs/mxcli#916", "mendixlabs/mxcli#1122"]} +{"area": "cmd/mxcli", "date": "2026-09-25", "symptom": "Starlark rule API docs name microflow_type values the linter never returns: the write-lint-rules skill said \"microflow\"/\"nanoflow\", example_microflow.star (copied into every project by mxcli init) said \"Microflow\"/\"Nanoflow\", only mccabe_complexity.star had \"MICROFLOW\"/\"NANOFLOW\", and none listed \"RULE\". The skill's entity table also omitted has_created_date/has_changed_date/has_owner/has_changed_by.", "cause": "LintContext.Microflows() passes the catalog's MicroflowType through raw (MICROFLOW/NANOFLOW/RULE) while Entities() CASE-normalizes EntityType to TitleCase, so the two adjacent iterators have opposite conventions and each doc author guessed. Nothing tied the documented literals to what the API emits, so two passes over the same skill file (59db6e7b, #1165) fixed instances and left this one.", "file": ".claude/skills/mendix/write-lint-rules/SKILL.md, .claude/lint-rules/example_microflow.star, .claude/lint-rules/mccabe_complexity.star, mdl/linter/starlark_documented_values_test.go", "insight": "Fix the class, not the row: observe the emitted field names and enum values by running a Starlark rule (dir(e), e.entity_type) over a fixture holding every stored kind, then assert the skill tables, the shipped rules' `# .field - ...` headers, and every `.entity_type/.microflow_type ==` comparison in shipped rules against that observed set. Equality, not subset, for the docs -- an undocumented value (RULE) is a flavour a rule silently mistreats. Normalizing MicroflowType instead would have broken every user rule already comparing \"MICROFLOW\" correctly.", "refs": "mendixlabs/mxcli#1178, mendixlabs/mxcli#1164"} diff --git a/.claude/skills/mendix/write-lint-rules/SKILL.md b/.claude/skills/mendix/write-lint-rules/SKILL.md index 3e6bf7d842..8dc5e3844f 100644 --- a/.claude/skills/mendix/write-lint-rules/SKILL.md +++ b/.claude/skills/mendix/write-lint-rules/SKILL.md @@ -164,6 +164,10 @@ def check(): | `validation_rule_count` | int | Number of validation rules | | `has_event_handlers` | bool | True if entity has event handlers | | `is_external` | bool | True if entity is from an external service | +| `has_created_date` | bool | True if the entity stores `createdDate` (an audit member, not counted in `attribute_count`) | +| `has_changed_date` | bool | True if the entity stores `changedDate` | +| `has_owner` | bool | True if the entity stores `owner` | +| `has_changed_by` | bool | True if the entity stores `changedBy` | ### microflow | Property | Type | Example | @@ -173,7 +177,7 @@ def check(): | `qualified_name` | string | `"Sales.ACT_Customer_Create"` | | `module_name` | string | `"Sales"` | | `folder` | string | `"microflows/Customer"` — folder path within module | -| `microflow_type` | string | `"microflow"` or `"nanoflow"` | +| `microflow_type` | string | exactly `"MICROFLOW"`, `"NANOFLOW"` or `"RULE"` — upper-case, unlike `entity_type`. `microflows()` yields all three flavours, so a rule meant for microflows only must filter on `"MICROFLOW"` | | `description` | string | Documentation text | | `return_type` | string | Return type | | `parameter_count` | int | Number of parameters | diff --git a/mdl/linter/starlark_documented_values_test.go b/mdl/linter/starlark_documented_values_test.go new file mode 100644 index 0000000000..36f45864e6 --- /dev/null +++ b/mdl/linter/starlark_documented_values_test.go @@ -0,0 +1,285 @@ +// SPDX-License-Identifier: Apache-2.0 + +package linter_test + +import ( + "database/sql" + "os" + "path/filepath" + "regexp" + "slices" + "sort" + "strconv" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/catalog" + "github.com/mendixlabs/mxcli/mdl/linter" + _ "modernc.org/sqlite" +) + +// A Starlark rule that compares a field to a value the API never emits does not +// error: it skips every item, reports nothing, and its output is byte-identical to +// a clean project. ARCH002/ARCH003 compared entity_type to "PERSISTENT" that way +// (#1164), and the rule-authoring skill documented microflow_type as "microflow" / +// "nanoflow" while the example rule every project gets said "Microflow" / +// "Nanoflow" -- neither of which the linter has ever returned (#1178). +// +// Two passes over the same skill file fixed instances and missed the next one, +// because nothing tied the documented literals to the API. These tests do: the +// emitted values and field names are observed by running a rule against a fixture +// holding every stored kind, and every documented literal must be one of them. + +const ( + lintSkillPath = "../../.claude/skills/mendix/write-lint-rules/SKILL.md" + lintRulesDir = "../../.claude/lint-rules" +) + +// observed is what a Starlark rule actually receives from entities() and +// microflows(): the field names on each struct and the distinct enum values. +type observed struct { + entityFields, microflowFields []string + entityTypes, microflowTypes []string +} + +func observeStarlarkAPI(t *testing.T) observed { + t.Helper() + src := ` +RULE_ID = "TEST_OBSERVE" +RULE_NAME = "Observe" +DESCRIPTION = "reports every value the API hands a rule" +CATEGORY = "quality" +SEVERITY = "info" + +def check(): + out = [] + for e in entities(): + out.append(violation(message="entity|" + e.entity_type + "|" + ",".join(dir(e)))) + for mf in microflows(): + out.append(violation(message="microflow|" + mf.microflow_type + "|" + ",".join(dir(mf)))) + return out +` + path := filepath.Join(t.TempDir(), "observe.star") + if err := os.WriteFile(path, []byte(src), 0o644); err != nil { + t.Fatal(err) + } + r, err := linter.LoadStarlarkRule(path) + if err != nil { + t.Fatalf("LoadStarlarkRule: %v", err) + } + var o observed + for _, v := range r.Check(linter.NewLintContextFromDB(everyKindFixtureDB(t))) { + parts := strings.SplitN(v.Message, "|", 3) + if len(parts) != 3 { + t.Fatalf("unexpected message %q", v.Message) + } + fields := strings.Split(parts[2], ",") + switch parts[0] { + case "entity": + o.entityTypes = appendUnique(o.entityTypes, parts[1]) + o.entityFields = fields + case "microflow": + o.microflowTypes = appendUnique(o.microflowTypes, parts[1]) + o.microflowFields = fields + } + } + // The fixture holds three kinds of each; seeing fewer means the fixture or + // the iterator dropped rows, and every assertion below would be vacuous. + if len(o.entityTypes) != 3 || len(o.microflowTypes) != 3 { + t.Fatalf("observed entity types %v and microflow types %v, want three of each", + o.entityTypes, o.microflowTypes) + } + return o +} + +// One row per value the catalog builder stores: EntityType from +// mdl/catalog/builder_modules.go, MicroflowType from builder_microflows.go (all +// three flavours share one table). +func everyKindFixtureDB(t *testing.T) catalog.CatalogDB { + t.Helper() + db, err := sql.Open("sqlite", ":memory:") + if err != nil { + t.Fatal(err) + } + stmts := []string{ + `CREATE TABLE modules (Id TEXT, Name TEXT, Source TEXT)`, + `INSERT INTO modules VALUES ('m1', 'Sales', '')`, + `CREATE TABLE entities ( + Id TEXT, Name TEXT, QualifiedName TEXT, ModuleName TEXT, Folder TEXT, + EntityType TEXT, Description TEXT, Generalization TEXT, + AttributeCount INTEGER, AccessRuleCount INTEGER, ValidationRuleCount INTEGER, + HasEventHandlers INTEGER, IsExternal INTEGER, + HasCreatedDate INTEGER, HasChangedDate INTEGER, + HasOwner INTEGER, HasChangedBy INTEGER)`, + `INSERT INTO entities VALUES + ('e1','A','Sales.A','Sales','','PERSISTENT','','',0,0,0,0,0,0,0,0,0), + ('e2','B','Sales.B','Sales','','NON_PERSISTENT','','',0,0,0,0,0,0,0,0,0), + ('e3','C','Sales.C','Sales','','VIEW','','',0,0,0,0,0,0,0,0,0)`, + `CREATE TABLE microflows ( + Id TEXT, Name TEXT, QualifiedName TEXT, ModuleName TEXT, Folder TEXT, + MicroflowType TEXT, Description TEXT, ReturnType TEXT, + ParameterCount INTEGER, ActivityCount INTEGER, Complexity INTEGER)`, + `INSERT INTO microflows VALUES + ('f1','MF','Sales.MF','Sales','','MICROFLOW','','',0,0,1), + ('f2','NF','Sales.NF','Sales','','NANOFLOW','','',0,0,1), + ('f3','RU','Sales.RU','Sales','','RULE','','',0,0,1)`, + } + for _, s := range stmts { + if _, err := db.Exec(s); err != nil { + t.Fatalf("fixture %q: %v", s, err) + } + } + return catalog.WrapSqlDB(db) +} + +// The skill's entity and microflow tables must name exactly the fields the +// struct exposes, and the enum rows must list exactly the values it emits: +// a documented value that never occurs is a rule that silently reports nothing, +// and an undocumented one ("RULE") is a flavour a rule silently mistreats. +func TestLintSkillDocumentsWhatTheAPIEmits(t *testing.T) { + o := observeStarlarkAPI(t) + skill, err := os.ReadFile(lintSkillPath) + if err != nil { + t.Fatal(err) + } + + for _, tc := range []struct { + section, enumField string + fields, values []string + }{ + {"entity", "entity_type", o.entityFields, o.entityTypes}, + {"microflow", "microflow_type", o.microflowFields, o.microflowTypes}, + } { + t.Run(tc.section, func(t *testing.T) { + rows := skillTableRows(t, string(skill), tc.section) + var documented []string + for name := range rows { + documented = append(documented, name) + } + assertSameSet(t, "fields in the skill's "+tc.section+" table", documented, tc.fields) + + row, ok := rows[tc.enumField] + if !ok { + t.Fatalf("skill's %s table has no %s row", tc.section, tc.enumField) + } + assertSameSet(t, "values documented for "+tc.enumField+" in the skill", + quotedLiterals(row), tc.values) + }) + } +} + +// The comment headers of the shipped rules are what a new rule is copied from +// (example_microflow.star exists for exactly that, and `mxcli init` ships it into +// every project). Each `.entity_type` / `.microflow_type` line there must list +// exactly the emitted values. +func TestShippedRuleHeadersDocumentWhatTheAPIEmits(t *testing.T) { + o := observeStarlarkAPI(t) + want := map[string][]string{"entity_type": o.entityTypes, "microflow_type": o.microflowTypes} + header := regexp.MustCompile(`^#\s+\.(entity_type|microflow_type)\s+-\s+(.*)$`) + + seen := 0 + for _, path := range shippedRules(t) { + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + for i, line := range strings.Split(string(data), "\n") { + m := header.FindStringSubmatch(line) + if m == nil { + continue + } + seen++ + assertSameSet(t, filepath.Base(path)+":"+strconv.Itoa(i+1)+" ."+m[1], + quotedLiterals(m[2]), want[m[1]]) + } + } + if seen == 0 { + t.Fatal("no .entity_type / .microflow_type header lines found -- the pattern no longer matches the files") + } +} + +// Every literal a shipped rule compares one of these fields to must be a value +// the API emits. This is the #1164 shape directly: `entity_type == "PERSISTENT"`. +func TestShippedRulesCompareOnlyToEmittedValues(t *testing.T) { + o := observeStarlarkAPI(t) + want := map[string][]string{"entity_type": o.entityTypes, "microflow_type": o.microflowTypes} + cmp := regexp.MustCompile(`\.(entity_type|microflow_type)\s*(?:==|!=|in|not in)\s*(\[[^\]]*\]|\([^)]*\)|"[^"]*")`) + + seen := 0 + for _, path := range shippedRules(t) { + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + for _, m := range cmp.FindAllStringSubmatch(string(data), -1) { + seen++ + for _, lit := range quotedLiterals(m[2]) { + if !slices.Contains(want[m[1]], lit) { + t.Errorf("%s compares .%s to %q, which the linter never returns (it returns %v): "+ + "the rule silently skips every item", filepath.Base(path), m[1], lit, want[m[1]]) + } + } + } + } + if seen == 0 { + t.Fatal("no comparisons found -- the pattern no longer matches the shipped rules") + } +} + +func shippedRules(t *testing.T) []string { + t.Helper() + paths, err := filepath.Glob(filepath.Join(lintRulesDir, "*.star")) + if err != nil || len(paths) == 0 { + t.Fatalf("no shipped rules under %s: %v", lintRulesDir, err) + } + return paths +} + +// skillTableRows returns the "###
" table of the skill as +// property name -> the rest of the row. +func skillTableRows(t *testing.T, skill, section string) map[string]string { + t.Helper() + _, after, ok := strings.Cut(skill, "\n### "+section+"\n") + if !ok { + t.Fatalf("skill has no ### %s section", section) + } + row := regexp.MustCompile("^\\|\\s*`([a-z_]+)`\\s*\\|(.*)$") + rows := map[string]string{} + for _, line := range strings.Split(after, "\n") { + if strings.HasPrefix(line, "### ") { + break + } + if m := row.FindStringSubmatch(line); m != nil { + rows[m[1]] = m[2] + } + } + if len(rows) == 0 { + t.Fatalf("### %s section has no property rows", section) + } + return rows +} + +func quotedLiterals(s string) []string { + var out []string + for _, m := range regexp.MustCompile(`"([^"]*)"`).FindAllStringSubmatch(s, -1) { + out = appendUnique(out, m[1]) + } + return out +} + +func assertSameSet(t *testing.T, what string, got, want []string) { + t.Helper() + g, w := slices.Clone(got), slices.Clone(want) + sort.Strings(g) + sort.Strings(w) + if !slices.Equal(g, w) { + t.Errorf("%s: documented %v, API emits %v", what, g, w) + } +} + +func appendUnique(s []string, v string) []string { + if slices.Contains(s, v) { + return s + } + return append(s, v) +} From 1ae6737f00c3a30287447006fbc1b92132e93d17 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 25 Sep 2026 14:18:40 +0000 Subject: [PATCH 45/47] fix: resolve every widget keyword's design properties through the $Type it writes groupbox, tabcontainer, navigationtree, menubar, simplemenubar, button, row and column had no entry in the keyword -> theme-key table, so each resolved to itself ("groupbox"), which no design-properties.json defines. The validator skipped the widget and the builder saw only the "Widget" base group: a custom colour on a group box's ColorPicker "Style", or on a top-level row/column's DivContainer "Background color", checked clean, exec'd, and failed the build with CE6085 "Unknown option #ff0000". radiobuttons and snippetcall named keys mxbuild does not apply ("RadioButtons", "SnippetCall" - CE6083 when a theme declares them; the classes are RadioButtonGroup and SnippetCallWidget), and header/footer named groups for widgets the builder writes as a Forms$DivContainer. The $Type table used the non-existent Forms$RadioButtons and Forms$Gallery and lacked Forms$ImageViewer (dynamic image's storage name). The keyword table now maps keyword -> the $Type the builder writes, and the key is read off the single $Type table, so the inline and stored paths cannot drift. Tests build every keyword through buildWidgetV3 and compare the $Type, read buildWidgetV3's case list from source so a new keyword cannot land unmapped, and hold every key to the generated metamodel's class name (measured: theme keys are qualified class names). A row inside a layoutgrid, a column inside a row, and a dataview footer are slots whose DesignProperties the builder drops; check now warns MDL-WIDGET07 instead of staying silent, and a slot keyword under a pluggable widget (a datagrid column) is left unresolved as before. Co-Authored-By: Claude Opus 5.5 --- .../fix-issue/findings/mdl-executor.jsonl | 1 + .../design-property-keyword-mappings.mdl | 63 +++ .../design_property_keyword_keys_test.go | 382 ++++++++++++++++++ mdl/executor/design_property_routing_test.go | 89 ++-- mdl/executor/rule_id_uniqueness_test.go | 5 + mdl/executor/theme_reader.go | 203 ++++++---- mdl/executor/validate_alter_styling.go | 4 +- mdl/executor/validate_design_properties.go | 79 +++- .../validate_design_properties_test.go | 18 +- mdl/executor/widget_rule_ids_test.go | 5 + 10 files changed, 708 insertions(+), 141 deletions(-) create mode 100644 mdl-examples/bug-tests/design-property-keyword-mappings.mdl create mode 100644 mdl/executor/design_property_keyword_keys_test.go diff --git a/.claude/skills/fix-issue/findings/mdl-executor.jsonl b/.claude/skills/fix-issue/findings/mdl-executor.jsonl index 9e7ad45b83..bf2785ea2f 100644 --- a/.claude/skills/fix-issue/findings/mdl-executor.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-executor.jsonl @@ -707,3 +707,4 @@ {"area": "mdl/executor", "symptom": "describe → exec of Administration.Account_New fails `mx check` with [CE0642] \"Property 'Caption' is required.\" at Combo box 'comboBox2' (Account_Edit comboBox4 too); check and exec report success", "cause": "The ComboBox caption is an EXPRESSION (optionsSourceAssociationCaptionType=expression, optionsSourceAssociationCaptionExpression='$currentObject/Description'); describe read only the attribute caption, so the widget was rewritten with none. The write side already worked via the explicit-property pass, but MDL-WIDGET06 claimed both keys 'will be dropped'", "file": "`mdl/executor/cmd_pages_describe_parse.go` (combobox branch), `cmd_pages_describe_output.go`, `validate_widget_explicit_writable.go` (`persistedByExplicitPass`), `validate_widgets.go` (MDL-WIDGET06)", "insight": "**Resolve pluggable-widget properties by key before theorising**: a 20-line script mapping each Property's TypePointer to its PropertyKey through the widget's own Type.ObjectType settled the cause in one run, where the ndsl dump shows only anonymous values. Then TEST THE WRITE SIDE before building one: adding the two storage keys to the describe output by hand persisted both and built clean, which shrank the fix to describe + a false warning — no new syntax. A validator rule that asserts 'not persisted' must be tied to what the write path handles (here: the explicit pass writes Expression/TextTemplate/Attribute and scalar types), or it goes stale when the writer grows; the #643 test pinned the stale claim. A dedicated extractor for a 'known' pluggable widget silently drops every property it does not map — generic widgets emit them, known ones do not", "refs": ["ako/mxcli#664", "ako/mxcli#643"], "ce": ["CE0642"], "rules": ["MDL-WIDGET06"], "date": "2026-09-25"} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "A `label` widget's DesignProperties were never checked: describe → check --references of FeedbackModule.ShareFeedback_Logo (Feedback v4.0.2, Atlas Core 4.1.3, Mendix 11.13.0) warned MDL-WIDGET11 'renamed' on the containers but not on label1's 'Spacing bottom': 'Outer none' (CE6087 in mxbuild). Same gap on the write side: `label l (DesignProperties: ['Style': '#ff0000'])` checked clean, exec'd, and failed mx check with [CE6085] \"Unknown option #ff0000 for design property Style.\" at Label", "cause": "The `label` keyword (PR #670, writes Forms$Label) was never added to mdlKeywordToDesignPropsKey, so resolveDesignPropsKey returned 'label' — no design-properties.json key. validateWidgetDesignProps skips a widget whose key is absent (meant for unknown pluggables), and the builder's GetPropertiesForWidget returned only the 'Widget' base group, so the Label's own ColorPicker 'Style' was unknown and a free colour fell to the Option default. bsonTypeToDesignPropsKey already had Forms$Label → Label; only the keyword half was missing", "file": "mdl/executor/theme_reader.go", "insight": "Adding a native widget keyword has a third registration nobody asks for: mdlKeywordToDesignPropsKey. Missing it is silent in both directions — the validator's 'no theme key → skip' rule turns an unmapped keyword into approval, and the builder still gets the Widget base group, so Spacing/Hide on write correctly and only the type-specific properties (Label's ColorPicker Style) mis-type. Test a type-specific property with an off-list value; a Widget-base property passes either way. Quick audit: a keyword should resolve to the same key as its stored $Type — groupbox (GroupBox, which has a ColorPicker Style), tabcontainer, navigationtree, menubar, simplemenubar, row/column (LayoutGridRow/Column) are still unmapped. Also seen: MDL-WIDGET12 warns on a ColorPicker free colour that the builder writes as Custom and mxbuild accepts (0 errors) — a pre-existing false positive, not changed here", "refs": ["#670", "#679"], "rules": ["MDL-WIDGET11", "MDL-WIDGET12"], "ce": ["CE6085", "CE6087"]} {"area": "mdl/executor", "date": "2026-09-25", "symptom": "`mxcli check` passed a microflow with a commit inside a loop \u2014 one database round trip per iteration \u2014 that `mxcli lint` already flagged as CONV011. The defect surfaced only at project-wide lint time, long after the write.", "cause": "CONV011 reads the STORED model, so it cannot speak until `exec` has written the microflow; `check` reads the MDL and had no equivalent rule. The gap is temporal, not a missing capability on either side.", "file": "`mdl/executor/validate_commit_in_loop.go` (MDL-PERF01, hooked in `validate_microflow.go`), test `validate_commit_in_loop_test.go`, example `mdl-examples/bug-tests/1186-commit-in-loop.mdl`", "insight": "**When adding a check-time rule that anticipates an existing lint rule, pin the BOUNDARY to the lint rule's, not to the better one, and say why in the code.** A `while true` is built as an ExclusiveMerge back-edge rather than a LoopedActivity, so CONV011 (which walks LoopedActivity) does not flag a commit inside one. A commit there is arguably still N+1, and the tempting move is to be more correct \u2014 but two rules for one concept that disagree on what counts is precisely how a pair drifts, and this repo already has CONV010's three successive short allowlists as the worked example. If the case is worth reporting it is worth reporting in BOTH, and the stored-model rule is the one that sees the built flow. The test that pins this carries a control on the control: `while true` is exempt, `while ` is not, so the exemption cannot silently become 'never flag a while'. Name the sibling rule in the message (`lint reports this as CONV011`) so a reader hitting one recognises the other rather than filing it twice. Also worth reusing: `checkReturnInLoop` already had the depth-tracking walk over Loop/While/If/EnumSplit/InheritanceSplit \u2014 copying its shape got the nesting cases right for free.", "refs": ["ako/mxcli#681", "mendixlabs/mxcli#1186"], "rules": ["MDL-PERF01"]} +{"area": "mdl/executor", "date": "2026-09-25", "symptom": "A design property on `groupbox`, `tabcontainer`, `navigationtree`, `menubar`, `simplemenubar`, `button`, `row` or `column` was never checked (typos passed `check`), and a custom colour on a ColorPicker in that widget's own group checked clean, exec'd, and failed the build: CE6085 \"Unknown option #ff0000 for design property Style\" at Group box, and \"Unknown option #00ff00 for design property Background color\" at Container for a top-level `row`/`column`.", "cause": "mdlKeywordToDesignPropsKey (keyword -> theme key) was a second hand-written copy of bsonTypeToDesignPropsKey ($Type -> key) and had drifted from it and from the builder: eight keywords had no entry (fell through as themselves, no theme defines \"groupbox\", validator skipped, builder saw only the Widget base group so a ColorPicker value typed as Option); `radiobuttons`/`snippetcall` named storage-ish keys (\"RadioButtons\", \"SnippetCall\") mxbuild refuses with CE6083, and the $Type table used the non-existent storage name Forms$RadioButtons and lacked Forms$ImageViewer (dynamic image's real storage name), Forms$GroupBox, Forms$TabControl and the menu widgets. `header`/`footer` named groups for widgets built as Forms$DivContainer.", "file": "`mdl/executor/theme_reader.go` (mdlKeywordStorageType, storageTypeThemeKeys, resolveDesignPropsKey), `mdl/executor/validate_design_properties.go` (designPropsSlotOf), tests `design_property_keyword_keys_test.go`, example `mdl-examples/bug-tests/design-property-keyword-mappings.mdl`", "insight": "**A theme key is the Mendix CLASS (qualified) name, not the storage name, plus its ancestors.** Settled in one mxbuild run by adding probe groups to a COPY's themesource//web/design-properties.json (one Toggle per candidate key) and setting each on a real widget: accepted = the key applies, CE6083 \"not supported by your theme\" = it does not. TabContainer/RadioButtonGroup/SnippetCallWidget accepted, TabControl/RadioButtons/SnippetCall refused, and both Button and ActionButton accepted on an action button (ancestor groups apply). That makes the generated metamodel the oracle: `codec.DefaultRegistry.Lookup(\"Forms$X\")` returns the gen struct whose Go type name IS the class name, so the key table is now tested against it instead of against memory. Structural fix: the keyword table maps keyword -> the $Type the builder writes, and the key is read off the $Type table, so one table holds keys; a test BUILDS every keyword through buildWidgetV3 and compares $Type, and another reads buildWidgetV3's `case` strings with go/parser so a new keyword cannot land unmapped. Trap to skip: `row`/`column`/`footer` are widgets (DivContainer) only at the top level — inside layoutgrid/row/dataview they are slots whose DesignProperties the builder silently DROPS (buildLayoutGridRowV3/ColumnV3 never call applyWidgetAppearance), and inside a pluggable they are object-list entries (datagrid `column`). Mapping the keyword without the parent context would validate a dropped value against the wrong group; check now warns MDL-WIDGET07 on the dropped ones. Measured 17 described PedApp pages: 18 warnings before and after, identical.", "refs": ["ako/mxcli#686"], "ce": ["CE6085", "CE6083"], "rules": ["MDL-WIDGET11", "MDL-WIDGET07"]} diff --git a/mdl-examples/bug-tests/design-property-keyword-mappings.mdl b/mdl-examples/bug-tests/design-property-keyword-mappings.mdl new file mode 100644 index 0000000000..c62cf8d055 --- /dev/null +++ b/mdl-examples/bug-tests/design-property-keyword-mappings.mdl @@ -0,0 +1,63 @@ +-- ============================================================================ +-- Widget keywords whose design properties were neither checked nor typed +-- ============================================================================ +-- +-- Symptom (Mendix 11.13.0, Atlas Core 4.1.3): this page checked clean and +-- exec'd, and mxbuild then refused it: +-- [CE6085] "Unknown option #ff0000 for design property Style." at Group box 'gbCustom' +-- [CE6085] "Unknown option #00ff00 for design property Background color." at Container 'r1' +-- [CE6085] "Unknown option #0000ff for design property Background color." at Container 'k1' +-- and a typo in any design property of a groupbox, tabcontainer, +-- navigationtree, menubar, simplemenubar, button, row or column went unreported. +-- +-- Cause: the MDL keyword → theme key table had no entry for those keywords, so +-- each resolved to itself ("groupbox"), which no design-properties.json +-- defines. The validator skipped the widget; the builder saw only the "Widget" +-- base group, so a ColorPicker's custom colour was written as an Option. +-- `radiobuttons` and `snippetcall` named keys mxbuild does not apply +-- ("RadioButtons", "SnippetCall"; the classes are RadioButtonGroup and +-- SnippetCallWidget), and `header`/`footer` named groups for what the builder +-- writes as a Forms$DivContainer. +-- +-- Fix: a keyword now names the $Type it writes (mdlKeywordStorageType) and the +-- key is read off that $Type — the table the stored-widget path already used — +-- so the two cannot drift. A top-level `row`/`column` is a DivContainer; a row +-- inside a layoutgrid, a column inside a row, and a dataview footer are parts +-- of their parent whose design properties mxcli does not write, and `check` +-- now says so (MDL-WIDGET07) instead of staying silent. +-- +-- Verify: `mxcli check --references` — no MDL-WIDGET11 below (the custom +-- colours draw MDL-WIDGET12 until the ColorPicker fix lands); change +-- 'Callout style' to 'Callout' and it must warn MDL-WIDGET11. exec, then +-- `mxcli docker check` — 0 errors. +-- ============================================================================ + +create or replace page MyFirstModule.DesignPropKeywords_Test +( Title: 'Design property keyword mappings', Layout: Atlas_Core.Atlas_Default ) +{ + container c1 { + groupbox gbCustom (Caption: 'Custom', DesignProperties: ['Style': '#ff0000', 'Callout style': on]) { + dynamictext t1 (Content: 'x') + } + groupbox gbSwatch (Caption: 'Swatch', DesignProperties: ['Style': 'Brand Primary']) { + dynamictext t2 (Content: 'y') + } + tabcontainer tc1 (DesignProperties: ['Style': 'Pills', 'Tab position': 'Center', 'Justify': on]) { + tabpage tp1 (Caption: 'One') { + dynamictext t3 (Content: 'z') + } + } + navigationtree nt1 (profile: 'Responsive', DesignProperties: ['Hide icons': on]) + row r1 (DesignProperties: ['Background color': '#00ff00']) { + column rc1 { + dynamictext t4 (Content: 'r') + } + } + column k1 (DesignProperties: ['Background color': '#0000ff', 'Card style': on]) { + dynamictext t5 (Content: 'k') + } + } +} +/ + +describe page MyFirstModule.DesignPropKeywords_Test; diff --git a/mdl/executor/design_property_keyword_keys_test.go b/mdl/executor/design_property_keyword_keys_test.go new file mode 100644 index 0000000000..586339dc7b --- /dev/null +++ b/mdl/executor/design_property_keyword_keys_test.go @@ -0,0 +1,382 @@ +// SPDX-License-Identifier: Apache-2.0 + +package executor + +import ( + "go/ast" + "go/parser" + "go/token" + "reflect" + "sort" + "strconv" + "strings" + "testing" + + mdlast "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/mdl/backend/mock" + "github.com/mendixlabs/mxcli/mdl/visitor" + "github.com/mendixlabs/mxcli/model" + "github.com/mendixlabs/mxcli/modelsdk/codec" + _ "github.com/mendixlabs/mxcli/modelsdk/gen/pages" // registers the Forms$ types + "github.com/mendixlabs/mxcli/sdk/pages" +) + +// ownGroupThemeRegistry is the renamed-theme fixture plus the Atlas Core 4.1.3 +// groups of the widgets whose MDL keyword had no theme-key mapping. Copied from +// themesource/atlas_core/web/design-properties.json, trimmed to what the tests +// touch. DivContainer is replaced wholesale: its "Background color" is a +// ColorPicker in Atlas, which is what makes a custom colour on `row`/`column` +// build-breaking. +func ownGroupThemeRegistry(t *testing.T) *ThemeRegistry { + t.Helper() + reg := renamedThemeRegistry(t) + swatches := []ThemeOption{{Name: "Brand Primary"}, {Name: "Brand Secondary"}} + reg.WidgetProperties["GroupBox"] = []ThemeProperty{ + {Name: "Style", Type: "ColorPicker", Options: swatches}, + {Name: "Callout style", Type: "Toggle"}, + } + reg.WidgetProperties["TabContainer"] = []ThemeProperty{ + {Name: "Style", Type: "ToggleButtonGroup", Options: []ThemeOption{{Name: "Pills"}, {Name: "Lined"}}}, + {Name: "Tab position", Type: "ToggleButtonGroup", Options: []ThemeOption{{Name: "Left"}, {Name: "Center"}}}, + {Name: "Justify", Type: "Toggle"}, + } + for _, k := range []string{"NavigationTree", "MenuBar", "SimpleMenuBar"} { + reg.WidgetProperties[k] = []ThemeProperty{{Name: "Hide icons", Type: "Toggle"}} + } + reg.WidgetProperties["DivContainer"] = []ThemeProperty{ + {Name: "Background color", Type: "ColorPicker", Options: swatches}, + {Name: "Card style", Type: "Toggle"}, + } + reg.WidgetProperties["Button"] = []ThemeProperty{ + {Name: "Size", Type: "ToggleButtonGroup", Options: []ThemeOption{{Name: "Small"}, {Name: "Large"}}}, + } + return reg +} + +// Each of these keywords writes a widget whose theme group Atlas declares, and +// none of them resolved to it: the keyword fell through as-is ("groupbox"), no +// design-properties.json defines that key, and validateWidgetDesignProps +// returned early. A typo in a group box's design property checked clean and was +// written — Studio Pro then reports CE6083 "not supported by your theme". +func TestValidateDesignProperties_OwnGroupWidgetsAreChecked(t *testing.T) { + reg := ownGroupThemeRegistry(t) + vs := allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + groupbox gb (Caption: 'G', DesignProperties: ['Nonexistent': 'x']) + tabcontainer tc (DesignProperties: ['Nonexistent': 'x']) { tabpage tp (Caption: 'T') } + navigationtree nt (profile: 'Responsive', DesignProperties: ['Nonexistent': on]) + menubar mb (profile: 'Responsive', DesignProperties: ['Nonexistent': on]) + simplemenubar smb (profile: 'Responsive', DesignProperties: ['Nonexistent': on]) + row r (DesignProperties: ['Nonexistent': on]) { column rc } + column c (DesignProperties: ['Nonexistent': on]) + button b (Caption: 'B', DesignProperties: ['Nonexistent': on]) + header h (DesignProperties: ['Nonexistent': on]) +}`, reg) + + for _, name := range []string{"gb", "tc", "nt", "mb", "smb", "r", "c", "b", "h"} { + var found bool + for _, v := range vs { + if v.RuleID == "MDL-WIDGET11" && strings.Contains(v.Message, `widget "`+name+`"`) && + strings.Contains(v.Message, "not defined") { + found = true + } + } + if !found { + t.Errorf("widget %q: undefined design property not reported — the widget was not checked", name) + } + } +} + +// CONTROL: what Atlas offers each of those widgets must not warn — its own +// group's properties and the "Widget" base. +func TestValidateDesignProperties_OwnGroupValidPropsPass(t *testing.T) { + reg := ownGroupThemeRegistry(t) + vs := allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + groupbox gb (Caption: 'G', DesignProperties: ['Style': 'Brand Primary', 'Callout style': on, 'Align self': 'Left']) + tabcontainer tc (DesignProperties: ['Style': 'Pills', 'Tab position': 'Center', 'Justify': on]) { tabpage tp (Caption: 'T') } + navigationtree nt (profile: 'Responsive', DesignProperties: ['Hide icons': on]) + menubar mb (profile: 'Responsive', DesignProperties: ['Hide icons': on]) + simplemenubar smb (profile: 'Responsive', DesignProperties: ['Hide icons': on]) + row r (DesignProperties: ['Background color': 'Brand Primary', 'Card style': on]) { column rc } + column c (DesignProperties: ['Card style': on, 'Spacing': ['margin-bottom': 'M']]) + button b (Caption: 'B', DesignProperties: ['Size': 'Small']) +}`, reg) + for _, v := range vs { + t.Errorf("unexpected %s: %s", v.RuleID, v.Message) + } +} + +// The write path resolves the same key to TYPE each value. Unmapped, a group +// box's ColorPicker "Style" and a row's/column's DivContainer "Background color" +// were unknown to the builder, so a free colour fell through to the "option" +// default and mxbuild refused the page (measured on a copy of PedApp, 11.13.0): +// +// [CE6085] "Unknown option #ff0000 for design property Style." at Group box 'gbCustom' +// [CE6085] "Unknown option #00ff00 for design property Background color." at Container 'r1' +func TestBuildWidget_OwnGroupColourIsCustom(t *testing.T) { + reg := ownGroupThemeRegistry(t) + prog, errs := visitor.Build(`create page M.P (layout: Atlas_Core.Atlas_Default) { + groupbox gbCustom (Caption: 'G', DesignProperties: ['Style': '#ff0000']) + groupbox gbSwatch (Caption: 'G', DesignProperties: ['Style': 'Brand Primary']) + row r1 (DesignProperties: ['Background color': '#00ff00']) { column rc } + column k1 (DesignProperties: ['Background color': '#0000ff']) +}`) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + want := map[string]string{"gbCustom": "custom", "gbSwatch": "option", "r1": "custom", "k1": "custom"} + for _, w := range prog.Statements[0].(*mdlast.CreatePageStmtV3).Widgets { + pb := &pageBuilder{backend: &mock.MockBackend{}, widgetScope: map[string]model.ID{}, themeRegistry: reg} + built, err := pb.buildWidgetV3(w) + if err != nil { + t.Fatalf("%s: %v", w.Name, err) + } + dps := built.(interface{ GetBaseWidget() *pages.BaseWidget }).GetBaseWidget().DesignProperties + if len(dps) != 1 { + t.Fatalf("%s: %d design properties written, want 1", w.Name, len(dps)) + } + if got := dps[0].ValueType; got != want[w.Name] { + t.Errorf("%s: %q written as %q, want %q", w.Name, dps[0].Key, got, want[w.Name]) + } + } +} + +// A row inside a layoutgrid, a column inside a row, and a dataview's footer are +// SLOTS, not widgets: buildLayoutGridRowV3 / buildLayoutGridColumnV3 and the +// dataview footer split never call applyWidgetAppearance, so their design +// properties are dropped on write. Resolving the keyword as a DivContainer there +// would validate against a group the value never reaches; staying silent reads +// as approval of a value that is thrown away. +func TestValidateDesignProperties_SlotDesignPropsAreReportedDropped(t *testing.T) { + reg := ownGroupThemeRegistry(t) + vs := allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + layoutgrid lg { + row nestedRow (DesignProperties: ['Card style': on]) { + column nestedCol (DesignProperties: ['Card style': on]) { + dynamictext t (Content: 'x') + } + } + } + row topRow { column topRowCol (DesignProperties: ['Card style': on]) } + dataview dv { + footer ft (DesignProperties: ['Card style': on]) { dynamictext f (Content: 'x') } + } +}`, reg) + for _, name := range []string{"nestedRow", "nestedCol", "topRowCol", "ft"} { + var found bool + for _, v := range vs { + if v.RuleID == "MDL-WIDGET07" && strings.Contains(v.Message, `"`+name+`"`) && + strings.Contains(v.Message, "dropped") { + found = true + } + } + if !found { + t.Errorf("slot %q: design properties dropped on write but not reported (%d violations)", name, len(vs)) + } + } + for _, v := range vs { + if v.RuleID != "MDL-WIDGET07" { + t.Errorf("a slot must not be validated as a widget, got %s: %s", v.RuleID, v.Message) + } + } +} + +// CONTROL: the same keywords as a CHILD of a pluggable widget belong to the +// widget's own object lists (a data grid `column`), which the pluggable engine +// builds — not a DivContainer and not a layout-grid slot. They must stay +// unvalidated, as before. +func TestValidateDesignProperties_PluggableChildKeywordsNotResolved(t *testing.T) { + reg := ownGroupThemeRegistry(t) + vs := allDesignPropViolations(t, `create page M.P (layout: Atlas_Core.Atlas_Default) { + datagrid dg (DataSource: database M.E) { + column colA (Attribute: Name, DesignProperties: ['Nonexistent': on]) + } +}`, reg) + for _, v := range vs { + t.Errorf("pluggable child reported: %s: %s", v.RuleID, v.Message) + } +} + +// The keyword table is DERIVED from what the builder writes: keyword → stored +// $Type → theme key. This holds the first hop to the builder itself — every +// keyword is built and its $Type compared — so the table cannot drift from it +// the way `radiobuttons` → "RadioButtons" did (the builder writes +// Forms$RadioButtonGroup; mxbuild accepts a "RadioButtonGroup" design property on +// it and refuses a "RadioButtons" one with CE6083). +func TestKeywordStorageTypesMatchBuilder(t *testing.T) { + src := `create page M.P (layout: Atlas_Core.Atlas_Default) { + container k_container + customcontainer k_customcontainer + header k_header + footer k_footer + controlbar k_controlbar + template k_template + filter k_filter + row k_row { column rc } + column k_column + actionbutton k_actionbutton (Caption: 'x') + linkbutton k_linkbutton (Caption: 'x') + button k_button (Caption: 'x') + textbox k_textbox (Attribute: Name) + textarea k_textarea (Attribute: Name) + datepicker k_datepicker (Attribute: D) + checkbox k_checkbox (Attribute: B) + radiobuttons k_radiobuttons (Attribute: B) + dropdown k_dropdown (Attribute: E) + dataview k_dataview + listview k_listview + layoutgrid k_layoutgrid + dynamictext k_dynamictext (Content: 'a') + label k_label (Content: 'a') + title k_title (Content: 'a') + staticimage k_staticimage + dynamicimage k_dynamicimage + navigationlist k_navigationlist + snippetcall k_snippetcall (Snippet: M.S) + tabcontainer k_tabcontainer { tabpage tp (Caption: 'a') } + groupbox k_groupbox (Caption: 'a') + scrollcontainer k_scrollcontainer { region center { placeholder Main } } + navigationtree k_navigationtree (profile: 'Responsive') + menubar k_menubar (profile: 'Responsive') + simplemenubar k_simplemenubar (profile: 'Responsive') + placeholder k_placeholder +}` + prog, errs := visitor.Build(src) + if len(errs) > 0 { + t.Fatalf("parse error: %v", errs[0]) + } + seen := map[string]bool{} + for _, w := range prog.Statements[0].(*mdlast.CreatePageStmtV3).Widgets { + kw := strings.ToLower(w.Type) + seen[kw] = true + pb := &pageBuilder{ + backend: &mock.MockBackend{}, widgetScope: map[string]model.ID{}, + paramEntityNames: map[string]string{}, paramScope: map[string]model.ID{}, + entityContext: "M.E", + execCache: &executorCache{createdSnippets: map[string]*createdSnippetInfo{ + "M.S": {ID: "s1", Name: "S", ModuleName: "M"}}}, + } + built, err := pb.buildWidgetV3(w) + if err != nil { + t.Errorf("%s: %v", kw, err) + continue + } + if got, want := built.GetTypeName(), mdlKeywordStorageType[kw]; got != want { + t.Errorf("keyword %q builds %s, but mdlKeywordStorageType says %q", kw, got, want) + } + } + for kw := range mdlKeywordStorageType { + if !seen[kw] { + t.Errorf("mdlKeywordStorageType has %q but this test does not build it", kw) + } + } +} + +// Every keyword buildWidgetV3 dispatches on must be in mdlKeywordStorageType or +// be named here with the reason it writes no native widget of its own. Read +// from the switch's source so a new `case` cannot land without a decision. +func TestKeywordStorageTypesCoverBuilderDispatch(t *testing.T) { + notANativeWidget := map[string]string{ + "legacydatagrid": "refused (not implemented)", + "slot": "refused outside a fragment body", + "tabpage": "refused outside a tabcontainer", + "region": "refused outside a scrollcontainer", + "item": "refused outside a navigationlist", + "text": "refused: writes Forms$Text, which Mendix does not have", + "statictext": "refused: writes Forms$Text, which Mendix does not have", + "image": "pluggable (com.mendix.widget.web.image.Image) — pluggableKeywordIDs", + } + for _, kw := range buildWidgetV3Cases(t) { + _, mapped := mdlKeywordStorageType[kw] + _, exempt := notANativeWidget[kw] + if mapped == exempt { + t.Errorf("buildWidgetV3 case %q: mapped=%v exempt=%v — exactly one must hold", kw, mapped, exempt) + } + } +} + +func buildWidgetV3Cases(t *testing.T) []string { + t.Helper() + f, err := parser.ParseFile(token.NewFileSet(), "cmd_pages_builder_v3.go", nil, 0) + if err != nil { + t.Fatalf("parse builder source: %v", err) + } + var out []string + ast.Inspect(f, func(n ast.Node) bool { + fn, ok := n.(*ast.FuncDecl) + if !ok || fn.Name.Name != "buildWidgetV3" { + return true + } + ast.Inspect(fn.Body, func(n ast.Node) bool { + cc, ok := n.(*ast.CaseClause) + if !ok { + return true + } + for _, e := range cc.List { + if lit, ok := e.(*ast.BasicLit); ok && lit.Kind == token.STRING { + s, _ := strconv.Unquote(lit.Value) + out = append(out, s) + } + } + return true + }) + return false + }) + if len(out) < 20 { + t.Fatalf("found only %d cases in buildWidgetV3 — the source walk is broken, not the table", len(out)) + } + sort.Strings(out) + return out +} + +// Every theme key is a Mendix CLASS name: the qualified name, not the storage +// name. Measured with probe groups added to a copy of PedApp's theme (11.13.0): +// mxbuild accepts a "TabContainer" / "RadioButtonGroup" / "SnippetCallWidget" +// design property and refuses "TabControl" / "RadioButtons" / "SnippetCall" with +// CE6083 "not supported by your theme". The generated metamodel carries exactly +// that mapping (Forms$TabControl decodes into gen type TabContainer), so the key +// table is held to it — the one deliberate departure is an ANCESTOR class, which +// Studio Pro applies too (Atlas declares "Button"; a probe "ActionButton" group +// was accepted as well). +func TestDesignPropsKeysAreMetamodelClassNames(t *testing.T) { + ancestor := map[string]string{"ActionButton": "Button"} + // Keyed by the Forms$ spelling: the generated metamodel registers only that + // prefix. The Pages$ twins in bsonTypeToDesignPropsKey are the same classes. + for suffix, key := range storageTypeThemeKeys { + storage := "Forms$" + suffix + factory, ok := codec.DefaultRegistry.Lookup(storage) + if !ok { + t.Errorf("%s is not a type in the metamodel — nothing is ever stored as it", storage) + continue + } + class := reflect.TypeOf(factory()).Elem().Name() + want := class + if a, ok := ancestor[class]; ok { + want = a + } + if key != want { + t.Errorf("%s (class %s) maps to theme key %q, want %q", storage, class, key, want) + } + } +} + +// Both prefixes and each keyword's $Type resolve, so the derivation never falls +// through to the bare keyword for a native widget. +func TestResolveDesignPropsKey_NativeKeywords(t *testing.T) { + want := map[string]string{ + "groupbox": "GroupBox", "tabcontainer": "TabContainer", "navigationtree": "NavigationTree", + "menubar": "MenuBar", "simplemenubar": "SimpleMenuBar", "row": "DivContainer", + "column": "DivContainer", "header": "DivContainer", "footer": "DivContainer", + "button": "Button", "radiobuttons": "RadioButtonGroup", "snippetcall": "SnippetCallWidget", + "label": "Label", "container": "DivContainer", "dynamicimage": "DynamicImageViewer", + } + for kw, key := range want { + if got := resolveDesignPropsKey(strings.ToUpper(kw)); got != key { + t.Errorf("resolveDesignPropsKey(%q) = %q, want %q", kw, got, key) + } + } + for kw, storage := range mdlKeywordStorageType { + if _, ok := bsonTypeToDesignPropsKey[storage]; !ok { + t.Errorf("keyword %q writes %s, which has no theme key", kw, storage) + } + } +} diff --git a/mdl/executor/design_property_routing_test.go b/mdl/executor/design_property_routing_test.go index 8dec6141d8..26ebf4fa8e 100644 --- a/mdl/executor/design_property_routing_test.go +++ b/mdl/executor/design_property_routing_test.go @@ -149,36 +149,30 @@ func TestDesignPropertyAssignment(t *testing.T) { } } -// The two hand-written maps describe the same concept from opposite directions, -// and the $Type one had NO CALLERS until this change — so it had never been held -// to anything at all. Neither is a subset of the other, and every exclusive entry -// has a measured reason, so this pins both sets rather than asserting a -// consistency that does not hold. +// The keyword path and the stored-$Type path resolve through ONE key table: a +// keyword names the $Type it writes (mdlKeywordStorageType) and the key is read +// off that $Type (bsonTypeToDesignPropsKey). The two used to be separate +// hand-written keyword→key and $Type→key maps, and they disagreed — `groupbox`, +// `tabcontainer`, the menu widgets, `row`/`column`/`button` missing on one side, +// `radiobuttons`/`snippetcall`/`header`/`footer` naming keys no stored widget of +// that type has. // -// $Type-only — correct, and the reason the stored path resolves MORE than the -// inline one: +// What remains exclusive is one-directional and pinned: types MDL cannot build +// but Studio Pro can store, reachable only from a stored $Type. // -// - "DataGrid" and "Gallery" are the NATIVE widgets. The MDL keywords no longer -// produce them: `datagrid` resolves through pluggableKeywordIDs to Data grid -// 2's widget id, `gallery` to the pluggable Gallery. Atlas declares both -// native groups, and a Studio Pro-authored widget of either type is reachable -// only from its $Type. -// -// Keyword-only — inert today, and at least two of them wrong: -// -// - `header` and `footer` map to "Header"/"Footer", but MDL builds BOTH as a -// Forms$DivContainer (cmd_pages_builder_v3_layout.go), so a stored one -// resolves to "DivContainer". Atlas declares no Header or Footer group, so -// the inline lookup misses and validateWidgetDesignProps returns early — -// silence reading as approval, the shape pluggableKeywordIDs already records -// for combobox/gallery/image. Not corrected here: it changes what existing -// pages validate against, which is its own change. -// - `snippetcall` maps to "SnippetCall" (stored Forms$SnippetCallWidget); Atlas -// declares no such group either. +// - "DataGrid" is the NATIVE data grid. MDL's `datagrid` resolves through +// pluggableKeywordIDs to Data grid 2's widget id; a Studio Pro-authored +// Forms$DataGrid is reachable only from its $Type. +// - "ReferenceSelector": there is no `referenceselector` builder (the keyword +// parses and exec refuses it as an unsupported widget type). func TestBsonTypeAndKeywordDesignPropsKeysAgree(t *testing.T) { fromKeyword := map[string]bool{} - for _, v := range mdlKeywordToDesignPropsKey { - fromKeyword[v] = true + for kw := range mdlKeywordStorageType { + key := resolveDesignPropsKey(kw) + if _, ok := storageTypeThemeKeys[strings.TrimPrefix(mdlKeywordStorageType[kw], "Forms$")]; !ok { + t.Errorf("keyword %q writes %s, which has no theme key", kw, mdlKeywordStorageType[kw]) + } + fromKeyword[key] = true } fromBsonType := map[string]bool{} for _, v := range bsonTypeToDesignPropsKey { @@ -188,35 +182,28 @@ func TestBsonTypeAndKeywordDesignPropsKeysAgree(t *testing.T) { t.Fatal("a map is empty — a passing run would prove nothing") } - exclusive := func(a, b map[string]bool) map[string]bool { - out := map[string]bool{} - for k := range a { - if !b[k] { - out[k] = true - } + for k := range fromKeyword { + if !fromBsonType[k] { + t.Errorf("keyword-only key %q — a keyword resolved to a key no stored $Type has", k) } - return out - } - eq := func(got, want map[string]bool, label string) { - t.Helper() - for k := range got { - if !want[k] { - t.Errorf("%s gained %q — measure which theme group that widget resolves to "+ - "on BOTH paths before adding it, and say so in this test's comment", label, k) - } + } + storedOnly := map[string]bool{} + for k := range fromBsonType { + if !fromKeyword[k] { + storedOnly[k] = true } - for k := range want { - if !got[k] { - t.Errorf("%s lost %q — if it is now reachable from both maps, confirm they "+ - "agree on the group rather than just deleting the expectation", label, k) - } + } + want := map[string]bool{"DataGrid": true, "ReferenceSelector": true} + for k := range storedOnly { + if !want[k] { + t.Errorf("$Type-only key %q gained — if MDL cannot build that widget, add it here with the reason", k) + } + } + for k := range want { + if !storedOnly[k] { + t.Errorf("$Type-only key %q lost — if a keyword now builds it, drop it from this list", k) } } - - eq(exclusive(fromBsonType, fromKeyword), - map[string]bool{"DataGrid": true, "Gallery": true}, "$Type-only keys") - eq(exclusive(fromKeyword, fromBsonType), - map[string]bool{"Header": true, "Footer": true, "SnippetCall": true}, "keyword-only keys") } // The mutator accessor returns raw storage facts, not a resolved key — the diff --git a/mdl/executor/rule_id_uniqueness_test.go b/mdl/executor/rule_id_uniqueness_test.go index 70d86c2bf2..3eb62b202c 100644 --- a/mdl/executor/rule_id_uniqueness_test.go +++ b/mdl/executor/rule_id_uniqueness_test.go @@ -45,6 +45,11 @@ var ruleIDsSharedDeliberately = map[string]string{ "MDL-WIDGET12": "one rule, two sites: a design-property VALUE the theme does not allow — an " + "off-list option, or a single value where the property takes a SET. Same two paths as " + "MDL-WIDGET11, and the same argument against splitting (ako/mxcli#511)", + "MDL-WIDGET07": "one rule, two sites: a widget property the builder silently drops on " + + "write. validate_widgets.go covers a property key no builder reads; " + + "validate_design_properties.go covers DesignProperties on a slot of its parent (a " + + "layoutgrid row, a row's column, a dataview footer), which the builder assembles " + + "without applying appearance. Someone suppressing MDL-WIDGET07 means both", "MDL059": "one rule, two sites: an annotation that parses and does nothing. " + "validate_flow_parameters.go covers one written on a PARAMETER, " + "validate_document_annotations.go one written before a CREATE. Someone " + diff --git a/mdl/executor/theme_reader.go b/mdl/executor/theme_reader.go index fb0accb882..d0ce2793e4 100644 --- a/mdl/executor/theme_reader.go +++ b/mdl/executor/theme_reader.go @@ -155,40 +155,67 @@ func (r *ThemeRegistry) GetPropertiesForWidget(widgetTypeKey string) []ThemeProp return result } -// mdlKeywordToDesignPropsKey maps MDL widget type keywords to the keys used in -// design-properties.json — for the NATIVE widgets only. +// mdlKeywordStorageType maps each MDL keyword that builds a NATIVE widget to the +// $Type the builder writes for it. The theme key is then read off that $Type +// (bsonTypeToDesignPropsKey), so an inline widget and a stored one resolve +// through the same table and cannot disagree about which group applies. // -// A pluggable widget is keyed in design-properties.json by its widget id, and -// which widget a keyword produces is decided elsewhere (keywordDispatchTable and -// the embedded widget definitions). Naming one here is how DATAGRID came to be -// validated against the wrong widget: see pluggableKeywordIDs. -var mdlKeywordToDesignPropsKey = map[string]string{ - "container": "DivContainer", - "customcontainer": "DivContainer", - "actionbutton": "Button", - "linkbutton": "Button", - "textbox": "TextBox", - "textarea": "TextArea", - "datepicker": "DatePicker", - "checkbox": "CheckBox", - "radiobuttons": "RadioButtons", - "dropdown": "DropDown", - "referenceselector": "ReferenceSelector", - "dataview": "DataView", - "listview": "ListView", - "layoutgrid": "LayoutGrid", - "dynamictext": "DynamicText", - "statictext": "Label", - // `label` writes Forms$Label (bsonTypeToDesignPropsKey below reads the same - // key). Unmapped, the validator skipped every label and the builder typed a - // Label's "Style" colour as an option — CE6085 at build time. - "label": "Label", - "staticimage": "StaticImageViewer", - "dynamicimage": "DynamicImageViewer", - "navigationlist": "NavigationList", - "snippetcall": "SnippetCall", - "header": "Header", - "footer": "Footer", +// It used to map keywords straight to keys — a second hand-written copy of the +// same concept — and it drifted: `groupbox`, `tabcontainer`, `navigationtree`, +// `menubar`, `simplemenubar`, `button`, `row` and `column` had no entry, so the +// keyword fell through as-is, no design-properties.json defines "groupbox", the +// validator skipped the widget, and the builder saw only the "Widget" base group +// — a custom colour on a group box's ColorPicker "Style" was written as an +// option and mxbuild refused it with CE6085. `radiobuttons` and `snippetcall` +// named keys mxbuild does not apply ("RadioButtons", "SnippetCall" — CE6083 when +// a theme declares them), and `header`/`footer` named groups for widgets the +// builder writes as a Forms$DivContainer. +// +// A keyword is a widget only where buildWidgetV3 builds it. `row`, `column` and +// `footer` are also SLOTS of a layoutgrid / row / dataview, where no widget of +// this $Type exists — validateDesignPropsSubtree tells the two apart. +// +// TestKeywordStorageTypesMatchBuilder builds every entry and compares the $Type; +// TestKeywordStorageTypesCoverBuilderDispatch holds it to buildWidgetV3's switch. +var mdlKeywordStorageType = map[string]string{ + "container": "Forms$DivContainer", + "customcontainer": "Forms$DivContainer", + // Top-level `row` / `column` build a Forms$DivContainer holding a one-row + // layout grid; their design properties land on that container. + "row": "Forms$DivContainer", + "column": "Forms$DivContainer", + "header": "Forms$DivContainer", + "footer": "Forms$DivContainer", + "controlbar": "Forms$DivContainer", + "template": "Forms$DivContainer", + "filter": "Forms$DivContainer", + + "actionbutton": "Forms$ActionButton", + "linkbutton": "Forms$ActionButton", + "button": "Forms$ActionButton", + "textbox": "Forms$TextBox", + "textarea": "Forms$TextArea", + "datepicker": "Forms$DatePicker", + "checkbox": "Forms$CheckBox", + "radiobuttons": "Forms$RadioButtonGroup", + "dropdown": "Forms$DropDown", + "dataview": "Forms$DataView", + "listview": "Forms$ListView", + "layoutgrid": "Forms$LayoutGrid", + "dynamictext": "Forms$DynamicText", + "label": "Forms$Label", + "title": "Forms$Title", + "staticimage": "Forms$StaticImageViewer", + "dynamicimage": "Forms$ImageViewer", + "navigationlist": "Forms$NavigationList", + "snippetcall": "Forms$SnippetCallWidget", + "tabcontainer": "Forms$TabControl", + "groupbox": "Forms$GroupBox", + "scrollcontainer": "Forms$ScrollContainer", + "navigationtree": "Forms$NavigationTree", + "menubar": "Forms$MenuBar", + "simplemenubar": "Forms$SimpleMenuBar", + "placeholder": "Forms$Placeholder", } // pluggableKeywordIDs maps an MDL keyword to the pluggable widget id it writes, @@ -236,62 +263,82 @@ var pluggableKeywordIDs = sync.OnceValue(func() map[string]string { // "CONTAINER") to the design-properties.json key (e.g., "DivContainer"). // // A keyword that writes a PLUGGABLE widget resolves to that widget's id, which -// is how design-properties.json keys them. Native keywords use the table above. -// An unrecognised type falls through as-is — a pluggable id written directly is -// already the right key. +// is how design-properties.json keys them. A native keyword resolves through the +// $Type it writes (mdlKeywordStorageType) to the key a stored widget of that +// type has. An unrecognised type falls through as-is — a pluggable id written +// directly is already the right key. func resolveDesignPropsKey(mdlKeyword string) string { lower := strings.ToLower(mdlKeyword) if id, ok := pluggableKeywordIDs()[lower]; ok { return id } - if key, ok := mdlKeywordToDesignPropsKey[lower]; ok { - return key + if storage, ok := mdlKeywordStorageType[lower]; ok { + if key, ok := bsonTypeToDesignPropsKey[storage]; ok { + return key + } } return mdlKeyword } -// bsonTypeToDesignPropsKey maps BSON $Type values to design-properties.json keys. -var bsonTypeToDesignPropsKey = map[string]string{ - "Forms$DivContainer": "DivContainer", - "Pages$DivContainer": "DivContainer", - "Forms$ActionButton": "Button", - "Pages$ActionButton": "Button", - "Forms$TextBox": "TextBox", - "Pages$TextBox": "TextBox", - "Forms$TextArea": "TextArea", - "Pages$TextArea": "TextArea", - "Forms$DatePicker": "DatePicker", - "Pages$DatePicker": "DatePicker", - "Forms$CheckBox": "CheckBox", - "Pages$CheckBox": "CheckBox", - "Forms$RadioButtons": "RadioButtons", - "Pages$RadioButtons": "RadioButtons", - "Forms$ReferenceSelector": "ReferenceSelector", - "Pages$ReferenceSelector": "ReferenceSelector", - "Forms$DropDown": "DropDown", - "Pages$DropDown": "DropDown", - "Forms$DataGrid": "DataGrid", - "Pages$DataGrid": "DataGrid", - "Forms$DataView": "DataView", - "Pages$DataView": "DataView", - "Forms$ListView": "ListView", - "Pages$ListView": "ListView", - "Forms$LayoutGrid": "LayoutGrid", - "Pages$LayoutGrid": "LayoutGrid", - "Forms$DynamicText": "DynamicText", - "Pages$DynamicText": "DynamicText", - "Forms$Label": "Label", - "Pages$Label": "Label", - "Forms$StaticImageViewer": "StaticImageViewer", - "Pages$StaticImageViewer": "StaticImageViewer", - "Forms$DynamicImageViewer": "DynamicImageViewer", - "Pages$DynamicImageViewer": "DynamicImageViewer", - "Forms$Gallery": "Gallery", - "Pages$Gallery": "Gallery", - "Forms$NavigationList": "NavigationList", - "Pages$NavigationList": "NavigationList", +// storageTypeThemeKeys maps a stored widget $Type, without its "Forms$" / +// "Pages$" prefix, to the design-properties.json group Studio Pro applies to it. +// +// A theme key is a Mendix CLASS name — the qualified name, NOT the storage name. +// Measured with probe groups added to a copy of PedApp's theme (Mendix 11.13.0): +// mxbuild accepts a "TabContainer", "RadioButtonGroup" or "SnippetCallWidget" +// design property on Forms$TabControl / Forms$RadioButtonGroup / +// Forms$SnippetCallWidget, and refuses "TabControl", "RadioButtons" and +// "SnippetCall" with CE6083 "not supported by your theme". Studio Pro also +// applies the groups of ANCESTOR classes — "Widget" for every widget, and +// "Button" (Atlas's key) as well as "ActionButton" for an action button. This +// table names the one group per type that Atlas declares; GetPropertiesForWidget +// adds "Widget". +// +// TestDesignPropsKeysAreMetamodelClassNames holds every entry to the generated +// metamodel's class for its storage name, so a storage/qualified mix-up like the +// ones above fails a test instead of a build. +var storageTypeThemeKeys = map[string]string{ + "DivContainer": "DivContainer", + "ActionButton": "Button", + "TextBox": "TextBox", + "TextArea": "TextArea", + "DatePicker": "DatePicker", + "CheckBox": "CheckBox", + "RadioButtonGroup": "RadioButtonGroup", + "ReferenceSelector": "ReferenceSelector", + "DropDown": "DropDown", + "DataGrid": "DataGrid", + "DataView": "DataView", + "ListView": "ListView", + "LayoutGrid": "LayoutGrid", + "DynamicText": "DynamicText", + "Label": "Label", + "Title": "Title", + "StaticImageViewer": "StaticImageViewer", + "ImageViewer": "DynamicImageViewer", // DynamicImageViewer's storage name + "DynamicImageViewer": "DynamicImageViewer", + "NavigationList": "NavigationList", + "TabControl": "TabContainer", // TabContainer's storage name + "GroupBox": "GroupBox", + "ScrollContainer": "ScrollContainer", + "NavigationTree": "NavigationTree", + "MenuBar": "MenuBar", + "SimpleMenuBar": "SimpleMenuBar", + "Placeholder": "Placeholder", + "SnippetCallWidget": "SnippetCallWidget", } +// bsonTypeToDesignPropsKey maps BSON $Type values to design-properties.json keys: +// storageTypeThemeKeys under both prefixes. +var bsonTypeToDesignPropsKey = func() map[string]string { + out := make(map[string]string, 2*len(storageTypeThemeKeys)) + for storage, key := range storageTypeThemeKeys { + out["Forms$"+storage] = key + out["Pages$"+storage] = key + } + return out +}() + // widgetTypeDisplayName maps BSON $Type to a short display name for output. var widgetTypeDisplayName = map[string]string{ "Forms$DivContainer": "Container", diff --git a/mdl/executor/validate_alter_styling.go b/mdl/executor/validate_alter_styling.go index 919ddeea1d..22c31b5725 100644 --- a/mdl/executor/validate_alter_styling.go +++ b/mdl/executor/validate_alter_styling.go @@ -44,8 +44,8 @@ import ( // check that cannot see what it is judging. // // Deliberately NOT done: mapping the stored $Type to a registry key to make this -// precise. That would be a third resolver for one concept — mdlKeywordToDesignPropsKey -// maps MDL keyword → key, and an unused bsonTypeToDesignPropsKey maps $Type → +// precise. That would be a third resolver for one concept — mdlKeywordStorageType +// maps MDL keyword → $Type, and an unused bsonTypeToDesignPropsKey maps $Type → // key — and duplicate resolvers drifting apart is the failure this area keeps // producing (mendixlabs/mxcli#1069's buildPropKeyMap was two copies of one // derivation, and one copy was the bug). The precise variant needs a widget-type diff --git a/mdl/executor/validate_design_properties.go b/mdl/executor/validate_design_properties.go index 526dc330d9..63fd798b1b 100644 --- a/mdl/executor/validate_design_properties.go +++ b/mdl/executor/validate_design_properties.go @@ -81,19 +81,94 @@ func ValidateDesignPropertiesForStatement(stmt ast.Statement, reg *ThemeRegistry } func validateDesignPropsTree(widgets []*ast.WidgetV3, reg *ThemeRegistry, locationPrefix string) []linter.Violation { + return validateDesignPropsSubtree(nil, widgets, reg, locationPrefix) +} + +func validateDesignPropsSubtree(parent *ast.WidgetV3, widgets []*ast.WidgetV3, reg *ThemeRegistry, locationPrefix string) []linter.Violation { var out []linter.Violation for _, w := range widgets { if w == nil { continue } - out = append(out, validateWidgetDesignProps(w, reg, locationPrefix)...) + switch designPropsSlotOf(parent, w) { + case slotDropped: + out = append(out, droppedSlotDesignProps(parent, w, locationPrefix)...) + case slotOfPluggable: + // Built by the pluggable engine from the parent's object lists; the + // keyword names no native widget here, so there is nothing to resolve. + default: + out = append(out, validateWidgetDesignProps(w, reg, locationPrefix)...) + } if len(w.Children) > 0 { - out = append(out, validateDesignPropsTree(w.Children, reg, locationPrefix)...) + out = append(out, validateDesignPropsSubtree(w, w.Children, reg, locationPrefix)...) } } return out } +type designPropsSlot int + +const ( + notASlot designPropsSlot = iota + // slotDropped: a structural part of its native parent that the builder + // assembles itself, never through buildWidgetV3, so applyWidgetAppearance + // never sees its design properties. + slotDropped + // slotOfPluggable: a keyword that is a native widget elsewhere, used as a + // child of a pluggable widget — one of its object-list entries or slots. + slotOfPluggable +) + +// slotKeywords are the keywords that build a Forms$DivContainer on their own but +// name a part of their parent when nested in one: a layout grid's row, a row's +// column, a dataview's footer, a data grid's column or a gallery's template. +var slotKeywords = map[string]bool{ + "row": true, "column": true, "header": true, "footer": true, + "controlbar": true, "template": true, "filter": true, +} + +// designPropsSlotOf reports whether child is a slot of parent rather than a +// widget. mdlKeywordStorageType maps `row` / `column` / `footer` to the +// Forms$DivContainer they build at the top level; that is only true where +// buildWidgetV3 builds them. +func designPropsSlotOf(parent, child *ast.WidgetV3) designPropsSlot { + if parent == nil || child == nil { + return notASlot + } + p, c := strings.ToLower(parent.Type), strings.ToLower(child.Type) + switch { + case p == "layoutgrid" && c == "row", // buildLayoutGridRowV3 + p == "row" && c == "column", // buildLayoutGridColumnV3 + p == "dataview" && c == "footer": // children moved into FooterWidgets + return slotDropped + } + if !slotKeywords[c] { + return notASlot + } + if _, native := mdlKeywordStorageType[p]; !native { + return slotOfPluggable + } + return notASlot +} + +// droppedSlotDesignProps reports design properties written on a slot the builder +// drops them from. Silence here read as approval of a value that never reaches +// the model — the "check passes, exec succeeds, the value is gone" shape. +func droppedSlotDesignProps(parent, w *ast.WidgetV3, locationPrefix string) []linter.Violation { + if len(w.GetDesignProperties()) == 0 { + return nil + } + return []linter.Violation{{ + RuleID: "MDL-WIDGET07", + Severity: linter.SeverityWarning, + Message: fmt.Sprintf("%s: %s %q inside %s %q sets DesignProperties, but it is part of its parent, "+ + "not a widget — mxcli does not write design properties there, so they are silently dropped", + locationPrefix, strings.ToLower(w.Type), w.Name, strings.ToLower(parent.Type), parent.Name), + Location: linter.Location{DocumentType: "page", DocumentName: locationPrefix}, + Suggestion: fmt.Sprintf("Put them on a container inside the %s instead.", strings.ToLower(w.Type)), + }} +} + func validateWidgetDesignProps(w *ast.WidgetV3, reg *ThemeRegistry, locationPrefix string) []linter.Violation { astProps := w.GetDesignProperties() if len(astProps) == 0 { diff --git a/mdl/executor/validate_design_properties_test.go b/mdl/executor/validate_design_properties_test.go index 28b211f92c..1cfaa72be6 100644 --- a/mdl/executor/validate_design_properties_test.go +++ b/mdl/executor/validate_design_properties_test.go @@ -201,13 +201,15 @@ func TestResolveDesignPropsKey_PluggableKeywordsUseTheirWidgetID(t *testing.T) { // ever writes. func TestResolveDesignPropsKey_NativeKeywordsUnchanged(t *testing.T) { for keyword, want := range map[string]string{ - "container": "DivContainer", - "actionbutton": "Button", - "dataview": "DataView", - "listview": "ListView", - "layoutgrid": "LayoutGrid", - "referenceselector": "ReferenceSelector", - "staticimage": "StaticImageViewer", + "container": "DivContainer", + "actionbutton": "Button", + "dataview": "DataView", + "listview": "ListView", + "layoutgrid": "LayoutGrid", + "staticimage": "StaticImageViewer", + // `referenceselector` is gone from this list: it parses, but no builder + // writes it (exec refuses it as an unsupported widget type), so it has no + // $Type to resolve through. A stored Forms$ReferenceSelector still maps. } { if got := resolveDesignPropsKey(keyword); got != want { t.Errorf("resolveDesignPropsKey(%q) = %q, want %q", keyword, got, want) @@ -229,7 +231,7 @@ func TestResolveDesignPropsKey_UnknownFallsThrough(t *testing.T) { // and the next person to edit it changes nothing. func TestDesignPropsKeyTablesDoNotOverlap(t *testing.T) { for keyword := range pluggableKeywordIDs() { - if native, ok := mdlKeywordToDesignPropsKey[keyword]; ok { + if native, ok := mdlKeywordStorageType[keyword]; ok { t.Errorf("%q is in both tables (native %q and a pluggable id). "+ "The native entry is dead — remove it.", keyword, native) } diff --git a/mdl/executor/widget_rule_ids_test.go b/mdl/executor/widget_rule_ids_test.go index 5b64d61ca2..f90fa7e468 100644 --- a/mdl/executor/widget_rule_ids_test.go +++ b/mdl/executor/widget_rule_ids_test.go @@ -38,6 +38,11 @@ var widgetRuleIDsRaisedFromSeveralFiles = map[string][]string{ // off-list option, or a single value where the property takes a set // (ako/mxcli#511). Same two paths. "MDL-WIDGET12": {"validate_alter_styling.go", "validate_design_properties.go"}, + // One rule about a property the builder silently drops on write — a key no + // builder reads, or DesignProperties on a slot of the parent (a layoutgrid + // row, a row's column, a dataview footer) that never reaches + // applyWidgetAppearance. + "MDL-WIDGET07": {"validate_design_properties.go", "validate_widgets.go"}, } // Matched as a QUOTED literal rather than after `RuleID:`, because an id is From 7c8619bbdefb2e59844d68d6ff497c8a2c459f92 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 15:13:41 +0000 Subject: [PATCH 46/47] fix(lint): name nanoflows and rules correctly in shipped Starlark rules microflows() yields microflows, nanoflows and rules, which share one catalog table. Six shipped rules (CONV009, CONV010, QUAL001, QUAL003, QUAL004, CUSTOM002) hardcoded document_type="Microflow" and a "Microflow '...'" message, so a nanoflow with 30 activities was reported as "Microflow 'X' has 30 activities". The wrong type also reached the documentType field of the JSON and report output. The Go rules were fixed earlier with Microflow.DocumentNoun(), but Starlark rules could not call it. Expose it on the microflow struct as document_noun / document_noun_title, use it in the six rules, and document both fields in the write-lint-rules skill. Output for microflows is unchanged. QUAL004 needed only the label: rule calls from decisions are already recorded as 'call' refs, so called rules are not reported as orphaned. The test runs every shipped rule that walks microflows() over a fixture with one flow per flavour. It requires a finding on each flavour, so the assertions cannot pass vacuously, and a guard fails if a new such rule is not added to the list. Follow-up to mendixlabs/mxcli#1178 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_011AvRv9GAQJbrHgrgMmfsBM --- .../conv009_max_microflow_objects.star | 8 +- .../conv010_act_microflow_content.star | 20 +-- .claude/lint-rules/example_microflow.star | 7 +- .claude/lint-rules/long_microflows.star | 8 +- .claude/lint-rules/mccabe_complexity.star | 10 +- .claude/lint-rules/orphaned_elements.star | 4 +- .../skills/fix-issue/findings/cmd-mxcli.jsonl | 1 + .../skills/mendix/write-lint-rules/SKILL.md | 4 +- mdl/linter/starlark.go | 5 + mdl/linter/starlark_flow_noun_test.go | 156 ++++++++++++++++++ 10 files changed, 198 insertions(+), 25 deletions(-) create mode 100644 mdl/linter/starlark_flow_noun_test.go diff --git a/.claude/lint-rules/conv009_max_microflow_objects.star b/.claude/lint-rules/conv009_max_microflow_objects.star index 0afc2baa1b..44ee47318f 100644 --- a/.claude/lint-rules/conv009_max_microflow_objects.star +++ b/.claude/lint-rules/conv009_max_microflow_objects.star @@ -18,15 +18,15 @@ def check(): for mf in microflows(): if mf.activity_count > MAX_ACTIVITIES: violations.append(violation( - message="Microflow '{}' has {} activities (convention max: {}). Split into sub-microflows.".format( - mf.name, mf.activity_count, MAX_ACTIVITIES + message="{} '{}' has {} activities (convention max: {}). Split into sub-{}s.".format( + mf.document_noun_title, mf.name, mf.activity_count, MAX_ACTIVITIES, mf.document_noun ), location=location( module=mf.module_name, - document_type="Microflow", + document_type=mf.document_noun_title, document_name=mf.qualified_name, ), - suggestion="Extract logical sections into SUB_ microflows to keep each under {} activities".format(MAX_ACTIVITIES), + suggestion="Extract logical sections into SUB_ {}s to keep each under {} activities".format(mf.document_noun, MAX_ACTIVITIES), )) return violations diff --git a/.claude/lint-rules/conv010_act_microflow_content.star b/.claude/lint-rules/conv010_act_microflow_content.star index c248203290..c54cbc464b 100644 --- a/.claude/lint-rules/conv010_act_microflow_content.star +++ b/.claude/lint-rules/conv010_act_microflow_content.star @@ -92,31 +92,31 @@ def check(): continue violations.append(violation( - message="ACT_ microflow '{}' contains '{}' action. Delegate business logic to a SUB_ microflow.".format( - mf.name, act.action_type + message="ACT_ {} '{}' contains '{}' action. Delegate business logic to a SUB_ {}.".format( + mf.document_noun, mf.name, act.action_type, mf.document_noun ), location=location( module=mf.module_name, - document_type="Microflow", + document_type=mf.document_noun_title, document_name=mf.qualified_name, ), - suggestion="Move the '{}' action to a SUB_ microflow and call it from '{}'".format( - act.action_type, mf.name + suggestion="Move the '{}' action to a SUB_ {} and call it from '{}'".format( + act.action_type, mf.document_noun, mf.name ), )) elif act.activity_type not in ALLOWED_ACTIVITY_TYPES: # Any other non-allowed activity type violations.append(violation( - message="ACT_ microflow '{}' contains '{}' activity. Delegate to a SUB_ microflow.".format( - mf.name, act.activity_type + message="ACT_ {} '{}' contains '{}' activity. Delegate to a SUB_ {}.".format( + mf.document_noun, mf.name, act.activity_type, mf.document_noun ), location=location( module=mf.module_name, - document_type="Microflow", + document_type=mf.document_noun_title, document_name=mf.qualified_name, ), - suggestion="Move the '{}' to a SUB_ microflow called from '{}'".format( - act.activity_type, mf.name + suggestion="Move the '{}' to a SUB_ {} called from '{}'".format( + act.activity_type, mf.document_noun, mf.name ), )) diff --git a/.claude/lint-rules/example_microflow.star b/.claude/lint-rules/example_microflow.star index 1a6d4d9a3f..3f7dc4badf 100644 --- a/.claude/lint-rules/example_microflow.star +++ b/.claude/lint-rules/example_microflow.star @@ -19,6 +19,8 @@ # .return_type - Return type # .parameter_count - Number of parameters # .activity_count - Number of activities +# .document_noun - "microflow", "nanoflow" or "rule" (microflows() yields all three) +# .document_noun_title - the same, capitalised: use it for document_type= RULE_ID = "CUSTOM002" RULE_NAME = "Microflow Prefix Convention" @@ -48,11 +50,12 @@ def check(): if not has_valid_prefix: loc = location( module=mf.module_name, - document_type="Microflow", + document_type=mf.document_noun_title, document_name=mf.qualified_name ) v = violation( - message="Microflow '{}' should start with a standard prefix ({})".format( + message="{} '{}' should start with a standard prefix ({})".format( + mf.document_noun_title, name, ", ".join(VALID_PREFIXES) ), diff --git a/.claude/lint-rules/long_microflows.star b/.claude/lint-rules/long_microflows.star index 6a9db55808..b19813f094 100644 --- a/.claude/lint-rules/long_microflows.star +++ b/.claude/lint-rules/long_microflows.star @@ -31,14 +31,16 @@ def check(): if mf.activity_count > MAX_ACTIVITIES: loc = location( module=mf.module_name, - document_type="Microflow", + document_type=mf.document_noun_title, document_name=mf.qualified_name ) v = violation( - message="Microflow '{}' has {} activities (max: {}). Consider splitting into smaller microflows.".format( + message="{} '{}' has {} activities (max: {}). Consider splitting into smaller {}s.".format( + mf.document_noun_title, mf.name, mf.activity_count, - MAX_ACTIVITIES + MAX_ACTIVITIES, + mf.document_noun ), location=loc, suggestion="Extract logical sections into SUB_ microflows. This improves readability, testability, and reusability." diff --git a/.claude/lint-rules/mccabe_complexity.star b/.claude/lint-rules/mccabe_complexity.star index 208bf8b65c..31283c20f6 100644 --- a/.claude/lint-rules/mccabe_complexity.star +++ b/.claude/lint-rules/mccabe_complexity.star @@ -24,6 +24,8 @@ # .parameter_count - Number of parameters # .activity_count - Number of activities # .complexity - McCabe cyclomatic complexity +# .document_noun - "microflow", "nanoflow" or "rule" (microflows() yields all three) +# .document_noun_title - the same, capitalised: use it for document_type= RULE_ID = "QUAL001" RULE_NAME = "McCabe Complexity" @@ -45,14 +47,16 @@ def check(): if mf.complexity > MAX_COMPLEXITY: loc = location( module=mf.module_name, - document_type="Microflow", + document_type=mf.document_noun_title, document_name=mf.qualified_name ) v = violation( - message="Microflow '{}' has complexity {} (max: {}). Consider splitting into smaller microflows.".format( + message="{} '{}' has complexity {} (max: {}). Consider splitting into smaller {}s.".format( + mf.document_noun_title, mf.name, mf.complexity, - MAX_COMPLEXITY + MAX_COMPLEXITY, + mf.document_noun ), location=loc, suggestion="Break down complex logic into sub-microflows. Extract decision branches into separate SUB_ microflows." diff --git a/.claude/lint-rules/orphaned_elements.star b/.claude/lint-rules/orphaned_elements.star index 39c04c01e8..5fe5189a6b 100644 --- a/.claude/lint-rules/orphaned_elements.star +++ b/.claude/lint-rules/orphaned_elements.star @@ -89,11 +89,11 @@ def check(): if not has_callers: loc = location( module=mf.module_name, - document_type="Microflow", + document_type=mf.document_noun_title, document_name=mf.qualified_name ) v = violation( - message="Microflow '{}' is not called from anywhere.".format(mf.name), + message="{} '{}' is not called from anywhere.".format(mf.document_noun_title, mf.name), location=loc, suggestion="Remove if unused, or rename with ACT_/SCH_ prefix if it's an entry point." ) diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl index 2350194b7c..74ed2762ef 100644 --- a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl +++ b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl @@ -126,3 +126,4 @@ {"area": "cmd/mxcli/diag", "date": "2026-09-23", "symptom": "`mxcli diag loop-report` showed all 5 `test` runs as 'did not close' although every test passed, and inflated the `-c` (111) and `exec` (180) counts in the same log. Reported as 'the command apparently skips the summary record on success too' \u2014 which is not what happens: `test` returns normally, PersistentPostRun fires, and the session_end IS written.", "cause": "mxcli runs mxcli. Measured from a real `mxcli test` with MXCLI_LOG_DIR pointed at a scratch dir: one parent session_start (pid 1071) followed by THREE child session_starts \u2014 `-c DESCRIBE SETTINGS`, `-c SHOW MODULES`, and an `exec` of the generated runner \u2014 before a single test executes. `new`, `eval`, `tui` and the LSP self-spawn the same way (six os.Executable() sites). buildInvocations segmented on 'next session_end OR next session_start, whichever comes first', so a child's start closed the parent's invocation and the parent's own end landed on whatever was open by then. session_end carried no pid, so pairing by process was impossible.", "file": "`mdl/diaglog/diaglog.go` (pid on session_end; parentPIDEnv marker set once in Init and inherited by every child), `cmd/mxcli/diag_loop_report.go` (buildInvocations pairs by pid with the positional rule as fallback; spawned runs excluded from the table and wall time, counted on their own line), tests `cmd/mxcli/diag_loop_report_test.go`", "insight": "The segmentation rule documented itself as exact 'for sequential invocations, which is what an agent loop produces' \u2014 and the thing that breaks that assumption is the tool itself, not concurrency by the user. When a tool can invoke itself, EVERY per-process measurement over it needs a parent link, not just a pid: a pid alone fixes the pairing but still counts three phantom agent calls per test run. The marker belongs on the ENVIRONMENT, not at each spawn site: exec.Command inherits the parent's environment (explicitly via os.Environ(), implicitly when Cmd.Env is nil), so one os.Setenv in Init covers all six self-spawn sites and any added later \u2014 six edits that would each have to be remembered become zero. Second-order trap: spawned runs must be excluded from WALL TIME too, not just the count, because a child's seconds are already inside its parent's; the test asserts 10s for a parent with three 1s children, and the reverted code says 3. Prove-by-revert done on the measured record shape: the positional rule gives Invocations=4 (want 1), Unclosed=1 for a parent whose tests all passed, Wall=3 (want 10).", "refs": ["ako/mxcli#617", "ako/mxcli#629"]} {"area": "cmd/mxcli/syntax", "date": "2026-09-23", "symptom": "`mxcli syntax page datasource` documented `DataSource: MICROFLOW Module.MF($P)`. That form is a parse error: `dataview dv (datasource: microflow M.DS_X($State))` gives 'line 2:15 no viable alternative at input datasource'. Only the NAMED form `M.DS_X(State: $State)` parses. Hit in a real build, diagnosed from the error rather than the doc.", "cause": "The syntax entry was written from the intended shape rather than from something that had been run through the parser. Nothing checks it: the Syntax and Example fields are free text.", "file": "`cmd/mxcli/syntax/features_page.go` (page.datasource entry now shows `MICROFLOW Module.MF(Param: $P)` and states that the positional form is a parse error)", "insight": "CLAUDE.md deliberately points at `mxcli syntax` instead of restating syntax, so that it cannot go stale \u2014 which makes a wrong entry there worse than a wrong entry in prose, because it is the thing consulted INSTEAD of checking. The cost lands in the agent loop: read it, write it, fail to parse, diagnose, retry. Measured before reaching for the systemic guard: 42 of 164 syntax examples fail `mxcli check` today, but the large majority are fragments by design (a microflow body like `IF \u2026`, a widget snippet like `DATAGRID \u2026`, an OQL fragment) and are legitimately not standalone top-level MDL \u2014 so a blanket 'every example must parse' test would be mostly noise, and making it useful needs a way to mark which examples are standalone. Measuring that first is what stopped a plausible-sounding guard from being built wrong.", "refs": ["ako/mxcli#630"]} {"area": "cmd/mxcli/test", "date": "2026-09-23", "symptom": "Windows: `mxcli test tests/ -p MyApp.mpr --local` fails with `local runtime: starting mxbuild serve: mxbuild --serve did not become ready` after caching a Linux ELF in %USERPROFILE%\\.mxcli\\mxbuild, and there is \"no flag, environment variable, or mechanism to redirect mxcli to the Windows mxbuild.exe already present in the Studio Pro installation\"", "cause": "Two layers. The platform part (downloading/exec'ing the Linux binary, serving from the cache instead of the resolved binary) was already fixed by #916 and #1122, both after the reporter's v0.21.0. What remained on main: `test` never registered `--mxbuild-path` — `run` gained it in #1125, but `test --local` boots through the same `ResolveMxBuildForLocal` and prints the same 'pass --mxbuild-path' guidance while answering `unknown flag`. `RunOptions` had no field and `localAppOptions` never set `LocalAppOptions.MxBuildPath`, though StartLocalApp honoured it. No env override existed anywhere", "file": "`cmd/mxcli/main.go` + `cmd_test_run.go` (flag), `testrunner/runner.go` + `localapp_options.go` (plumbing), `docker/mxbuild_platform.go` (`MxBuildPathEnv`, read in `resolveMxBuildForLocalOn` after the flag)", "insight": "**The guard for #1125 asserted its invariant against one command** — `TestErrorGuidanceNamesAFlagThatExists` checked only `runCmd`, while the guidance it polices is emitted by a resolver two commands share. When a test pins 'the advertised flag exists', enumerate the callers of the code that ADVERTISES it, not the command the report named; it now iterates `run` and `test`. **Before fixing a platform report, date it against the fixes**: the reporter's first two suggestions ('download platform-correct binary', 'auto-discover Studio Pro') were already on main, and re-implementing them would have been churn — only the override was missing. Put the env var in the resolver, not the CLI, so `run --local` and `test --local` both get it from one line; a flag-level env read would have been one more per-command copy to drift. Control: the env test runs as goos=windows with an unmatched version, so without the override it fails fast with the 'Linux binary cannot run natively on windows' refusal instead of hitting the CDN; removing only the `MxBuildPath:` line in localAppOptions fails the plumbing test for both runners. **Unverified**: no Windows host; code-level with OS-injected tests", "refs": ["mendixlabs/mxcli#1086", "mendixlabs/mxcli#1125", "mendixlabs/mxcli#916", "mendixlabs/mxcli#1122"]} +{"area": "cmd/mxcli", "date": "2026-09-25", "symptom": "Shipped Starlark rules CONV009, CONV010, QUAL001, QUAL003, QUAL004 and CUSTOM002 report nanoflows and rules as microflows: \"Microflow 'NF_Foo' has 30 activities\", and documentType \"Microflow\" in the JSON and report output.", "cause": "microflows() yields all three flow flavours (one catalog table, MicroflowType MICROFLOW/NANOFLOW/RULE), and each rule hardcoded document_type=\"Microflow\" and a \"Microflow '...'\" message. The Go rules had already been fixed with Microflow.DocumentNoun() after MPR002 called a rule a microflow, but that method was never exposed to Starlark, so every Starlark rule had to re-derive it and none did.", "file": "mdl/linter/starlark.go, .claude/lint-rules/{conv009_max_microflow_objects,conv010_act_microflow_content,example_microflow,long_microflows,mccabe_complexity,orphaned_elements}.star, mdl/linter/starlark_flow_noun_test.go", "insight": "When a Go-side fix lives in a method, check whether the Starlark projection can reach it; a fix Starlark cannot call is a fix for half the rules. Exposed as document_noun / document_noun_title on the microflow struct, and the test runs every shipped rule over a fixture with one flow per flavour, requiring a finding on each (else vacuous) plus a guard that any new `for x in microflows()` rule joins the list. Skip the wrong turn of excluding nanoflows/rules from QUAL004: rule calls from decisions ARE emitted as 'call' refs (builder_references.go collectRuleCalls), so only the label was wrong.", "refs": "mendixlabs/mxcli#1178"} diff --git a/.claude/skills/mendix/write-lint-rules/SKILL.md b/.claude/skills/mendix/write-lint-rules/SKILL.md index 3241920a64..d0e25fe53a 100644 --- a/.claude/skills/mendix/write-lint-rules/SKILL.md +++ b/.claude/skills/mendix/write-lint-rules/SKILL.md @@ -50,7 +50,7 @@ silently return empty results (issue #721). | Function | Returns | Description | |----------|---------|-------------| | `entities()` | list of entity | All non-system entities | -| `microflows()` | list of microflow | All non-system microflows | +| `microflows()` | list of microflow | All non-system microflows, nanoflows **and rules** — they share one catalog table. Name the document with `document_noun_title`, never a hardcoded `"Microflow"` | | `pages()` | list of page | All non-system pages | | `enumerations()` | list of enumeration | All non-system enumerations | | `constants()` | list of constant | All non-system constants | @@ -179,6 +179,8 @@ def check(): | `parameter_count` | int | Number of parameters | | `activity_count` | int | Number of activities | | `complexity` | int | McCabe cyclomatic complexity | +| `document_noun` | string | `"microflow"`, `"nanoflow"` or `"rule"` — for mid-sentence use in a message | +| `document_noun_title` | string | `"Microflow"`, `"Nanoflow"` or `"Rule"` — for `document_type=` and a message that opens with it | ### page | Property | Type | Example | diff --git a/mdl/linter/starlark.go b/mdl/linter/starlark.go index 84d84be701..a854bfa0c0 100644 --- a/mdl/linter/starlark.go +++ b/mdl/linter/starlark.go @@ -876,6 +876,11 @@ func microflowToStarlark(mf Microflow) starlark.Value { "parameter_count": starlark.MakeInt(mf.ParameterCount), "activity_count": starlark.MakeInt(mf.ActivityCount), "complexity": starlark.MakeInt(mf.Complexity), + // microflows() yields all three flow flavours, so a rule naming the + // document in a message or a location must not hardcode "Microflow". + // Title case matches the document_type spelling Starlark rules use. + "document_noun": starlark.String(mf.DocumentNoun()), + "document_noun_title": starlark.String(mf.DocumentNounTitle()), }) } diff --git a/mdl/linter/starlark_flow_noun_test.go b/mdl/linter/starlark_flow_noun_test.go new file mode 100644 index 0000000000..9cd8b1de45 --- /dev/null +++ b/mdl/linter/starlark_flow_noun_test.go @@ -0,0 +1,156 @@ +// SPDX-License-Identifier: Apache-2.0 + +package linter_test + +import ( + "database/sql" + "os" + "path/filepath" + "regexp" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/mdl/catalog" + "github.com/mendixlabs/mxcli/mdl/linter" + _ "modernc.org/sqlite" +) + +// microflows() yields microflows, nanoflows and rules, because all three share +// one catalog table. A shipped rule that loops over it and hardcodes +// document_type="Microflow" and "Microflow '…'" reports a nanoflow or a rule +// under the wrong doctype -- in the message and in the documentType field of +// the JSON and report output. The Go rules fixed this with +// Microflow.DocumentNoun(); these tests hold the Starlark rules to the same. + +// shippedFlowRules are the shipped rules that walk every flow flavour. A rule +// that filters to microflow_type == "MICROFLOW" never reports a nanoflow and is +// not listed; TestEveryFlowWalkingRuleIsCovered keeps this list complete. +const flowRulesDir = "../../.claude/lint-rules" + +var shippedFlowRules = []string{ + "conv009_max_microflow_objects.star", + "conv010_act_microflow_content.star", + "example_microflow.star", + "long_microflows.star", + "mccabe_complexity.star", + "orphaned_elements.star", +} + +func TestShippedFlowRulesNameNanoflowsAndRulesCorrectly(t *testing.T) { + db := flowKindsFixtureDB(t) + wantType := map[string]string{"MF": "Microflow", "NF": "Nanoflow", "RU": "Rule"} + + for _, name := range shippedFlowRules { + t.Run(name, func(t *testing.T) { + r, err := linter.LoadStarlarkRule(filepath.Join(flowRulesDir, name)) + if err != nil { + t.Fatalf("LoadStarlarkRule: %v", err) + } + seen := map[string]bool{} + for _, v := range r.Check(linter.NewLintContextFromDB(db)) { + doc := v.Location.DocumentName + kind := doc[strings.LastIndex(doc, "_")+1:] + want, ok := wantType[kind] + if !ok { + continue + } + seen[kind] = true + if v.Location.DocumentType != want { + t.Errorf("%s: document_type %q, want %q", doc, v.Location.DocumentType, want) + } + if kind != "MF" && strings.Contains(strings.ToLower(v.Message), "microflow '") { + t.Errorf("%s: message calls a %s a microflow: %q", doc, strings.ToLower(want), v.Message) + } + } + // Without a finding on each flavour the assertions above are vacuous. + for kind := range wantType { + if !seen[kind] { + t.Errorf("no finding on the %s fixture flow -- the fixture no longer triggers this rule", kind) + } + } + }) + } +} + +// A new shipped rule that walks microflows() without filtering to MICROFLOW +// must join shippedFlowRules, or it can hardcode "Microflow" unnoticed. +func TestEveryFlowWalkingRuleIsCovered(t *testing.T) { + loop := regexp.MustCompile(`for \w+ in microflows\(\)`) + paths, err := filepath.Glob(filepath.Join(flowRulesDir, "*.star")) + if err != nil || len(paths) == 0 { + t.Fatalf("no shipped rules under %s: %v", flowRulesDir, err) + } + for _, path := range paths { + data, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + src := string(data) + if !loop.MatchString(src) || strings.Contains(src, `microflow_type != "MICROFLOW"`) { + continue + } + name := filepath.Base(path) + found := false + for _, n := range shippedFlowRules { + found = found || n == name + } + if !found { + t.Errorf("%s walks microflows() (which yields nanoflows and rules too) but is not in shippedFlowRules", name) + } + } +} + +// Two flows per flavour, each shaped to trip every rule in shippedFlowRules: +// Big_ has no naming prefix, and ACT_ contains a forbidden action; both +// are large, complex and unreferenced. +func flowKindsFixtureDB(t *testing.T) catalog.CatalogDB { + t.Helper() + db, err := sql.Open("sqlite", ":memory:") + if err != nil { + t.Fatal(err) + } + stmts := []string{ + `CREATE TABLE modules (Id TEXT, Name TEXT, Source TEXT)`, + `INSERT INTO modules VALUES ('m1', 'Sales', '')`, + `CREATE TABLE microflows ( + Id TEXT, Name TEXT, QualifiedName TEXT, ModuleName TEXT, Folder TEXT, + MicroflowType TEXT, Description TEXT, ReturnType TEXT, + ParameterCount INTEGER, ActivityCount INTEGER, Complexity INTEGER)`, + `INSERT INTO microflows VALUES + ('f1','Big_MF','Sales.Big_MF','Sales','','MICROFLOW','','',0,100,50), + ('f2','Big_NF','Sales.Big_NF','Sales','','NANOFLOW','','',0,100,50), + ('f3','Big_RU','Sales.Big_RU','Sales','','RULE','','',0,100,50), + ('f4','ACT_MF','Sales.ACT_MF','Sales','','MICROFLOW','','',0,100,50), + ('f5','ACT_NF','Sales.ACT_NF','Sales','','NANOFLOW','','',0,100,50), + ('f6','ACT_RU','Sales.ACT_RU','Sales','','RULE','','',0,100,50)`, + `CREATE TABLE activities ( + Id TEXT, Name TEXT, Caption TEXT, ActivityType TEXT, ActionType TEXT, + MicroflowId TEXT, MicroflowQualifiedName TEXT, ModuleName TEXT, EntityRef TEXT, + ServiceRef TEXT, ActionRef TEXT, UseRequestTimeout INTEGER, TimeoutExpression TEXT, + Sequence INTEGER)`, + `INSERT INTO activities VALUES + ('a1','','','ActionActivity','ChangeObjectAction','f4','Sales.ACT_MF','Sales','','','',0,'',1), + ('a2','','','ActionActivity','ChangeObjectAction','f5','Sales.ACT_NF','Sales','','','',0,'',1), + ('a3','','','ActionActivity','ChangeObjectAction','f6','Sales.ACT_RU','Sales','','','',0,'',1)`, + // One unrelated row, so the refs table reads as populated. + `CREATE TABLE refs ( + SourceType TEXT, SourceId TEXT, SourceName TEXT, TargetType TEXT, + TargetId TEXT, TargetName TEXT, RefKind TEXT, ModuleName TEXT)`, + `INSERT INTO refs VALUES ('PAGE','p1','Sales.P','ENTITY','e1','Sales.E','parameter','Sales')`, + `CREATE TABLE pages (Id TEXT, Name TEXT, QualifiedName TEXT, ModuleName TEXT, Folder TEXT, + Title TEXT, URL TEXT, Description TEXT, WidgetCount INTEGER)`, + `CREATE TABLE entities ( + Id TEXT, Name TEXT, QualifiedName TEXT, ModuleName TEXT, Folder TEXT, + EntityType TEXT, Description TEXT, Generalization TEXT, + AttributeCount INTEGER, AccessRuleCount INTEGER, ValidationRuleCount INTEGER, + HasEventHandlers INTEGER, IsExternal INTEGER, + HasCreatedDate INTEGER, HasChangedDate INTEGER, + HasOwner INTEGER, HasChangedBy INTEGER)`, + } + for _, s := range stmts { + if _, err := db.Exec(s); err != nil { + t.Fatalf("fixture %q: %v", s, err) + } + } + return catalog.WrapSqlDB(db) +} From f1fd11c382528bd8cb0bf13ccf285ca5a1e23344 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 25 Sep 2026 15:20:15 +0000 Subject: [PATCH 47/47] fix(java-actions): primitive-named type parameters and list of a type parameter Two gaps left after the catalog fix for #1183, both in how a Java action's type parameters survive MDL: A type parameter named after a primitive (Studio Pro allows `String`) could not be declared -- `entity ` was a parse error -- and DESCRIBE printed its references bare, so `returns String` re-parsed as the primitive. The declaration slot now takes identifierOrKeyword, and DESCRIBE renders every type-parameter name through mdlIdent: quoted `"String"` is the type parameter, unquoted `String` the primitive. describe -> exec -> describe is now identical. `list of T` for a type parameter T went down the entity path and was written as a list of the entity `.T` (mx check CE1613), and a Studio Pro "List of " read back as a bare `List`. ListType now carries a type-parameter reference: read and written as a ParameterizedEntityType list element (Model SDK: createInListTypeUnderParameter, metamodel 7.21.0+), bound at CREATE, described as `List of T`, and cataloged as `List of TypeParameter:T`. mx check on 11.6.6: previous build CE1613 x2; fixed build 0 errors, also with the type parameter named String. Refs mendixlabs/mxcli#1183 Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G2c72jsT9JsY1c3V2eViLQ --- .../fix-issue/findings/mdl-backend.jsonl | 1 + .../fix-issue/findings/mdl-grammar.jsonl | 1 + .claude/skills/mendix/java-actions/SKILL.md | 5 ++ docs/01-project/MDL_QUICK_REFERENCE.md | 2 +- ...alog-java-action-type-parameter-return.mdl | 48 ++++++++++------ mdl/backend/modelsdk/java_read.go | 39 +++++++++---- mdl/backend/modelsdk/java_read_test.go | 53 +++++++++++++++++ mdl/backend/modelsdk/java_write.go | 36 +++++++----- mdl/backend/modelsdk/javascript_read.go | 29 ++++++++-- mdl/catalog/builder_java_actions_test.go | 6 ++ mdl/catalog/builder_modules.go | 9 ++- mdl/executor/cmd_javaactions.go | 57 +++++++++++++++++-- mdl/executor/cmd_javaactions_test.go | 45 +++++++++++++++ mdl/executor/cmd_javascript_actions.go | 8 +-- mdl/grammar/domains/MDLDomainModel.g4 | 2 +- mdl/types/javaaction_types.go | 13 ++++- mdl/visitor/visitor_helpers.go | 6 +- mdl/visitor/visitor_javaaction_test.go | 37 ++++++++++++ 18 files changed, 333 insertions(+), 64 deletions(-) diff --git a/.claude/skills/fix-issue/findings/mdl-backend.jsonl b/.claude/skills/fix-issue/findings/mdl-backend.jsonl index ba2d818091..ed2d0cbed4 100644 --- a/.claude/skills/fix-issue/findings/mdl-backend.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-backend.jsonl @@ -132,3 +132,4 @@ {"area": "mdl/backend", "date": "2026-09-23", "symptom": "`describe java action` prints `ContextObject: entity <>` for a parameter declared `entity not null`; a bare type-parameter reference (`Obj: pEntity`, `returns pEntity`) reads back nameless too. The description no longer round-trips", "cause": "The stored parameter type (`CodeActions$EntityTypeParameterType` / `ParameterizedEntityType`) holds only a BY_ID pointer to the `CodeActions$TypeParameter`. `javaActionFromGen` carried the ID into the semantic type but never resolved it to the name, and the name is all the describer prints. `javascript_read.go` had always done this resolution pass; the Java reader was ported without it", "file": "`mdl/backend/modelsdk/java_read.go` (`resolveJavaActionTypeParameterNames`)", "insight": "The executor's `entity <>` fallback in `formatJavaActionType` is the tell: an empty name at DESCRIBE means the *reader* dropped a by-ID resolution, not that the writer lost it \u2014 the write path sets both ID and name, so a create\u2192read unit test in the backend reproduces it without any project. When a JS and a Java reader cover the same `CodeActions$` shapes, diff their post-processing first; any pass one has and the other lacks is a candidate. The write was never wrong (replaying the fixed description into a fresh project describes identically), so no `mx check` run is needed. Issue mendixlabs/mxcli#1034", "refs": ["mendixlabs/mxcli#1034"]} {"area": "mdl/backend", "date": "2026-09-23", "symptom": "`CREATE OR MODIFY PERSISTENT ENTITY` was REFUSED by the #1119 storage-GUID guard on a doctype script that had been passing for months: `failed to update entity: refusing to write unit d82b0484-…: 1 element(s) kept their $ID but would be written with a different GUID — dff2ced1-… (DomainModels$Attribute): stored 4b52b36b-…, would write dff2ced1-…`. Two independent defects wore that one message.", "cause": "(1) A REAL data loss the guard caught: `mergeDeclaredOntoStoredEntity` sets `merged.Attributes = declared.Attributes` and `merged.Indexes = declared.Indexes` — the lists the STATEMENT declares, built from text by the visitor and carrying no element ID — and `carryChildIdentity` keyed entirely on that ID, reading an empty one as 'a genuinely new member, so a fresh GUID is right'. Every attribute of a re-declared entity was therefore re-minted, i.e. #1119 through a second executor path. (2) A FALSE POSITIVE in the guard itself: `canon.TransplantIDs` pairs STRUCTURALLY ($Type + shape, LCS-anchored), so on a statement that drops six differently-named attributes and adds one, it paired the NEW attribute with a REMOVED one and handed it that stored `$ID`; the codec had written `GUID = $ID` and the transplant substitutes over every 16-byte binary, so the GUID followed. The guard's premise — written into its own doc comment as 'unambiguously' — that a shared `$ID` after the transplant means the same element, is false.", "file": "`mdl/backend/modelsdk/domainmodel_child_identity.go` (`carryAttributeIdentity`, `carryIndexIdentity`), `modelsdk/canon/storageguid.go` (`sameMember`, `elementGUIDs`)", "insight": "ONE ERROR MESSAGE, TWO DEFECTS, AND FIXING EITHER ALONE LEAVES IT RED — which is why the first fix (the name fallback) changed nothing and the SAME element and GUIDs came back byte-for-byte. That repetition was the signal: an identical failure after a real fix means the reproduction is exercising a different code path than the one reasoned about. What settled it was describing the actual subject: the marketplace `PublishedBusinessEvent` has six attributes and NONE is named `EventId`, so there was no member to carry — the pairing itself was spurious. Reproduce against the real stored document before believing any theory about which elements correspond. METHOD that made this cheap: the CI failure reproduced locally in 0.5s as a backend unit test (strip the IDs off a fixture entity's attributes, call UpdateEntity) versus 26s for the integration subtest, but ONLY the integration subtest could have found the second defect, because the false pairing needs a real drop-six-add-one document. Run both. TRADE-OFF worth restating: the guard now pairs on `$ID` + `$Type` + `Name`, which loses one arm — a RENAME that re-mints a GUID is no longer refused, since the name is what changed — and that arm is covered directly by the carry tests where it is decidable. A backstop that refuses correct writes is worse than a backstop with a hole: the first makes documented statements unusable, and this one already had. Also: feeding a deliberately-approximate pairing to a guard promotes its error rate into refusals. TransplantIDs' correctness bar is low ON PURPOSE (a wrong match only makes a diff bigger); anything that reads its output as identity has to add its own test of identity.", "refs": ["mendixlabs/mxcli#1119", "mendixlabs/mxcli#1169", "ako/mxcli#643"], "ce": []} {"area":"mdl/backend","date":"2026-09-25","symptom":"A data view with `DataSource: nanoflow Module.NF` (e.g. describe → exec of Feedback v4.0.2's FeedbackModule.ShareFeedback) passes `mxcli check` and exec, then mxbuild 11.13.0 reports CE2633 \"No nanoflow configured for the data source of this data view\". Same result through `alter page … set DataSource = nanoflow X on dv`","cause":"Both writers nested the name in a `Forms$NanoflowSettings` child (ParameterMappings marker 3) by analogy with `Forms$MicroflowSource`, which really does nest `Forms$MicroflowSettings`. Studio Pro's `Forms$NanoflowSource` is FLAT: ForceFullObjects, Nanoflow, ParameterMappings (marker 2) directly on the source — exactly gen's shape. mxbuild found no Nanoflow key. The raw nested builder also never read d.ParameterMappings, so a parameterized source nanoflow lost its arguments. On the read side, the ALTER PAGE flow-context lookup (`flowFromDataSourceDoc`) and describe's argument reader (`flowSourceArgs`) only knew the nested shape, so Studio Pro-authored nanoflow sources yielded no entity context / no arguments","file":"`mdl/backend/modelsdk/widget_write_legacy_gaps.go` (`nanoflowSourceToGen` via gen + `Forms$NanoflowSource` TypeDefaults in `widget_write.go`), `mdl/backend/pagemutator/mutator.go` (`serializeDataSourceBson`, `flowFromDataSourceDoc`), `mdl/executor/cmd_pages_describe_datasource.go` (`flowSourceArgs`)","insight":"**The code comment asserted the wrong shape as a measured fact** (\"Studio Pro nests it in a Forms$NanoflowSettings child … Legacy's shape is the one with a working project behind it\") and a unit test pinned it — both were parity-with-legacy, never measured. `Forms$NanoflowSettings` is not a type in modelsdk/gen or generated/metamodel: **when gen and a hand-rolled builder disagree about a type's shape, grep gen for the type the builder invents before trusting the builder**. What settled it in one step: a 60-line scanner that `bson.Unmarshal`s every mprcontents unit and prints the key-set (with list markers) of each `$Type` instance — 5 of 5 flat nanoflow sources, and 4 of 4 microflow sources nested as gen says, so the microflow path needed nothing. **Enumerate every writer of the type, not just the reported one**: the ALTER PAGE setter had its own copy of the same wrong literal, and the read-side lookups keyed on the wrong shape meant Studio Pro pages were the ones silently mis-read. Readers keep the nested fallback for pages written before the fix. Verified: exec + `mx check` 11.13.0 CE2633 → 0 errors for CREATE PAGE (ShareFeedback round trip, repro script) and ALTER PAGE; ShareFeedback's dataView5 DataSource ndsl now matches Studio Pro exactly. Control: implementation reverted → the 5 new tests fail with the nested key set. Repro `mdl-examples/bug-tests/dataview-nanoflow-source-ce2633.mdl`","refs":[],"ce":["CE2633"]} +{"area": "mdl/backend", "date": "2026-09-25", "symptom": "`returns list of pEntity` for a declared type parameter produced mx check CE1613 \"The selected entity '.pEntity' no longer exists.\" on the action and on a `list of pEntity` parameter; a Studio Pro \"List of \" read back as a bare `List` (DESCRIBE and catalog) and a rewrite of it would serialize a list of an unnamed entity.", "cause": "types.ListType carried only Entity. The reader handled only a ConcreteEntityType list element, the writer always emitted one, and CREATE sent `list of T` down the entity path (Module \"\" + \".\" + T).", "file": "mdl/types/javaaction_types.go (ListType.TypeParameterID), mdl/backend/modelsdk/java_read.go (listTypeFromGen), java_write.go (codeActionListTypeToGen), javascript_read.go, mdl/executor/cmd_javaactions.go (listOfTypeParameter)", "insight": "The Model SDK is the arbiter for which element a slot accepts: `ParameterizedEntityType.createInListTypeUnderParameter` (metamodel 7.21.0+) settles that a list element may be a type parameter, so no version gate. No fixture had a Studio Pro-authored instance, so the evidence is mx check on 11.6.6: previous build CE1613 x2, fixed build 0 errors, clean baseline 0. The JavaScript writer reuses the Java converter, so one write fix covers both; the JS reader is separate raw-map code and needed its own case.", "refs": ["mendixlabs/mxcli#1183"]} diff --git a/.claude/skills/fix-issue/findings/mdl-grammar.jsonl b/.claude/skills/fix-issue/findings/mdl-grammar.jsonl index 324517e988..fcafec92cc 100644 --- a/.claude/skills/fix-issue/findings/mdl-grammar.jsonl +++ b/.claude/skills/fix-issue/findings/mdl-grammar.jsonl @@ -62,3 +62,4 @@ {"area": "mdl/grammar", "date": "2026-09-22", "symptom": "A `create workflow` clause written in the \"wrong\" position is a parse error — `on created microflow` anywhere but between the targeting clauses and `entity` gives `line 6:4 mismatched input 'ON' expecting ';'`, and a header clause out of place gives `mismatched input 'DISPLAY' expecting {ON, BEGIN, EXPORT, DUE, OVERVIEW}`. Neither names the clause or the rule, and one misplaced clause cascades into 3–7 more errors including a bogus `extraneous input 'END'`. The reporter reverse-engineered the order empirically and wrote it into their notes", "cause": "`createWorkflowStatement` and `workflowUserTaskStmt` were a fixed SEQUENCE of optional groups — each clause optional, its POSITION not — and the VISITOR depended on that: it read qualified names by COUNTING (`names[1]` or `names[2]` for the overview page depending on whether PARAMETER was present; `nameIdx` walked page → targeting → on-created → entity) and strings by index off `AllSTRING_LITERAL()`. So the grammar could not simply be relaxed", "file": "`mdl/grammar/domains/MDLWorkflow.g4` (new `workflowHeaderClause`, `workflowUserTaskClause`, `workflowMultiUserTaskClause`), `mdl/visitor/visitor_workflow.go` (`applyWorkflowUserTaskClause`), `mdl/visitor/visitor_workflow_clauses.go` (`checkWorkflowClausesAtMostOnce`)", "insight": "**Positional reading is what makes a clause order load-bearing, so the grammar fix is a visitor fix.** The tell is `names[idx++]` in an exit-listener: the rule already carried a comment warning that reading strings by position had nearly mis-assigned FOLDER, and the same hazard had simply been left standing for qualified names. **A clause set must re-add the at-most-once rule the sequence gave for free**, or `page M.A page M.B` starts parsing with the second silently winning — a worse failure than the parse error it replaces. Enforce it in the visitor, not the grammar: only there can the message say `duplicate PAGE clause on user task Review (already given on line 12)`. **Two spellings that fill one model slot are ONE clause**: `targeting microflow` + `targeting xpath` were both accepted and the LAST one won, though a user task stores one UserSource — order-dependence in its most damaging form, and now a duplicate. **Keep the MULTI alternative's own clause rule rather than collapsing to `MULTI?`** — relaxing the order must not relax the vocabulary, or a single user task starts accepting `decide by`. **Control that settles it**: build a `bin/mxcli` from HEAD in a `git worktree`, exec the canonical-order script with it, and compare the written `.mxunit` against the fixed binary's output for BOTH orders — 6,038 bytes each, identical in every string ≥8 chars, differing only in the randomly minted element `$ID`s. AST `reflect.DeepEqual` between the two orders is the unit-level version of the same claim; both-parse is not enough, since a relaxed grammar over a positional visitor parses and mis-assigns. Found in passing and NOT fixed here: `create workflow … overview page X` writes nothing (`mdl/backend/modelsdk/workflow_write.go` has no `OverviewPage`), while `alter workflow … set overview page` does. Tests `mdl/visitor/visitor_workflow_clause_order_test.go`; repro `mdl-examples/bug-tests/workflow-586-clause-order.mdl` with its `-canonical.mdl` control and `-duplicate-clause.fail.mdl` sibling", "refs": ["ako/mxcli#586"]} {"area": "mdl/grammar", "date": "2026-09-23", "symptom": "`alter page M.P { set 'createFileAction' = microflow M.ACT_CreateFile on fileUploader1; };` (quoted or bare key) fails to parse: `line 2:37 extraneous input 'MyModule' expecting {DROP, ADD, SET, INSERT, REPLACE, '}'}` — a pluggable widget's NAMED action slot, writable on CREATE PAGE since #956, could only be retargeted by REPLACEing the whole widget", "cause": "`alterPageAssignment` special-cased `Action = actionExprV3` and sent every other key to `propertyValueV3`, which has no `microflow ` form. Below the grammar there was also no route: `SetWidgetProperty` would have stringified an action into `PrimitiveValue`, and `SetWidgetAction` writes the built-in click action, not a pluggable slot", "file": "`mdl/grammar/MDLParser.g4` (`alterPageAssignment`: `STRING_LITERAL|identifierOrKeyword EQUALS actionExprV3`), `mdl/visitor/visitor_alter_page.go` (keeps the author's key), `mdl/executor/cmd_alter_page.go` (`applySetPropertyMutator` routes any `*ast.ActionV3` not keyed `Action`), `mdl/backend/pagemutator/mutator.go` (`SetWidgetNamedAction`); tests `mdl/visitor/visitor_alter_page_named_action_test.go`, `mdl/backend/pagemutator/mutator_named_action_test.go`, `mdl/executor/alter_set_named_action_test.go`; example `mdl-examples/bug-tests/995-alter-page-set-named-action-slot.mdl`", "insight": "**Decide action-slot-ness by the stored PropertyType's `ValueType.Type == \"Action\"`, never by the presence of `Value.Action`** — every WidgetValue carries an Action (a NoAction by default) whatever its type, so a field-presence check would \"succeed\" writing into an Integer and change nothing. Unlike CREATE, ALTER has no datasource overlap to yield to (DataSource is its own alternative), so the action alternative goes before the scalar ones and `microflow M.X` parses straight to an action — no DataSourceV3 conversion. The check-time probe (validate_alter_set.go) dry-runs the same setter, so the wrong-type refusal surfaced at `check -p --references` for free. Measured on 11.12.1: set on a DataGrid 2 with Selection unset (slot hidden) also passed `mx check` at 0 errors — the CE0463 in #956's notes was 11.13.0, so ALTER does not yet run MDL-WIDGET10; don't assume either way without a build on the target version", "refs": ["mendixlabs/mxcli#995", "mendixlabs/mxcli#956"]} {"area": "mdl/grammar", "date": "2026-09-24", "symptom": "`DROP PAGE IF EXISTS FieldService.Stub;` -> `line 1:13 extraneous input 'EXISTS' expecting the start of a statement`. No document-level DROP (35 alternatives: entity through folder) accepted IF EXISTS, so any script that dropped something was one-shot and the drop had to be deleted after its first run (ako/mxcli#531, ChipCoV3, 11.14.0).", "cause": "`dropStatement` in MDLParser.g4 never applied the existing `ifExists` rule; only sub-document drops (attribute, index, enum value) and the two security drops had it, each added one statement at a time with its own AST field and handler branch.", "file": "`mdl/grammar/MDLParser.g4` (dropStatement); `mdl/ast/ast_drop.go` (DropGuard embedded in 35 Drop*Stmt); `mdl/visitor/visitor_entity.go` (ExitDropStatement sets it once, via defer); `mdl/executor/registry.go` (Dispatch turns NotFoundError into a skip); `mdl/executor/validate.go` (check --references skips guarded drops); tests `mdl/executor/drop_if_exists_test.go`; example `mdl-examples/bug-tests/531-drop-if-exists.mdl`", "insight": "The per-statement recipe the previous IF EXISTS fixes used (field + handler branch) would have been 35 copies, and the next doctype would miss it. What made one central guard safe was a measurement, not an assumption: a table test driving every bare DROP at a missing target through the registry showed all 35 handlers already return mdlerrors.NotFoundError, and only at lookup, before any mutation — so Dispatch can key on that type and let every other error through. Keep that bare-form table test: it is what breaks if a new drop handler reports 'not found' with fmt.Errorf. Mock pitfall on the way: MockBackend's Get*ByQualifiedName defaults to (nil, nil), which the real backend never returns for a missing document — mirror its 'not found' error or the handler dereferences nil. And check --references resolved a drop's MODULE, so a guarded drop in a missing module passed exec but failed check until validateWithContext skipped it too."} +{"area": "mdl/grammar", "date": "2026-09-25", "symptom": "`create java action \u2026(EntityType: entity not null, \u2026)` failed with \"mismatched input 'String' expecting IDENTIFIER\"; Studio Pro accepts a type parameter named after a primitive, so such an action could not be authored, and DESCRIBE of a Studio Pro-authored one printed `returns String`, which re-parses as the primitive.", "cause": "dataType's `ENTITY LESS_THAN IDENTIFIER GREATER_THAN` excluded keyword tokens, and the describer emitted type-parameter names bare where a bare keyword means something else.", "file": "mdl/grammar/domains/MDLDomainModel.g4, mdl/executor/cmd_javaactions.go (formatCodeActionTypeParameterRef)", "insight": "Two halves of one round trip: the declaration slot needed identifierOrKeyword, and the *reference* slots needed no grammar change at all \u2014 a quoted `\"String\"` already parses as a bare qualifiedName, which isTypeParamRef resolves, while unquoted `String` stays the primitive. So the fix on the output side is mdlIdent (which lexes the name with the real lexer) on every type-parameter name DESCRIBE prints. Prove it with a describe \u2192 exec into a second module \u2192 describe diff, not by eyeballing one DESCRIBE.", "refs": ["mendixlabs/mxcli#1183"]} diff --git a/.claude/skills/mendix/java-actions/SKILL.md b/.claude/skills/mendix/java-actions/SKILL.md index 22130fc91d..5f31c97e2e 100644 --- a/.claude/skills/mendix/java-actions/SKILL.md +++ b/.claude/skills/mendix/java-actions/SKILL.md @@ -87,6 +87,10 @@ return true; $$; ``` +A list of type-parameter instances is `list of pEntity` (Studio Pro's "List of "). + +**A type parameter named after a primitive** (`String`, `Integer`, … — Studio Pro allows it): declare it with `entity `, and refer to it **quoted** — `"String"`, `list of "String"`. Unquoted `String` is always the primitive. DESCRIBE quotes such names for you, so its output re-creates the action. + Type parameter names can be mixed with regular parameter types: ```mdl @@ -203,6 +207,7 @@ message saying so. | `enum Module.EnumName` | Enumeration type | | `enumeration(Module.EnumName)` | Enumeration type (alternative syntax) | | `pEntity` (type param ref) | Type parameter reference (entity instance) | +| `list of pEntity` | List of type-parameter instances | ### Examples diff --git a/docs/01-project/MDL_QUICK_REFERENCE.md b/docs/01-project/MDL_QUICK_REFERENCE.md index 6bff72d057..b73b13cb73 100644 --- a/docs/01-project/MDL_QUICK_REFERENCE.md +++ b/docs/01-project/MDL_QUICK_REFERENCE.md @@ -1404,7 +1404,7 @@ Module.OrderResponse_CustomerInfo/Module.CustomerInfo as customer { **`AS $$ ... $$` is mandatory** — the body cannot be omitted. Omitting it causes `no viable alternative at input '...'`. Use `as $$ return false; $$;` as a stub. -**Parameter Types:** `string`, `integer`, `long`, `decimal`, `boolean`, `datetime`, `Module.Entity`, `list of Module.Entity`, `enum Module.EnumName`, `enumeration(Module.EnumName)`, `stringtemplate(sql)`, `stringtemplate(Oql)`, `entity ` (type parameter declaration), bare `pEntity` (type parameter reference). +**Parameter Types:** `string`, `integer`, `long`, `decimal`, `boolean`, `datetime`, `Module.Entity`, `list of Module.Entity`, `enum Module.EnumName`, `enumeration(Module.EnumName)`, `stringtemplate(sql)`, `stringtemplate(Oql)`, `entity ` (type parameter declaration), bare `pEntity` (type parameter reference), `list of pEntity` (list of type-parameter instances). A type parameter named after a primitive is referenced quoted (`"String"`); unquoted `String` is the primitive. **Type Parameters** allow generic entity handling. `entity ` declares the type parameter inline and becomes the entity type selector; bare `pEntity` parameters receive entity instances: ```sql diff --git a/mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl b/mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl index 52dfc9c15d..98fe9a4b3f 100644 --- a/mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl +++ b/mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl @@ -1,37 +1,53 @@ -- ============================================================================ --- Bug #1183: catalog shows a type-parameter return type as a bare name +-- Bug #1183: java action type parameters — catalog, DESCRIBE and list of T -- ============================================================================ -- -- Symptom (before fix): -- "when I create a Java action with a TypeParameter named String, which is -- allowed by Studio, and use that as return type, the catalog will show -- 'String', similar to the primitive 'String'." --- java_actions.ReturnType (and java_action_parameters.ParameterType) held the --- type parameter's own name — 'TypeParameter', 'TypeParEntity', … — with --- nothing marking it as a type parameter. +-- Also: +-- * `entity ` was a parse error ("expecting IDENTIFIER"), so such an +-- action could not be authored or re-created from its DESCRIBE output, and +-- DESCRIBE printed the bare `String` — which re-parses as the primitive. +-- * `list of T` for a type parameter T was written as a list of the entity +-- `.T` (mx check: CE1613 "The selected entity '.T' no longer exists."), and +-- a Studio Pro "List of " read back as a bare `List`. -- -- After fix: --- A type-parameter reference is encoded `TypeParameter:`, the entity-type --- selector `EntityTypeParameter:`; primitives are unchanged. --- --- MDL cannot declare a type parameter named `String` (`entity ` is a --- parse error), so this script uses Studio Pro's default name. The primitive- --- named case is covered by TestJavaActionTypeParameterNamedAfterPrimitive. +-- * Catalog: `TypeParameter:`, `List of TypeParameter:`, +-- `EntityTypeParameter:`; primitives unchanged. +-- * MDL: `entity ` declares the type parameter; the quoted `"String"` +-- refers to it, the unquoted `String` stays the primitive. DESCRIBE quotes +-- a type-parameter name only where the bare name would re-parse as +-- something else, so its output round-trips. +-- * `list of T` stores a ParameterizedEntityType element; mx check is clean. -- -- Usage: -- mxcli exec mdl-examples/bug-tests/1183-catalog-java-action-type-parameter-return.mdl -p app.mpr +-- mxcli -p app.mpr -c "describe java action BugTest1183.JA_Generic" -- mxcli -p app.mpr -c "refresh catalog" -- mxcli -p app.mpr -c "select QualifiedName, ReturnType from CATALOG.JAVA_ACTIONS where ModuleName = 'BugTest1183'" --- Expected: JA_ReturnsTypeParam -> TypeParameter:TypeParameter --- JA_ReturnsString -> String +-- Expected: JA_Generic -> List of TypeParameter:String +-- JA_SingleGeneric -> TypeParameter:pEntity +-- JA_ReturnsString -> String -- ============================================================================ create module BugTest1183; -create java action BugTest1183.JA_ReturnsTypeParam ( - EntityType: entity not null, - Input: TypeParameter -) returns TypeParameter as $$ +create java action BugTest1183.JA_Generic ( + EntityType: entity not null, + Input: "String", + Items: list of "String", + Text: String +) returns list of "String" as $$ + return Items; +$$; + +create java action BugTest1183.JA_SingleGeneric ( + EntityType: entity not null, + Input: pEntity +) returns pEntity as $$ return Input; $$; diff --git a/mdl/backend/modelsdk/java_read.go b/mdl/backend/modelsdk/java_read.go index 0bd514ebae..8ef40a3908 100644 --- a/mdl/backend/modelsdk/java_read.go +++ b/mdl/backend/modelsdk/java_read.go @@ -133,10 +133,23 @@ func resolveJavaActionTypeParameterNames(ja *javaactions.JavaAction) { if pt.TypeParameter == "" { pt.TypeParameter = ja.FindTypeParameterName(pt.TypeParameterID) } + case *javaactions.ListType: + resolveListTypeParameterName(ja, pt) } } - if tp, ok := ja.ReturnType.(*javaactions.TypeParameter); ok && tp.TypeParameter == "" { - tp.TypeParameter = ja.FindTypeParameterName(tp.TypeParameterID) + switch rt := ja.ReturnType.(type) { + case *javaactions.TypeParameter: + if rt.TypeParameter == "" { + rt.TypeParameter = ja.FindTypeParameterName(rt.TypeParameterID) + } + case *javaactions.ListType: + resolveListTypeParameterName(ja, rt) + } +} + +func resolveListTypeParameterName(ja *javaactions.JavaAction, l *javaactions.ListType) { + if l.TypeParameterID != "" && l.TypeParameter == "" { + l.TypeParameter = ja.FindTypeParameterName(l.TypeParameterID) } } @@ -182,7 +195,7 @@ func codeActionBasicFromGen(el element.Element) javaactions.CodeActionParameterT case *genCa.ConcreteEntityType: return &javaactions.EntityType{Entity: t.EntityQualifiedName()} case *genCa.ListType: - return &javaactions.ListType{Entity: listElementEntity(t)} + return listTypeFromGen(t) case *genCa.ParameterizedEntityType: return &javaactions.TypeParameter{TypeParameterID: model.ID(t.TypeParameterRefID())} case *genCa.BooleanType: @@ -209,7 +222,7 @@ func codeActionReturnTypeFromGen(el element.Element) javaactions.CodeActionRetur case *genCa.ConcreteEntityType: return &javaactions.EntityType{Entity: t.EntityQualifiedName()} case *genCa.ListType: - return &javaactions.ListType{Entity: listElementEntity(t)} + return listTypeFromGen(t) case *genCa.ParameterizedEntityType: return &javaactions.TypeParameter{TypeParameterID: model.ID(t.TypeParameterRefID())} case *genCa.BooleanType: @@ -225,13 +238,19 @@ func codeActionReturnTypeFromGen(el element.Element) javaactions.CodeActionRetur } } -// listElementEntity extracts the entity qualified name from a gen ListType's -// element parameter (a ConcreteEntityType). -func listElementEntity(l *genCa.ListType) string { - if ce, ok := l.Parameter().(*genCa.ConcreteEntityType); ok { - return ce.EntityQualifiedName() +// listTypeFromGen converts a gen ListType. Its element is a ConcreteEntityType +// or, for Studio Pro's "List of ", a ParameterizedEntityType — +// read as Entity "" before #1183, which described as a bare `List` and rewrote +// as a list of an unnamed entity. The type-parameter name is resolved later by +// resolveJavaActionTypeParameterNames. +func listTypeFromGen(l *genCa.ListType) *javaactions.ListType { + switch el := l.Parameter().(type) { + case *genCa.ConcreteEntityType: + return &javaactions.ListType{Entity: el.EntityQualifiedName()} + case *genCa.ParameterizedEntityType: + return &javaactions.ListType{TypeParameterID: model.ID(el.TypeParameterRefID())} } - return "" + return &javaactions.ListType{} } // readActionInfoBitmaps fills the four toolbox bitmaps from the sub-document's diff --git a/mdl/backend/modelsdk/java_read_test.go b/mdl/backend/modelsdk/java_read_test.go index 05bb598f6d..1b91aaf887 100644 --- a/mdl/backend/modelsdk/java_read_test.go +++ b/mdl/backend/modelsdk/java_read_test.go @@ -162,3 +162,56 @@ func TestReadJavaActionByName_ResolvesTypeParameterNames(t *testing.T) { t.Errorf("return type = %#v, want TypeParameter{pEntity}", got.ReturnType) } } + +// Studio Pro's "List of " is a ListType whose element is a +// ParameterizedEntityType. The reader only knew a ConcreteEntityType element, so +// this read back as ListType{Entity: ""}: DESCRIBE printed a bare `List`, the +// catalog 'List', and any rewrite of the action serialized a list of an entity +// with no name (mendixlabs/mxcli#1183). +func TestReadJavaActionByName_ListOfTypeParameter(t *testing.T) { + proj := copyFixture(t) + b := New() + if err := b.Connect(proj); err != nil { + t.Fatalf("connect: %v", err) + } + t.Cleanup(func() { _ = b.Disconnect() }) + + mod, err := b.GetModuleByName("MyFirstModule") + if err != nil || mod == nil { + t.Fatalf("GetModuleByName: %v", err) + } + tp := &javaactions.TypeParameterDef{Name: "String"} + tp.ID = model.ID("0b6f0d0e-1183-4a00-8000-000000000001") + ja := &javaactions.JavaAction{ + ContainerID: mod.ID, + Name: "ZzJaListOfTypeParam", + TypeParameters: []*javaactions.TypeParameterDef{tp}, + Parameters: []*javaactions.JavaActionParameter{ + {Name: "EntityType", IsRequired: true, + ParameterType: &javaactions.EntityTypeParameterType{TypeParameterID: tp.ID, TypeParameterName: "String"}}, + {Name: "Items", IsRequired: true, + ParameterType: &javaactions.ListType{TypeParameterID: tp.ID, TypeParameter: "String"}}, + }, + ReturnType: &javaactions.ListType{TypeParameterID: tp.ID, TypeParameter: "String"}, + } + if err := b.CreateJavaAction(ja); err != nil { + t.Fatalf("CreateJavaAction: %v", err) + } + + got, err := b.ReadJavaActionByName("MyFirstModule.ZzJaListOfTypeParam") + if err != nil { + t.Fatalf("ReadJavaActionByName: %v", err) + } + check := func(what string, typ any) { + l, ok := typ.(*javaactions.ListType) + if !ok { + t.Errorf("%s = %T, want *ListType", what, typ) + return + } + if l.Entity != "" || l.TypeParameterID != tp.ID || l.TypeParameter != "String" { + t.Errorf("%s = %+v, want a list of type parameter String (ID %s)", what, *l, tp.ID) + } + } + check("list parameter", got.Parameters[1].ParameterType) + check("return type", got.ReturnType) +} diff --git a/mdl/backend/modelsdk/java_write.go b/mdl/backend/modelsdk/java_write.go index e22b3faeae..e842a431cc 100644 --- a/mdl/backend/modelsdk/java_write.go +++ b/mdl/backend/modelsdk/java_write.go @@ -303,13 +303,7 @@ func codeActionInnerTypeToGen(t javaactions.CodeActionParameterType) element.Ele e.SetEntityQualifiedName(v.Entity) return e case *javaactions.ListType: - l := genCa.NewListType() - assignID(l) - ce := genCa.NewConcreteEntityType() - assignID(ce) - ce.SetEntityQualifiedName(v.Entity) - l.SetParameter(ce) - return l + return codeActionListTypeToGen(v) case *javaactions.TypeParameter: p := genCa.NewParameterizedEntityType() assignID(p) @@ -336,13 +330,7 @@ func codeActionReturnTypeToGen(t javaactions.CodeActionReturnType) element.Eleme e.SetEntityQualifiedName(v.Entity) return e case *javaactions.ListType: - l := genCa.NewListType() - assignID(l) - ce := genCa.NewConcreteEntityType() - assignID(ce) - ce.SetEntityQualifiedName(v.Entity) - l.SetParameter(ce) - return l + return codeActionListTypeToGen(v) case *javaactions.TypeParameter: p := genCa.NewParameterizedEntityType() assignID(p) @@ -353,6 +341,26 @@ func codeActionReturnTypeToGen(t javaactions.CodeActionReturnType) element.Eleme } } +// codeActionListTypeToGen converts a list type. The element is a +// ParameterizedEntityType for a list of a type parameter — Studio Pro's "List of +// " — and a ConcreteEntityType otherwise (#1183). +func codeActionListTypeToGen(v *javaactions.ListType) element.Element { + l := genCa.NewListType() + assignID(l) + if v.TypeParameterID != "" { + p := genCa.NewParameterizedEntityType() + assignID(p) + p.SetTypeParameterID(element.ID(v.TypeParameterID)) + l.SetParameter(p) + return l + } + ce := genCa.NewConcreteEntityType() + assignID(ce) + ce.SetEntityQualifiedName(v.Entity) + l.SetParameter(ce) + return l +} + // newPrimitiveCAType builds a bare CodeActions primitive type element by kind (the // element carries only $ID + $Type). func newPrimitiveCAType(kind string) element.Element { diff --git a/mdl/backend/modelsdk/javascript_read.go b/mdl/backend/modelsdk/javascript_read.go index 5ab56b683c..9108475a78 100644 --- a/mdl/backend/modelsdk/javascript_read.go +++ b/mdl/backend/modelsdk/javascript_read.go @@ -138,11 +138,20 @@ func jsActionFromRaw(raw map[string]any, id, containerID model.ID) *types.JavaSc if pt.TypeParameterID != "" && pt.TypeParameter == "" { pt.TypeParameter = jsa.FindTypeParameterName(pt.TypeParameterID) } + case *types.ListType: + if pt.TypeParameterID != "" && pt.TypeParameter == "" { + pt.TypeParameter = jsa.FindTypeParameterName(pt.TypeParameterID) + } } } - if tp, ok := jsa.ReturnType.(*types.TypeParameter); ok { - if tp.TypeParameterID != "" && tp.TypeParameter == "" { - tp.TypeParameter = jsa.FindTypeParameterName(tp.TypeParameterID) + switch rt := jsa.ReturnType.(type) { + case *types.TypeParameter: + if rt.TypeParameterID != "" && rt.TypeParameter == "" { + rt.TypeParameter = jsa.FindTypeParameterName(rt.TypeParameterID) + } + case *types.ListType: + if rt.TypeParameterID != "" && rt.TypeParameter == "" { + rt.TypeParameter = jsa.FindTypeParameterName(rt.TypeParameterID) } } @@ -180,7 +189,12 @@ func parseCodeActionReturnTypeRaw(raw map[string]any) types.CodeActionReturnType if entity := jsExtractString(raw["Entity"]); entity != "" { lt.Entity = entity } else if param := jsToMap(raw["Parameter"]); param != nil { - lt.Entity = jsExtractString(param["Entity"]) + // "List of " holds a ParameterizedEntityType (#1183). + if jsExtractString(param["$Type"]) == "CodeActions$ParameterizedEntityType" { + lt.TypeParameterID = model.ID(jsTypeParamPointer(param)) + } else { + lt.Entity = jsExtractString(param["Entity"]) + } } return lt case "CodeActions$FileDocumentType": @@ -250,7 +264,12 @@ func parseCodeActionParameterTypeRaw(raw map[string]any) types.CodeActionParamet if entity := jsExtractString(raw["Entity"]); entity != "" { lt.Entity = entity } else if param := jsToMap(raw["Parameter"]); param != nil { - lt.Entity = jsExtractString(param["Entity"]) + // "List of " holds a ParameterizedEntityType (#1183). + if jsExtractString(param["$Type"]) == "CodeActions$ParameterizedEntityType" { + lt.TypeParameterID = model.ID(jsTypeParamPointer(param)) + } else { + lt.Entity = jsExtractString(param["Entity"]) + } } return lt case "CodeActions$StringTemplateParameterType": diff --git a/mdl/catalog/builder_java_actions_test.go b/mdl/catalog/builder_java_actions_test.go index 90889942ca..4bd6e4eb04 100644 --- a/mdl/catalog/builder_java_actions_test.go +++ b/mdl/catalog/builder_java_actions_test.go @@ -38,6 +38,11 @@ func TestJavaActionTypeParameterNamedAfterPrimitive(t *testing.T) { Name: "Input", ParameterType: &javaactions.TypeParameter{TypeParameterID: "tp-string", TypeParameter: "String"}, }, + { + BaseElement: model.BaseElement{ID: "p-list"}, + Name: "Items", + ParameterType: &javaactions.ListType{TypeParameterID: "tp-string", TypeParameter: "String"}, + }, }, } primitive := &javaactions.JavaAction{ @@ -100,6 +105,7 @@ func TestJavaActionTypeParameterNamedAfterPrimitive(t *testing.T) { "Text": "String", "Input": "TypeParameter:String", "EntityType": "EntityTypeParameter:String", + "Items": "List of TypeParameter:String", } for name, w := range want { if params[name] != w { diff --git a/mdl/catalog/builder_modules.go b/mdl/catalog/builder_modules.go index 7beeab0c35..5f7c957ce1 100644 --- a/mdl/catalog/builder_modules.go +++ b/mdl/catalog/builder_modules.go @@ -362,8 +362,9 @@ func (b *Builder) buildEnumerations() error { // (mendixlabs/mxcli#1183). The prefix follows the `Kind:Name` shape // microflows_data.ReturnType already uses; no primitive contains a colon. // -// TypeParameter:T — an object of the entity bound to T -// EntityTypeParameter:T — the entity-type selector that binds T +// TypeParameter:T — an object of the entity bound to T +// List of TypeParameter:T — a list of them +// EntityTypeParameter:T — the entity-type selector that binds T // // DESCRIBE keeps the bare name, which is its MDL syntax; this is the catalog's // encoding only. @@ -373,6 +374,10 @@ func catalogCodeActionType(t interface{ TypeString() string }) string { return "TypeParameter:" + tp.TypeParameter case *javaactions.EntityTypeParameterType: return "EntityTypeParameter:" + tp.TypeParameterName + case *javaactions.ListType: + if tp.TypeParameter != "" { + return "List of TypeParameter:" + tp.TypeParameter + } } return t.TypeString() } diff --git a/mdl/executor/cmd_javaactions.go b/mdl/executor/cmd_javaactions.go index 4d883a6eb3..6d49aad68b 100644 --- a/mdl/executor/cmd_javaactions.go +++ b/mdl/executor/cmd_javaactions.go @@ -227,12 +227,8 @@ func formatJavaActionType(t javaactions.CodeActionParameterType) string { if t == nil { return "Object" } - // EntityTypeParameterType → ENTITY syntax - if etp, ok := t.(*javaactions.EntityTypeParameterType); ok { - if etp.TypeParameterName != "" { - return "entity <" + etp.TypeParameterName + ">" - } - return "entity <>" + if s, ok := formatCodeActionTypeParameterRef(t); ok { + return s } return t.TypeString() } @@ -242,9 +238,32 @@ func formatJavaActionReturnType(t javaactions.CodeActionReturnType) string { if t == nil { return "Void" } + if s, ok := formatCodeActionTypeParameterRef(t); ok { + return s + } return t.TypeString() } +// formatCodeActionTypeParameterRef renders the types that name a type parameter. +// Studio Pro accepts any name for one, including a primitive's, so the name goes +// through mdlIdent: `returns String` re-parses as the primitive, `returns +// "String"` as the type parameter (#1183). +func formatCodeActionTypeParameterRef(t any) (string, bool) { + switch v := t.(type) { + case *javaactions.EntityTypeParameterType: + return "entity <" + mdlIdent(v.TypeParameterName) + ">", true + case *javaactions.TypeParameter: + if v.TypeParameter != "" { + return mdlIdent(v.TypeParameter), true + } + case *javaactions.ListType: + if v.TypeParameter != "" { + return "List of " + mdlIdent(v.TypeParameter), true + } + } + return "", false +} + // execDropJavaAction handles DROP JAVA ACTION statements. func execDropJavaAction(ctx *ExecContext, s *ast.DropJavaActionStmt) error { if !ctx.ConnectedForWrite() { @@ -421,6 +440,8 @@ func execCreateJavaAction(ctx *ExecContext, s *ast.CreateJavaActionStmt) error { TypeParameterID: typeParamNameToID[tpName], TypeParameter: tpName, } + } else if l := listOfTypeParameter(param.Type, typeParamNameToID); l != nil { + jaParam.ParameterType = l } else { jaParam.ParameterType = astDataTypeToJavaActionParamType(param.Type) } @@ -435,6 +456,8 @@ func execCreateJavaAction(ctx *ExecContext, s *ast.CreateJavaActionStmt) error { TypeParameterID: typeParamNameToID[tpName], TypeParameter: tpName, } + } else if l := listOfTypeParameter(s.ReturnType, typeParamNameToID); l != nil { + ja.ReturnType = l } else { ja.ReturnType = astDataTypeToJavaActionReturnType(s.ReturnType) } @@ -739,3 +762,25 @@ func getTypeParamRefName(dt ast.DataType) string { } return "" } + +// listOfTypeParameter returns the list type for `list of T` when T is a type +// parameter declared on the action, and nil otherwise. Without it the element +// went through the entity path and became the entity `.T` — an empty module +// (#1183). Only an unqualified name can be a type parameter. +func listOfTypeParameter(dt ast.DataType, typeParamNameToID map[string]model.ID) *javaactions.ListType { + if dt.Kind != ast.TypeListOf || dt.EntityRef == nil || dt.EntityRef.Module != "" { + return nil + } + id, ok := typeParamNameToID[dt.EntityRef.Name] + if !ok { + return nil + } + return &javaactions.ListType{ + BaseElement: model.BaseElement{ + ID: model.ID(types.GenerateID()), + TypeName: "CodeActions$ListType", + }, + TypeParameter: dt.EntityRef.Name, + TypeParameterID: id, + } +} diff --git a/mdl/executor/cmd_javaactions_test.go b/mdl/executor/cmd_javaactions_test.go index 86ad4eab09..45027fa30f 100644 --- a/mdl/executor/cmd_javaactions_test.go +++ b/mdl/executor/cmd_javaactions_test.go @@ -6,6 +6,7 @@ import ( "testing" "github.com/mendixlabs/mxcli/mdl/ast" + "github.com/mendixlabs/mxcli/model" "github.com/mendixlabs/mxcli/sdk/javaactions" "github.com/mendixlabs/mxcli/sdk/microflows" ) @@ -395,3 +396,47 @@ func isType[T any](v any) bool { _, ok := v.(T) return ok } + +// A type parameter may carry a primitive's name (#1183). DESCRIBE must quote it +// wherever the bare name would re-parse as something else: `returns String` is +// the primitive, `returns "String"` the type parameter. +func TestFormatJavaActionTypes_TypeParameterNamedAfterPrimitive(t *testing.T) { + cases := []struct { + got, want string + }{ + {formatJavaActionReturnType(&javaactions.TypeParameter{TypeParameter: "String"}), `"String"`}, + {formatJavaActionType(&javaactions.TypeParameter{TypeParameter: "String"}), `"String"`}, + {formatJavaActionType(&javaactions.EntityTypeParameterType{TypeParameterName: "String"}), `entity <"String">`}, + {formatJavaActionReturnType(&javaactions.ListType{TypeParameter: "String"}), `List of "String"`}, + {formatJavaActionType(&javaactions.ListType{TypeParameter: "String"}), `List of "String"`}, + // Controls: an ordinary name stays bare, and the primitive is untouched. + {formatJavaActionReturnType(&javaactions.TypeParameter{TypeParameter: "pEntity"}), `pEntity`}, + {formatJavaActionReturnType(&javaactions.ListType{TypeParameter: "pEntity"}), `List of pEntity`}, + {formatJavaActionReturnType(&javaactions.StringType{}), `String`}, + {formatJavaActionReturnType(&javaactions.ListType{Entity: "Mod.Ent"}), `List of Mod.Ent`}, + } + for _, c := range cases { + if c.got != c.want { + t.Errorf("got %s, want %s", c.got, c.want) + } + } +} + +// `returns list of pEntity` for a declared type parameter pEntity became a list +// of the entity `.pEntity` — an empty module — instead of a list of the type +// parameter (#1183). +func TestListOfTypeParameter_BindsToTheTypeParameter(t *testing.T) { + ids := map[string]model.ID{"pEntity": "tp-1"} + dt := ast.DataType{Kind: ast.TypeListOf, EntityRef: &ast.QualifiedName{Name: "pEntity"}} + l := listOfTypeParameter(dt, ids) + if l == nil || l.TypeParameterID != "tp-1" || l.TypeParameter != "pEntity" || l.Entity != "" { + t.Fatalf("got %+v, want a list of type parameter pEntity", l) + } + // Controls: a qualified entity and an undeclared name are not type parameters. + if l := listOfTypeParameter(ast.DataType{Kind: ast.TypeListOf, EntityRef: &ast.QualifiedName{Module: "Mod", Name: "pEntity"}}, ids); l != nil { + t.Errorf("qualified entity bound as type parameter: %+v", l) + } + if l := listOfTypeParameter(ast.DataType{Kind: ast.TypeListOf, EntityRef: &ast.QualifiedName{Name: "Other"}}, ids); l != nil { + t.Errorf("undeclared name bound as type parameter: %+v", l) + } +} diff --git a/mdl/executor/cmd_javascript_actions.go b/mdl/executor/cmd_javascript_actions.go index 198611fee0..f4ad6216c0 100644 --- a/mdl/executor/cmd_javascript_actions.go +++ b/mdl/executor/cmd_javascript_actions.go @@ -263,12 +263,8 @@ func formatJavaScriptActionType(t javaactions.CodeActionParameterType) string { if t == nil { return "Object" } - // EntityTypeParameterType → ENTITY syntax - if etp, ok := t.(*javaactions.EntityTypeParameterType); ok { - if etp.TypeParameterName != "" { - return "entity <" + etp.TypeParameterName + ">" - } - return "entity <>" + if s, ok := formatCodeActionTypeParameterRef(t); ok { + return s } return t.TypeString() } diff --git a/mdl/grammar/domains/MDLDomainModel.g4 b/mdl/grammar/domains/MDLDomainModel.g4 index 0253288d31..501bae4602 100644 --- a/mdl/grammar/domains/MDLDomainModel.g4 +++ b/mdl/grammar/domains/MDLDomainModel.g4 @@ -109,7 +109,7 @@ dataType | CURRENCY_TYPE | FLOAT_TYPE | STRINGTEMPLATE_TYPE LPAREN templateContext RPAREN // StringTemplate(Sql) etc. - | ENTITY LESS_THAN IDENTIFIER GREATER_THAN // ENTITY type parameter declaration + | ENTITY LESS_THAN identifierOrKeyword GREATER_THAN // ENTITY ; a keyword name (String) is legal in Studio Pro, #1183 | ENUM_TYPE qualifiedName | ENUMERATION LPAREN qualifiedName RPAREN // Enumeration(Module.Enum) syntax | LIST_OF qualifiedName diff --git a/mdl/types/javaaction_types.go b/mdl/types/javaaction_types.go index 9ebcaa1c73..d566b73d15 100644 --- a/mdl/types/javaaction_types.go +++ b/mdl/types/javaaction_types.go @@ -165,14 +165,25 @@ func (e EntityType) TypeString() string { } // ListType represents a list type. +// +// The element is either a concrete entity (Entity) or a type parameter +// (TypeParameterID, a BY_ID reference to a TypeParameterDef, with TypeParameter +// its resolved name) — Studio Pro's "List of ". Reading the +// latter as Entity "" made it describe as a bare `List` and rewrite as a list of +// an entity with no name (mendixlabs/mxcli#1183). type ListType struct { model.BaseElement - Entity string `json:"entity,omitempty"` // Qualified entity name for list items + Entity string `json:"entity,omitempty"` // Qualified entity name for list items + TypeParameter string `json:"typeParameter,omitempty"` // resolved type-parameter name, when the element is one + TypeParameterID model.ID `json:"typeParameterId,omitempty"` // BY_ID reference to TypeParameterDef } func (ListType) isCodeActionReturnType() {} func (ListType) isCodeActionParameterType() {} func (l ListType) TypeString() string { + if l.TypeParameter != "" { + return "List of " + l.TypeParameter + } if l.Entity != "" { return "List of " + l.Entity } diff --git a/mdl/visitor/visitor_helpers.go b/mdl/visitor/visitor_helpers.go index e4c557eb03..dbcf1335c1 100644 --- a/mdl/visitor/visitor_helpers.go +++ b/mdl/visitor/visitor_helpers.go @@ -346,10 +346,12 @@ func buildDataType(ctx parser.IDataTypeContext) ast.DataType { } // Handle ENTITY — type parameter declaration for Java actions - if dtCtx.ENTITY() != nil && dtCtx.LESS_THAN() != nil && dtCtx.IDENTIFIER() != nil { + // The name may be a keyword or quoted: Studio Pro accepts any name for a + // type parameter, including a primitive's (`entity `, #1183). + if dtCtx.ENTITY() != nil && dtCtx.LESS_THAN() != nil && dtCtx.IdentifierOrKeyword() != nil { return ast.DataType{ Kind: ast.TypeEntityTypeParam, - TypeParamName: dtCtx.IDENTIFIER().GetText(), + TypeParamName: identifierOrKeywordText(dtCtx.IdentifierOrKeyword()), } } diff --git a/mdl/visitor/visitor_javaaction_test.go b/mdl/visitor/visitor_javaaction_test.go index 2dc12d3c07..bdac2e88e6 100644 --- a/mdl/visitor/visitor_javaaction_test.go +++ b/mdl/visitor/visitor_javaaction_test.go @@ -566,3 +566,40 @@ func TestJavaAction_OrModify(t *testing.T) { t.Error("Expected CreateOrModify=true") } } + +// Studio Pro accepts any name for a type parameter, including a primitive's. +// `entity ` was a parse error ("expecting IDENTIFIER"), so such an +// action could be neither authored nor re-created from its DESCRIBE output +// (mendixlabs/mxcli#1183). The bare reference to it is the quoted `"String"` — +// unquoted `String` stays the primitive. +func TestJavaAction_TypeParameterNamedAfterPrimitive(t *testing.T) { + for _, decl := range []string{`entity `, `entity <"String">`} { + input := `create java action MyModule.Gen( + EntityType: ` + decl + ` not null, + Input: "String", + Text: String +) returns list of "String" as $$ +return null; +$$;` + prog, errs := Build(input) + if len(errs) > 0 { + t.Fatalf("%s: parse errors: %v", decl, errs) + } + stmt := prog.Statements[0].(*ast.CreateJavaActionStmt) + if len(stmt.TypeParameters) != 1 || stmt.TypeParameters[0] != "String" { + t.Errorf("%s: type parameters = %v, want [String]", decl, stmt.TypeParameters) + } + if got := stmt.Parameters[0].Type; got.Kind != ast.TypeEntityTypeParam || got.TypeParamName != "String" { + t.Errorf("%s: selector = %+v, want entity type param String", decl, got) + } + if got := stmt.Parameters[1].Type; got.Kind != ast.TypeEnumeration || got.EnumRef == nil || got.EnumRef.Name != "String" || got.EnumRef.Module != "" { + t.Errorf("%s: quoted reference = %+v, want bare name String", decl, got) + } + if got := stmt.Parameters[2].Type; got.Kind != ast.TypeString { + t.Errorf("%s: unquoted String = %+v, want the primitive", decl, got) + } + if got := stmt.ReturnType; got.Kind != ast.TypeListOf || got.EntityRef == nil || got.EntityRef.Name != "String" { + t.Errorf("%s: return = %+v, want list of String", decl, got) + } + } +}