diff --git a/CHANGELOG.md b/CHANGELOG.md index 6cf2c71..642291c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ three commits past it), and a bug report can name a release instead of a sha nob Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- log(bug-logs/mxcli-bugs.md, bug-logs/pending-github-issues/marketplace-install-not-grouped-in-studio-pro.md, bug-logs/submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/, skills/learned-mdl-preflight.md row 27): **a module installed by `mxcli marketplace install` shows up in Studio Pro among the app's own modules, not under "Marketplace modules".** The installer stamps 3 of the 5 marketplace identity fields on the module (`AppStoreVersionGuid` and `AppStorePackageIdString` stay empty) and `--file` stamps none; Studio Pro reads all five. Measured on a fresh Mendix 11.13.0 probe (`--file`: `FromAppStore` false, four empty strings) against a real project's Studio Pro-installed Encryption module (all five set). Ledger entry, paste-ready issue draft (NOT YET FILED), a fix patch for upstream on `main` 95091765 (content id threaded into the stamp, all five fields written, 3 unit tests, marketplace + cmd packages green; Studio Pro regrouping not yet proven), preflight STOP row 27 (install by content id, then read the `SHOW MODULES` Source column) and a fifth trap on the #879 entry. — Maurits Visser, field report 2026-09-28 - fix(tests/wave2/test-mxbuild-version-match.sh): **master CI had been red since 2026-09-28 on one check-portability violation** — the test probed `python3`/`python` by name; it now sources `bin/lib/portable.sh` and uses `resolve_py`, keeping its SKIP when no Python 3 with sqlite3 is found. check-portability: 1 violation -> clean (164 files). The test's own PASS=15 FAIL=3 on a Mac with a Studio Pro 11.12.2 Beta in /Applications is unchanged by this and is a fixture-isolation gap (find_mxbuild sees the real /Applications), not a regression. - process(privacy): scrubbed a customer name and an internal project name that reached master through GitHub-merged PRs, where the pre-commit leak guard never runs (bin/check-no-client-data.sh: 32 files hit on master, 0 after). The customer is now "a customer" in prose and the placeholder `acme` in eval identifiers and grader patterns (the graders say to substitute the real prefix before regrading); the project is "a field project", and its inbox patch file is now `contrib/inbox/2026-09-27-field-project-patches.md`. Git history still holds the old text. - log(bug-logs/upstream-log-2026-09.md, bug-logs/pending-github-issues/2026-09-27-*.md): **one index of what went upstream this month.** It lists 4 filed mxcli issues (#1198–#1201; #1199 is already fixed upstream), 1 PR package ready to send, and 7 issue drafts that are not filed yet. The drafts use placeholders for project names. — field project, 2026-09-27 guest-groups build diff --git a/bug-logs/mxcli-bugs.md b/bug-logs/mxcli-bugs.md index 2f99037..e5bd1bf 100644 --- a/bug-logs/mxcli-bugs.md +++ b/bug-logs/mxcli-bugs.md @@ -919,6 +919,15 @@ no `mxcli fix security`, and forcing a recompute with an MDL `GRANT` on a UserCo tried and **does not clear it**. Plan that module for a Studio Pro session; do not install it into a project that must stay buildable headlessly in the meantime. +**5. (added 2026-09-28) A headless install is not a Studio Pro install in the model's eyes — check the +Source column, and know what it cannot tell you.** `marketplace install ` stamps three of the +five marketplace identity fields on the module (`FromAppStore`, `AppStoreVersion`, `AppStoreGuid`); +`--file` stamps none. Studio Pro reads all five, so the module lands among the app's own modules in the +App Explorer instead of under "Marketplace modules". After every install read `SHOW MODULES`: Source +must say `Marketplace v` (a `--file` install shows nothing there, and that is the first sign). +Even when it does, on ≤ v0.24.0 the Studio Pro grouping is still wrong — see +`BUG-DRAFT-marketplace-install-not-grouped-in-studio-pro` for the fix package. + --- ### HISTORICAL RECORD (the bug as it was, kept because a ledger entry that vanishes reads as a bug never found) @@ -6095,3 +6104,60 @@ filed upstream. and not probed here. The toolkit repeats the claim without evidence in `skills/learned-workflow-patterns.md` (the MPR006 row and the page-patterns note). Treat it as unconfirmed until someone runs an empty container. Ask upstream what the crash is. + +--- + +## BUG-DRAFT-marketplace-install-not-grouped-in-studio-pro: a module installed by `marketplace install` is listed among the app's own modules in Studio Pro, not under "Marketplace modules" — the stamp writes 3 of 5 identity fields, and `--file` writes none (2026-09-28) + +> **NOT YET FILED** — paste-ready draft in `bug-logs/pending-github-issues/marketplace-install-not-grouped-in-studio-pro.md`. +> Fix patch ready in `bug-logs/submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/` (file the issue first, then the PR). + +> **Status:** reported 2026-09-28 from a field project on v0.24.0 (modules installed by mxcli showed up +> in Studio Pro among the app's modules). Cause read from source on upstream `main` 95091765 and +> measured on a fresh 11.13.0 probe project plus a real project that has a Studio Pro-installed module. +> The Studio Pro regrouping after the fix is **not yet proven** — it needs a content-id install with the +> patched binary and a Studio Pro open. + +**Family:** the #879 entry above (its fix added the stamp this entry is about; trap 5 there is the +field rule), BUG-132 (`fix design-properties` on marketplace packages), BUG-133 (`_USE_ME` flows). + +**Discovered:** 2026-09-28. **mxcli version:** v0.24.0 (same code on `main` 95091765). **Severity:** +Medium. Nothing fails to build; the module is simply not a Marketplace module to Studio Pro, so it is +not shown as one, not offered updates there, and reads as the app's own code in every review. + +**Repro:** +``` +mxcli marketplace install 1011 -p App.mpr # Encryption, by content id +mxcli -p App.mpr -c "SHOW MODULES" # Source: Marketplace v11.x.y (looks right) +# open in Studio Pro: Encryption is among the app's own modules +``` + +**Expected:** the module carries the five fields a Studio Pro install writes on `Projects$ModuleImpl` +(`FromAppStore`, `AppStoreVersion`, `AppStoreGuid`, `AppStoreVersionGuid`, `AppStorePackageIdString`) +and is listed under "Marketplace modules". + +**Actual (measured):** +- Content-id install / `marketplace update`: `StampMarketplaceVersion` (`cmd/mxcli/marketplace/update.go`) + writes `FromAppStore`, `AppStoreVersion`, `AppStoreGuid` only. `AppStoreVersionGuid` and + `AppStorePackageIdString` stay empty. The content id is never passed down to the stamp. +- `--file` install: nothing is stamped. `FromAppStore` false, four empty strings (fresh 11.13.0 probe, + BusinessEvents package, unit decoded from `mprcontents`). `SHOW MODULES` shows no Source at all. +- Studio Pro-installed reference (Encryption in a real project): all five set, `AppStoreGuid` equals + `AppStoreVersionGuid` (the version UUID), `AppStorePackageIdString` is `"1011"`. +- `SHOW MODULES` and the catalog render `Marketplace v` from `FromAppStore` alone + (`mdl/catalog/builder_modules.go`, `mdl/executor/cmd_modules.go`), so the CLI cannot show the gap. + +**Impact:** every module installed headlessly on a project reads as the app's own module the first time +someone opens it in Studio Pro. Reviews count it as project code; Studio Pro's Marketplace pane does not +list it for update. The `--file` route is worse: the module is not a marketplace module even to mxcli. + +**Workaround (≤ v0.24.0):** install by content id, never `--file`, and read `SHOW MODULES` Source after +every install (preflight STOP row 27). That fixes the `--file` half. The grouping half needs the patch, +or a one-off model patch that copies `AppStoreGuid` into `AppStoreVersionGuid` and writes the content id +into `AppStorePackageIdString` (run it through `./bin/exec.sh --patch` so it is gated and restorable). + +**Fix (proposed upstream):** thread the content id through `installModule` → `PerformInstall` and the +update command → `PerformUpdate`; one helper `stampModuleDoc` writes all five, still only on keys the +document already has (ADR-0005). Unit tests for the helper; marketplace and cmd packages pass. +Scope B (`--file --content-id --version` resolving the UUID through the marketplace client) is a +separate ask in the issue draft. diff --git a/bug-logs/pending-github-issues/marketplace-install-not-grouped-in-studio-pro.md b/bug-logs/pending-github-issues/marketplace-install-not-grouped-in-studio-pro.md new file mode 100644 index 0000000..5702370 --- /dev/null +++ b/bug-logs/pending-github-issues/marketplace-install-not-grouped-in-studio-pro.md @@ -0,0 +1,71 @@ +**Repo:** `mendixlabs/mxcli` +**Source:** `bug-logs/mxcli-bugs.md`, `## BUG-DRAFT-marketplace-install-not-grouped-in-studio-pro` — found 2026-09-28 (field report on v0.24.0; cause read from source on upstream `main` 95091765) +**Status:** NOT YET FILED — fix patch ready in `bug-logs/submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/`; file this first, then open the PR with "closes #" +**Suggested labels:** bug, marketplace +**Duplicate check:** searched 2026-09-28 (`marketplace install FromAppStore`, `Marketplace modules App Explorer`, `AppStoreVersionGuid`). No match. #879 (closed, v0.21.0) was the format collapse; the stamp that this issue is about was added by its fix and has always written three fields. + +--- + +**Title:** `marketplace install` / `update` stamp 3 of 5 marketplace identity fields, so Studio Pro lists the module among the app's own modules instead of under "Marketplace modules" + +**Body:** + +## Summary + +A module Studio Pro installs from the Marketplace carries five fields on its `Projects$ModuleImpl`: `FromAppStore`, `AppStoreVersion`, `AppStoreGuid`, `AppStoreVersionGuid`, `AppStorePackageIdString`. `StampMarketplaceVersion` (`cmd/mxcli/marketplace/update.go`) writes only the first three, so a module installed by `mxcli marketplace install ` or updated by `marketplace update` has an empty `AppStoreVersionGuid` and `AppStorePackageIdString`. Studio Pro reads all five: the module is shown in the App Explorer among the app's own modules instead of under "Marketplace modules", and it is treated as the app's own code from then on. `marketplace install --file ` writes none of the five (`FromAppStore` stays `false`), so the same happens there, and `SHOW MODULES` (which reads `FromAppStore` only) shows `Marketplace v` in the first case and nothing in the second — neither says what Studio Pro will do. + +**Version:** mxcli v0.24.0 (`StampMarketplaceVersion` on `main` 95091765 is the same), Mendix 11.13.0, MPR v2. + +## Repro + +``` +mxcli new App --mendix-version 11.13.0 +mxcli marketplace install 1011 -p App/app/App.mpr # Encryption +mxcli -p App/app/App.mpr -c "SHOW MODULES" # Source: Marketplace v11.x.y +``` + +Open `App.mpr` in Studio Pro 11.13.0 → App Explorer: Encryption is listed among the app's modules, not under "Marketplace modules". + +The module's unit shows why (`mprcontents///.mxunit`, decoded BSON of the `Projects$ModuleImpl`): + +``` +FromAppStore true +AppStoreVersion "11.x.y" +AppStoreGuid "" +AppStoreVersionGuid "" <- empty +AppStorePackageIdString "" <- empty +``` + +The same module installed by Studio Pro in another project: + +``` +FromAppStore true +AppStoreVersion "11.1.1" +AppStoreGuid "" +AppStoreVersionGuid "" +AppStorePackageIdString "1011" +``` + +With `marketplace install --file Encryption.mpk` all five stay at their defaults (`FromAppStore` false, four empty strings). + +## Measured + +- Fresh 11.13.0 project, `--file` install of a marketplace package: module unit has `FromAppStore=false` and all four strings empty (read with a BSON decoder over the `.mxunit`). +- A Studio Pro-installed module in a real project: all five set, `AppStoreGuid == AppStoreVersionGuid`, `AppStorePackageIdString` is the content id as decimal text. +- Reading `cmd/mxcli/marketplace/update.go` on `main`: the stamp is three `set*Field` calls; the content id is not threaded into `PerformInstall` / `PerformUpdate` at all. + +## Expected + +After `marketplace install ` or `marketplace update`, the module carries the same five fields a Studio Pro install writes, and Studio Pro lists it under "Marketplace modules". + +## Proposed fix (PR follows) + +Thread the resolved content id into `PerformInstall` / `PerformUpdate` and have the stamp write the version UUID into both `AppStoreGuid` and `AppStoreVersionGuid` and the content id into `AppStorePackageIdString`, still only assigning keys the document already has (ADR-0005). One helper, unit-tested. `--file` stays unstamped in that PR (a package on disk has no identity). + +## Follow-up ask (separate) + +Let `marketplace install --file` take `--content-id ` and `--version ` and resolve the version UUID through the marketplace client, so an offline install can carry the same identity. Until then the workaround is to install by content id. + +## Workaround + +Install by content id, never `--file`, for a module that must read as a Marketplace module. Check `SHOW MODULES` Source reads `Marketplace v`. The Studio Pro grouping cannot be fixed from mxcli ≤ v0.24.0; a one-off BSON patch that copies `AppStoreGuid` into `AppStoreVersionGuid` and writes the content id into `AppStorePackageIdString` is what the fix does. diff --git a/bug-logs/submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/0001-marketplace-stamp-all-five-identity-fields-so-Studio.patch b/bug-logs/submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/0001-marketplace-stamp-all-five-identity-fields-so-Studio.patch new file mode 100644 index 0000000..ff41620 --- /dev/null +++ b/bug-logs/submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/0001-marketplace-stamp-all-five-identity-fields-so-Studio.patch @@ -0,0 +1,397 @@ +From 82808b5b7b499e08ab1c4185bdcd983621397717 Mon Sep 17 00:00:00 2001 +From: MendixMau +Date: Mon, 28 Sep 2026 05:18:57 +0000 +Subject: [PATCH] marketplace: stamp all five identity fields so Studio Pro + groups the module + +A module Studio Pro installs from the Marketplace carries five fields on +its Projects$ModuleImpl: FromAppStore, AppStoreVersion, AppStoreGuid, +AppStoreVersionGuid and AppStorePackageIdString. `marketplace install +` and `marketplace update` stamped only the first three, and +Studio Pro reads all of them: the module was listed among the app's own +modules in the App Explorer instead of under "Marketplace modules", and +nothing in mxcli's own output (SHOW MODULES reads FromAppStore only) said +so. + +Thread the content id from the resolved marketplace content through +installModule / installModuleFromFile / installByTransplant / +PerformInstall and through the update command into PerformUpdate, and +have StampMarketplaceVersion write the version UUID into both +AppStoreGuid and AppStoreVersionGuid and the content id into +AppStorePackageIdString via one helper, stampModuleDoc. The helper keeps +the ADR-0005 discipline: it only assigns keys the document already has. + +`--file` installs still record no identity (a package on disk has none); +that is a separate ask. + +Field: a fresh 11.13.0 project with a module installed via --file shows +FromAppStore=False and all four strings empty in the module unit; the +same project's Studio Pro-installed Encryption module shows all five set, +with AppStoreGuid == AppStoreVersionGuid and AppStorePackageIdString == +"1011". Unit tests cover the helper; the marketplace and cmd packages +pass. + +Co-Authored-By: Claude +Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw +--- + cmd/mxcli/cmd_marketplace_install.go | 23 ++--- + cmd/mxcli/cmd_marketplace_update.go | 5 +- + cmd/mxcli/marketplace/security_test.go | 6 +- + cmd/mxcli/marketplace/update.go | 38 +++++-- + cmd/mxcli/marketplace/update_stamp_test.go | 110 +++++++++++++++++++++ + 5 files changed, 158 insertions(+), 24 deletions(-) + create mode 100644 cmd/mxcli/marketplace/update_stamp_test.go + +diff --git a/cmd/mxcli/cmd_marketplace_install.go b/cmd/mxcli/cmd_marketplace_install.go +index c675e14fb..de4f36a14 100644 +--- a/cmd/mxcli/cmd_marketplace_install.go ++++ b/cmd/mxcli/cmd_marketplace_install.go +@@ -11,6 +11,7 @@ import ( + "os" + "os/exec" + "path/filepath" ++ "strconv" + "strings" + + modelsdk "github.com/mendixlabs/mxcli" +@@ -142,7 +143,7 @@ func runMarketplaceInstall(cmd *cobra.Command, args []string) error { + return err + } + } +- return installModule(cmd.Context(), client, version, mprPath, allowFormatChange, out) ++ return installModule(cmd.Context(), client, content.ContentID, version, mprPath, allowFormatChange, out) + default: + // Theme / Starter App / Sample / unknown: download + instruct rather + // than guess a placement we can't verify. +@@ -170,7 +171,7 @@ func installFromFile(ctx context.Context, mpkPath, mprPath string, allowFormatCh + } + switch { + case pkg.ModelerProject.Module.Name != "": +- return installModuleFromFile(ctx, mpkPath, mprPath, allowFormatChange, "", "", ++ return installModuleFromFile(ctx, mpkPath, mprPath, allowFormatChange, "", "", "", + "from "+filepath.Base(mpkPath), out) + case pkg.ClientModule.Name != "": + return placeWidgetFile(mpkPath, filepath.Dir(mprPath), out) +@@ -221,7 +222,7 @@ func printWidgetNext(out io.Writer) { + + // installModule imports a module .mpk into the project, but only when the module + // is not already present. An existing module is reported, not modified. +-func installModule(ctx context.Context, client *marketplace.Client, v *marketplace.Version, mprPath string, allowFormatChange bool, out io.Writer) error { ++func installModule(ctx context.Context, client *marketplace.Client, contentID int, v *marketplace.Version, mprPath string, allowFormatChange bool, out io.Writer) error { + // Download to a temp .mpk so we can inspect it and hand it to mx. + tmpDir, err := os.MkdirTemp("", "mxcli-install-") + if err != nil { +@@ -232,17 +233,17 @@ func installModule(ctx context.Context, client *marketplace.Client, v *marketpla + if err != nil { + return err + } +- return installModuleFromFile(ctx, mpkPath, mprPath, allowFormatChange, v.VersionNumber, v.VersionID, ++ return installModuleFromFile(ctx, mpkPath, mprPath, allowFormatChange, strconv.Itoa(contentID), v.VersionNumber, v.VersionID, + "version "+v.VersionNumber, out) + } + + // installModuleFromFile is the part of a module install that starts once the + // package is on disk — shared by the online path (after its download) and by +-// --file (which has no download). versionNumber and versionID are the +-// marketplace identity to record on the module; both are empty for a package +-// from disk, which has none. label is how the package is named in output. ++// --file (which has no download). contentID, versionNumber and versionID are ++// the marketplace identity to record on the module; all three are empty for a ++// package from disk, which has none. label is how the package is named in output. + func installModuleFromFile(ctx context.Context, mpkPath, mprPath string, allowFormatChange bool, +- versionNumber, versionID, label string, out io.Writer) error { ++ contentID, versionNumber, versionID, label string, out io.Writer) error { + + moduleName, err := moduleNameFromMpk(mpkPath) + if err != nil { +@@ -271,7 +272,7 @@ func installModuleFromFile(ctx context.Context, mpkPath, mprPath string, allowFo + // the project's storage format and works for theme modules. --allow-format-change + // selects the legacy `mx module-import`, which does neither. + if !allowFormatChange { +- res, ierr := installByTransplant(ctx, mpkPath, mprPath, moduleName, mendixVer, versionNumber, versionID) ++ res, ierr := installByTransplant(ctx, mpkPath, mprPath, moduleName, mendixVer, contentID, versionNumber, versionID) + if ierr != nil { + return ierr + } +@@ -310,7 +311,7 @@ func installModuleFromFile(ctx context.Context, mpkPath, mprPath string, allowFo + // installByTransplant builds a reference project from the package and copies the + // module out of it, so the destination keeps its MPR format. + func installByTransplant(ctx context.Context, mpkPath, mprPath, moduleName, mendixVer string, +- versionNumber, versionID string) (*mp.UpdateResult, error) { ++ contentID, versionNumber, versionID string) (*mp.UpdateResult, error) { + + work, err := os.MkdirTemp("", "mxinstall") + if err != nil { +@@ -326,7 +327,7 @@ func installByTransplant(ctx context.Context, mpkPath, mprPath, moduleName, mend + if err != nil { + return nil, fmt.Errorf("build a reference project from the package: %w", err) + } +- return mp.PerformInstall(mprPath, refMpr, mpkPath, moduleName, versionNumber, versionID, newBackendFactory()) ++ return mp.PerformInstall(mprPath, refMpr, mpkPath, moduleName, contentID, versionNumber, versionID, newBackendFactory()) + } + + // isMPRv2 reports whether the project at mprPath uses the MPR v2 storage format: +diff --git a/cmd/mxcli/cmd_marketplace_update.go b/cmd/mxcli/cmd_marketplace_update.go +index c96365cf5..19e38c9e4 100644 +--- a/cmd/mxcli/cmd_marketplace_update.go ++++ b/cmd/mxcli/cmd_marketplace_update.go +@@ -7,6 +7,7 @@ import ( + "io" + "os" + "path/filepath" ++ "strconv" + + "github.com/mendixlabs/mxcli/cmd/mxcli/marketplace" + "github.com/mendixlabs/mxcli/internal/auth" +@@ -170,8 +171,8 @@ func runMarketplaceUpdate(cmd *cobra.Command, args []string) error { + targetMpk := filepath.Join(work, "target.mpk") + + fmt.Fprintf(out, "\nUpdating %s %s → %s...\n", moduleName, installedVersion, target) +- res, err := marketplace.PerformUpdate(mprPath, targetRef, targetMpk, moduleName, installedVersion, target, +- targetVersion.VersionID, newBackendFactory()) ++ res, err := marketplace.PerformUpdate(mprPath, targetRef, targetMpk, moduleName, strconv.Itoa(contentID), ++ installedVersion, target, targetVersion.VersionID, newBackendFactory()) + if err != nil { + return fmt.Errorf("%w\n\nThe project may be mid-update — %s could be missing. Restore from version control", + err, moduleName) +diff --git a/cmd/mxcli/marketplace/security_test.go b/cmd/mxcli/marketplace/security_test.go +index 6f585582b..990cf537a 100644 +--- a/cmd/mxcli/marketplace/security_test.go ++++ b/cmd/mxcli/marketplace/security_test.go +@@ -61,7 +61,7 @@ func TestPerformUpdate_ReconcilesAnIncompleteAccessRule(t *testing.T) { + t.Fatalf("control: %s is still in the reference's rules %v — nothing was stripped", removed, got) + } + +- res, err := PerformUpdate(target, ref, emptyMpk(t), "Administration", ++ res, err := PerformUpdate(target, ref, emptyMpk(t), "Administration", "23513", + "4.3.2", "4.3.3", "00000000-0000-0000-0000-000000000000", eng.backend) + if err != nil { + t.Fatalf("PerformUpdate: %v", err) +@@ -90,7 +90,7 @@ func TestPerformUpdate_LeavesCompleteRulesAlone(t *testing.T) { + ref := copyFixture(t) + + before := memberRefs(t, ref, "Administration", "Account") +- res, err := PerformUpdate(target, ref, emptyMpk(t), "Administration", ++ res, err := PerformUpdate(target, ref, emptyMpk(t), "Administration", "23513", + "4.3.2", "4.3.3", "00000000-0000-0000-0000-000000000000", testBackend) + if err != nil { + t.Fatalf("PerformUpdate: %v", err) +@@ -113,7 +113,7 @@ func TestPerformInstall_ReconcilesAnIncompleteAccessRule(t *testing.T) { + removed := stripLastMemberAccess(t, ref, "Administration", "Account") + execMDL(t, target, "drop module Administration;") + +- res, err := PerformInstall(target, ref, emptyMpk(t), "Administration", ++ res, err := PerformInstall(target, ref, emptyMpk(t), "Administration", "23513", + "4.3.2", "00000000-0000-0000-0000-000000000000", testBackend) + if err != nil { + t.Fatalf("PerformInstall: %v", err) +diff --git a/cmd/mxcli/marketplace/update.go b/cmd/mxcli/marketplace/update.go +index 41f7131fe..47403f93c 100644 +--- a/cmd/mxcli/marketplace/update.go ++++ b/cmd/mxcli/marketplace/update.go +@@ -148,7 +148,7 @@ type SkippedFile struct { + // + // A failure partway leaves the project in a broken state. The caller must work + // on a copy, or hold a backup: this does not roll back. +-func PerformUpdate(mprPath, referenceMpr, targetMpk, moduleName, fromVersion, toVersion, toVersionID string, ++func PerformUpdate(mprPath, referenceMpr, targetMpk, moduleName, contentID, fromVersion, toVersion, toVersionID string, + newBackend func() backend.FullBackend) (*UpdateResult, error) { + + ids, err := CaptureIdentities(mprPath, moduleName) +@@ -177,7 +177,7 @@ func PerformUpdate(mprPath, referenceMpr, targetMpk, moduleName, fromVersion, to + if err != nil { + return nil, fmt.Errorf("restore role grants: %w", err) + } +- if err := StampMarketplaceVersion(mprPath, moduleName, toVersion, toVersionID); err != nil { ++ if err := StampMarketplaceVersion(mprPath, moduleName, contentID, toVersion, toVersionID); err != nil { + return nil, fmt.Errorf("record the installed version: %w", err) + } + files, skippedFiles, err := InstallPackageFiles(targetMpk, filepath.Dir(mprPath)) +@@ -220,7 +220,16 @@ func PerformUpdate(mprPath, referenceMpr, targetMpk, moduleName, fromVersion, to + // module by its AppStoreGuid — the marketplace *version* UUID — so a wrong stamp + // makes the module unrecognisable to the next update, which then reports that no + // module in the project came from this content. +-func StampMarketplaceVersion(mprPath, moduleName, versionNumber, versionID string) error { ++// ++// Studio Pro reads the same record, and it reads all of it. A module it ++// installs itself carries five fields: FromAppStore, AppStoreVersion, the ++// version UUID in both AppStoreGuid and AppStoreVersionGuid, and the content id ++// in AppStorePackageIdString. A module stamped with only the first three is ++// listed among the app's own modules in the App Explorer instead of under ++// "Marketplace modules", so the stamp writes all five. contentID is the ++// marketplace content id as decimal text ("1011" for Encryption); empty when ++// the caller has none, in which case that one field is left as it was. ++func StampMarketplaceVersion(mprPath, moduleName, contentID, versionNumber, versionID string) error { + reader, err := modelsdk.Open(mprPath) + if err != nil { + return fmt.Errorf("open %s: %w", mprPath, err) +@@ -242,9 +251,7 @@ func StampMarketplaceVersion(mprPath, moduleName, versionNumber, versionID strin + if !strings.EqualFold(name, moduleName) { + continue + } +- setStringField(doc, "AppStoreVersion", versionNumber) +- setStringField(doc, "AppStoreGuid", versionID) +- setBoolField(doc, "FromAppStore", true) ++ stampModuleDoc(doc, contentID, versionNumber, versionID) + enc, merr := bson.Marshal(doc) + if merr != nil { + reader.Disconnect() +@@ -266,6 +273,21 @@ func StampMarketplaceVersion(mprPath, moduleName, versionNumber, versionID strin + return writer.UpdateRawUnit(unitID, contents) + } + ++// stampModuleDoc writes the marketplace identity onto a Projects$ModuleImpl ++// document the way Studio Pro's own install leaves it: the version UUID goes ++// into AppStoreGuid and AppStoreVersionGuid alike, and the content id into ++// AppStorePackageIdString. It only assigns keys the document already has (see ++// setStringField), so a metamodel version without one of them is left intact. ++func stampModuleDoc(doc bson.D, contentID, versionNumber, versionID string) { ++ setStringField(doc, "AppStoreVersion", versionNumber) ++ setStringField(doc, "AppStoreGuid", versionID) ++ setStringField(doc, "AppStoreVersionGuid", versionID) ++ if contentID != "" { ++ setStringField(doc, "AppStorePackageIdString", contentID) ++ } ++ setBoolField(doc, "FromAppStore", true) ++} ++ + // setStringField assigns an existing key, and only an existing key. Inventing a + // property Mendix does not store for this version is the failure mode ADR-0005 + // warns about: mxbuild tolerates it and Studio Pro refuses to open the document. +@@ -425,7 +447,7 @@ func duplicateOfPackagedWidget(name string, twins map[string]string) (string, bo + // `mx module-import` is what makes it worth having — it preserves the project's + // storage format, where module-import rewrites MPR v2 as v1, and it works for + // theme modules, which module-import refuses outright. +-func PerformInstall(mprPath, referenceMpr, packageMpk, moduleName, version, versionID string, ++func PerformInstall(mprPath, referenceMpr, packageMpk, moduleName, contentID, version, versionID string, + newBackend func() backend.FullBackend) (*UpdateResult, error) { + + copied, err := TransplantModule(referenceMpr, mprPath, moduleName) +@@ -438,7 +460,7 @@ func PerformInstall(mprPath, referenceMpr, packageMpk, moduleName, version, vers + // none; leaving the module unstamped is the truthful record, and both then + // report — correctly — that no marketplace content is installed under it. + if versionID != "" { +- if err := StampMarketplaceVersion(mprPath, moduleName, version, versionID); err != nil { ++ if err := StampMarketplaceVersion(mprPath, moduleName, contentID, version, versionID); err != nil { + return nil, fmt.Errorf("record the installed version: %w", err) + } + } +diff --git a/cmd/mxcli/marketplace/update_stamp_test.go b/cmd/mxcli/marketplace/update_stamp_test.go +new file mode 100644 +index 000000000..00f38a02e +--- /dev/null ++++ b/cmd/mxcli/marketplace/update_stamp_test.go +@@ -0,0 +1,110 @@ ++// SPDX-License-Identifier: Apache-2.0 ++ ++package marketplace ++ ++import ( ++ "testing" ++ ++ "go.mongodb.org/mongo-driver/bson" ++) ++ ++func lookupString(t *testing.T, doc bson.D, key string) string { ++ t.Helper() ++ for _, e := range doc { ++ if e.Key == key { ++ s, ok := e.Value.(string) ++ if !ok { ++ t.Fatalf("%s: expected string, got %T", key, e.Value) ++ } ++ return s ++ } ++ } ++ t.Fatalf("%s: key absent", key) ++ return "" ++} ++ ++func docHasKey(doc bson.D, key string) bool { ++ for _, e := range doc { ++ if e.Key == key { ++ return true ++ } ++ } ++ return false ++} ++ ++// A Projects$ModuleImpl as Studio Pro writes it carries all five marketplace ++// identity fields. The stamp must fill every one of them the way Studio Pro's ++// own install does: the version UUID in AppStoreGuid AND AppStoreVersionGuid, ++// the content id in AppStorePackageIdString. A module with only three of the ++// five set is listed among the app's own modules in the App Explorer instead ++// of under "Marketplace modules". ++func TestStampModuleDoc_SetsAllFiveIdentityFields(t *testing.T) { ++ doc := bson.D{ ++ {Key: "$Type", Value: "Projects$ModuleImpl"}, ++ {Key: "Name", Value: "Encryption"}, ++ {Key: "FromAppStore", Value: false}, ++ {Key: "AppStoreVersion", Value: ""}, ++ {Key: "AppStoreGuid", Value: ""}, ++ {Key: "AppStoreVersionGuid", Value: ""}, ++ {Key: "AppStorePackageIdString", Value: ""}, ++ } ++ stampModuleDoc(doc, "1011", "11.1.1", "0d8f1c4e-2b5a-4f3c-9a7e-6c1d2e3f4a5b") ++ ++ if got := lookupString(t, doc, "AppStoreVersion"); got != "11.1.1" { ++ t.Errorf("AppStoreVersion = %q", got) ++ } ++ if got := lookupString(t, doc, "AppStoreGuid"); got != "0d8f1c4e-2b5a-4f3c-9a7e-6c1d2e3f4a5b" { ++ t.Errorf("AppStoreGuid = %q", got) ++ } ++ if got := lookupString(t, doc, "AppStoreVersionGuid"); got != "0d8f1c4e-2b5a-4f3c-9a7e-6c1d2e3f4a5b" { ++ t.Errorf("AppStoreVersionGuid = %q", got) ++ } ++ if got := lookupString(t, doc, "AppStorePackageIdString"); got != "1011" { ++ t.Errorf("AppStorePackageIdString = %q", got) ++ } ++ var from bool ++ for _, e := range doc { ++ if e.Key == "FromAppStore" { ++ from, _ = e.Value.(bool) ++ } ++ } ++ if !from { ++ t.Errorf("FromAppStore not set") ++ } ++} ++ ++// An unknown content id (a package installed from disk, or a caller that has ++// none) leaves AppStorePackageIdString as it was rather than blanking it. ++func TestStampModuleDoc_EmptyContentIDLeavesPackageId(t *testing.T) { ++ doc := bson.D{ ++ {Key: "FromAppStore", Value: false}, ++ {Key: "AppStoreVersion", Value: ""}, ++ {Key: "AppStoreGuid", Value: ""}, ++ {Key: "AppStoreVersionGuid", Value: ""}, ++ {Key: "AppStorePackageIdString", Value: "1011"}, ++ } ++ stampModuleDoc(doc, "", "11.2.0", "uuid-2") ++ if got := lookupString(t, doc, "AppStorePackageIdString"); got != "1011" { ++ t.Errorf("AppStorePackageIdString overwritten: %q", got) ++ } ++ if got := lookupString(t, doc, "AppStoreVersionGuid"); got != "uuid-2" { ++ t.Errorf("AppStoreVersionGuid = %q", got) ++ } ++} ++ ++// setStringField only assigns keys the document already has (ADR-0005), so a ++// metamodel version that lacks one of the five is left intact: no key is added. ++func TestStampModuleDoc_DoesNotAddAbsentKeys(t *testing.T) { ++ doc := bson.D{ ++ {Key: "FromAppStore", Value: false}, ++ {Key: "AppStoreVersion", Value: ""}, ++ {Key: "AppStoreGuid", Value: ""}, ++ } ++ stampModuleDoc(doc, "1011", "11.1.1", "uuid-1") ++ if docHasKey(doc, "AppStoreVersionGuid") || docHasKey(doc, "AppStorePackageIdString") { ++ t.Errorf("stamp added keys the document did not have: %v", doc) ++ } ++ if got := lookupString(t, doc, "AppStoreGuid"); got != "uuid-1" { ++ t.Errorf("AppStoreGuid = %q", got) ++ } ++} +-- +2.43.0 + diff --git a/bug-logs/submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/README.md b/bug-logs/submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/README.md new file mode 100644 index 0000000..b8182a5 --- /dev/null +++ b/bug-logs/submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/README.md @@ -0,0 +1,64 @@ +Status: READY TO SEND, NOT SENT (go test ./... 84 packages ok, 0 FAIL; gofmt/vet clean; on 95091765) · issue NOT YET FILED — draft in `bug-logs/pending-github-issues/marketplace-install-not-grouped-in-studio-pro.md`; file it, then open the PR with "closes #" · base upstream main 95091765 · field proof: partial (see below) + +# mxcli PR: `marketplace install` / `update` stamp all five marketplace identity fields + +Per `skills/upstream-feedback.md` §3a. Not yet sent. The patch is +`0001-marketplace-stamp-all-five-identity-fields-so-Studio.patch` (5 files, +109/-24 incl. a new test file): +`cmd/mxcli/marketplace/update.go` (the `stampModuleDoc` helper and the content id threaded into +`StampMarketplaceVersion` / `PerformInstall` / `PerformUpdate`), the two cobra commands that now pass the +resolved content id down, the existing security test's call sites, and `update_stamp_test.go` (3 tests). + +## How to send it + +```bash +git clone https://github.com//mxcli && cd mxcli +git fetch https://github.com/mendixlabs/mxcli main && git checkout -b fix/marketplace-install-fromappstore FETCH_HEAD +git am /path/to/0001-marketplace-stamp-all-five-identity-fields-so-Studio.patch +make test && make lint # re-run on the day; main moves +git push -u origin fix/marketplace-install-fromappstore +``` + +File the issue from the draft first, then open the PR with the body below and `closes #`. + +## What is proven, what is not + +- **Cause:** read from source on `main` 95091765. `StampMarketplaceVersion` wrote `FromAppStore`, + `AppStoreVersion`, `AppStoreGuid` and nothing else; the content id never reached it. +- **Reference values:** a Studio Pro-installed module (Encryption 11.1.1) in a real project has all five + fields set, `AppStoreGuid == AppStoreVersionGuid`, `AppStorePackageIdString == "1011"`. A `--file` + install on a fresh 11.13.0 probe has `FromAppStore` false and four empty strings. Both read by + decoding the module's `.mxunit` BSON. +- **Unit tests:** `TestStampModuleDoc_*` — all five set when the keys exist; empty content id leaves + `AppStorePackageIdString` alone; absent keys are not invented (ADR-0005). `go test` on + `cmd/mxcli/marketplace` and `cmd/mxcli` passes; full suite result is on the status line. +- **NOT proven:** a content-id install with the patched binary followed by a Studio Pro open showing the + module under "Marketplace modules". The content-id path needs a Mendix PAT and Studio Pro needs a Mac + or Windows machine, neither of which the drafting session had. **Before sending:** build the branch, + `mxcli marketplace install 1011 -p .mpr`, decode the module unit (or `SHOW MODULES` plus a + BSON dump) and confirm the five fields, then open it in Studio Pro and confirm the grouping. Put the + Mendix version and the result into the PR body's "Field" paragraph. + +## PR body (to send) + +**Closes #.** + +A module Studio Pro installs from the Marketplace carries five fields on its `Projects$ModuleImpl`: +`FromAppStore`, `AppStoreVersion`, `AppStoreGuid`, `AppStoreVersionGuid` and `AppStorePackageIdString`. +`marketplace install ` and `marketplace update` stamped only the first three, and Studio Pro +reads all of them: the module was listed among the app's own modules in the App Explorer instead of under +"Marketplace modules". `SHOW MODULES` reads `FromAppStore` only, so mxcli's own output could not show it. + +This threads the resolved content id from the marketplace lookup through `installModule` → +`installModuleFromFile` → `installByTransplant` → `PerformInstall`, and from the update command into +`PerformUpdate`, and has `StampMarketplaceVersion` write the version UUID into both `AppStoreGuid` and +`AppStoreVersionGuid` and the content id into `AppStorePackageIdString`, through one helper +(`stampModuleDoc`). The helper keeps the ADR-0005 rule: it only assigns keys the document already has. + +`--file` installs still record no identity (a package on disk has none); a `--content-id`/`--version` +pair for `--file` is a separate ask on the issue. + +**Tests:** three unit tests on the helper; the `security_test.go` call sites updated. `CGO_ENABLED=0 go test ./...` on +`main` 95091765: 84 packages ok, 0 FAIL; gofmt and go vet clean. + +**Field:** +project, the five fields read from the unit, Studio Pro shows Encryption under "Marketplace modules">. diff --git a/bug-logs/upstream-log-2026-09.md b/bug-logs/upstream-log-2026-09.md index b708fe8..ccec82a 100644 --- a/bug-logs/upstream-log-2026-09.md +++ b/bug-logs/upstream-log-2026-09.md @@ -32,7 +32,10 @@ This is the index of what we sent upstream or have ready to send. |---|---|---| | `submitted-prs/mxcli/2026-09-27-file-uploader-nested-visibility/` | #1198 | OPENED as [mendixlabs/mxcli#1227](https://github.com/mendixlabs/mxcli/pull/1227) on 2026-09-28; linked from #1198 | | `submitted-prs/mxcli/2026-09-28-alter-page-set-rendermode/` | #1228 | OPENED as [mendixlabs/mxcli#1229](https://github.com/mendixlabs/mxcli/pull/1229) on 2026-09-28; linked from #1228. Rebased from fork-only MendixMau/mxcli#1 (now closed) | +| `submitted-prs/mxcli/2026-09-28-marketplace-install-fromappstore/` | (issue not yet filed: draft `pending-github-issues/marketplace-install-not-grouped-in-studio-pro.md`) | READY, not sent. On main 95091765; unit tests + marketplace/cmd packages pass; Studio Pro regrouping not yet proven on a real model | ## Issue drafts The seven 2026-09-27 drafts in `pending-github-issues/` were all filed on 2026-09-27: four as #1223–#1226 and three as comments (tables above). Each file's status line carries its link and how the filed text differs from the draft. Three changed on retest against main 95091765: `create-or-modify-assoc-ignores-owner` narrowed to cross-module only, `partial-revoke-member-noop` changed symptom (false "No access rules found", exit 0), and `xpath-system-member-case-ce0161` widened to all bare XPath member names. + +One new draft on 2026-09-28, NOT YET FILED: `marketplace-install-not-grouped-in-studio-pro` (3 of 5 marketplace identity fields stamped; `--file` stamps none). Its fix PR package is in the table above and waits on the issue number. diff --git a/skills/learned-mdl-preflight.md b/skills/learned-mdl-preflight.md index a908f98..a217418 100644 --- a/skills/learned-mdl-preflight.md +++ b/skills/learned-mdl-preflight.md @@ -70,6 +70,7 @@ Once you've picked a mode per operation, run the STOP table below against every | 24 | Put a `DesignProperties: [...]` entry on a widget (Form orientation, Label width, Spacing, …) in an app scaffolded by `mxcli new` (any `--theme`) | **STOP → grep the theme first: `grep -l '' themesource/*/web/design-properties.json`. If it is not there, use the Atlas class instead (`Class: 'form-vertical'`, `spacing-outer-bottom-large`).** `mxcli new`'s theme layer ships a `design-properties.json` that does not carry Atlas's FormBase entries, so mxbuild rejects the property with CE6083 "not supported by your theme" — after `check --references` passed it. `design-spacing.md` says design properties and classes write the identical model; that holds only when the theme declares the property. | Greenfield pilot 2026-09-04, mxcli v0.20.0, Mendix 11.13.0, signal theme: `DesignProperties: ['Form orientation': 'Vertical']` on a DataView → exec ok, CE6083 at the gate, auto-restored. `Class: 'form-vertical'` built clean — and then changed nothing on screen, because in that theme the form layout is decided by the theme's own dataview rules, not the class (ui-loop.md "which side is wrong", row 1). The look caught it; no gate could. | | 25 | Write a microflow with any loop, >20 planned activities, or a list built from a list | **STOP → run `microflow-preflight.md` and post its checklist first.** | Lint runs only after exec and counts only top-level activities (a loop body is invisible to CONV009/QUAL003); CONV011 commit-in-loop and MPR008 overlap findings came back on field builds after the write, not before. | | 26 | Bind a DataGrid 2 column, or a grid `sort by`, to an attribute **over an association** | **STOP → write the column path unquoted, and do not sort over an association.** Use `Attribute: Assoc_A_B/Name`, never `"Assoc_A_B/Name"`. On ≤ v0.24.0 sort by a local attribute or use a microflow datasource. This is the one place a quote-every-identifier convention must break. | A quoted path is stored as one attribute named `Assoc_A_B/Name`, and `sort by Assoc/Attr` is written as a plain attribute of the grid entity. Both pass `check --references` (v0.24.0, re-verified 2026-09-25) and give CE1613 at build (v0.23.0 field run). `BUG-DRAFT-dg2-association-path-bindings` | +| 27 | Install or update a marketplace module (`marketplace install`, `marketplace update`) into a project someone will open in Studio Pro | **STOP → install by content id, never `--file`, for a module that must read as a Marketplace module; then read the Source column of `SHOW MODULES` — it must say `Marketplace v`. On ≤ v0.24.0 Studio Pro still lists the module among the app's own modules (3 of 5 identity fields written); say so in the register, or apply the fix package.** | `--file` stamps none of the five marketplace identity fields (`FromAppStore` stays false — no Source shown); a content-id install stamps three, and Studio Pro reads all five, so the module lands outside "Marketplace modules". Measured 2026-09-28 on a fresh 11.13.0 probe and a real project's Studio Pro-installed module. `BUG-DRAFT-marketplace-install-not-grouped-in-studio-pro` | **Default to mxcli for:** entities/attributes/enums, associations (after SHOW ASSOCIATIONS check), microflows (without inline assoc-sets), demo users, module roles/grants, navigation.