From 1ae6f4eca4bd7cbe83d17d5006d026974eaf8ba7 Mon Sep 17 00:00:00 2001 From: Ako Date: Fri, 4 Sep 2026 09:10:30 +0000 Subject: [PATCH] feat(brain): open questions, and a trigger for capturing decisions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two gaps, both surfaced by comparing the design against Anthropic's AI-native SDLC playbook. OPEN QUESTIONS. The store recorded what had been decided and what was going to be built, but not what was still undecided. A known unknown is brain-shaped by the store's own test — not derivable from the model, lost when the conversation ends, and expensive to rediscover — so it had nowhere to go and was simply forgotten. A question is a decision that has not been made yet, and it needs its own treatment for one reason: its anchors must NOT be checked. It routinely names something that does not exist, because the question is often precisely whether it should. Measured, with the identical anchor: exit 1 as a decision, exit 0 as a question. That is the same property that separates requirements from decisions — what a failed anchor MEANS — so this is a third point on an axis the store already had, not a new mechanism. Questions live beside the decisions they will join rather than in a file of their own, because the moment you need to see one is while reading what that module already decided. The OPEN marker therefore travels in the entry rather than being implied by the file, as a requirement's kind is. That is still one copy of the fact: what the store forbids is two. `brain resolve` converts a question into a decision IN PLACE, keeping its id and its position — an answered question is the same piece of knowledge as the question — with the question retained as the answer's context. From that moment its anchors are checked like any other decision, which is the transition the kind exists for and is asserted rather than assumed. A question filed against a slice is counted apart from its requirements: an unanswered question is not outstanding scope, and counting it as such would overstate what is left to do. A CAPTURE TRIGGER. Capture had no doctrine for WHEN, which left the two halves lopsided: the plan fills at bootstrap, while decisions fill only if someone remembers. The skill now names the trigger the playbook uses for CLAUDE.md — a correction you have had to make twice, because the second one predicts a third for someone else — plus a choice between real alternatives where the losing one would look reasonable later. `mxcli lint` now reports unanswered questions alongside unpromoted entries. Same reasoning as before: a question nobody answers is the one kind of entry that gets more expensive the longer it sits, and a report only `brain check` prints is a report nothing demands. Controls in both directions throughout, including that `--open` does not quietly disable the check for everything, that an entry written before the marker existed still reads as a settled decision, and that a resolved question stays where it was rather than moving to the end of the file. Co-Authored-By: Claude Opus 5 --- .claude/skills/mendix/project-brain/SKILL.md | 56 +++++- CLAUDE.md | 4 +- cmd/mxcli/brain/brain_test.go | 193 +++++++++++++++++++ cmd/mxcli/brain/check.go | 31 ++- cmd/mxcli/brain/entry.go | 58 +++++- cmd/mxcli/brain/shard.go | 9 +- cmd/mxcli/brain/store.go | 64 ++++++ cmd/mxcli/cmd_brain.go | 62 +++++- cmd/mxcli/cmd_lint.go | 33 +++- docs-site/src/tools/project-brain.md | 49 +++++ 10 files changed, 541 insertions(+), 18 deletions(-) diff --git a/.claude/skills/mendix/project-brain/SKILL.md b/.claude/skills/mendix/project-brain/SKILL.md index 132bbeda91..a41a997740 100644 --- a/.claude/skills/mendix/project-brain/SKILL.md +++ b/.claude/skills/mendix/project-brain/SKILL.md @@ -1,6 +1,6 @@ --- name: project-brain -description: "Project-specific knowledge mxcli cannot compute — the requirements and slices being built from (a spec, a prototype, a conversation), why a pattern was chosen here, which marketplace version broke what. Use when starting from requirements that live outside git, before designing something that looks like it was decided before, and when an mxbuild error is resolved by something non-obvious." +description: "Project-specific knowledge mxcli cannot compute — the requirements and slices being built from (a spec, a prototype, a conversation), why a pattern was chosen here, what is still undecided, which marketplace version broke what. Use when starting from requirements that live outside git, before designing something that looks like it was decided before, when you have had to correct the same thing twice, and when an mxbuild error is resolved by something non-obvious." --- # Project brain @@ -9,6 +9,8 @@ The brain holds what mxcli **cannot** compute about this project. Two halves: - **Decisions** — why a pattern was chosen here, which marketplace version broke what, what a recurring mxbuild error means in *this* app. +- **Open questions** — what is *not* decided yet, so it is not silently + forgotten and rediscovered expensively later. - **The plan** — the requirements being built from and the slices they are grouped into, when the source is a specification document, a prototype or a conversation rather than GitHub issues. @@ -143,6 +145,56 @@ A slice holds source material, so its budget is much larger than a decision shard's — and it is not loaded every session. But it is still a budget: **a slice too long to read is a slice that should be split.** +## When to capture a decision + +Capture is easy to postpone forever, so it needs a trigger rather than good +intentions. Two, and the first is the reliable one: + +1. **You have had to correct the same thing twice.** The second correction is + the signal: it will happen a third time to someone else. Capture what the + right answer is and why, anchored at whatever you were working on. +2. **You chose between real alternatives** and the losing one would look + reasonable to the next person. Record the choice *and* what ruled the other + out — a decision without its reason gets re-litigated. + +If you are unsure whether something qualifies, capture it. Staging costs +nothing and is reversible; a person decides what is worth committing. + +## Recording what is NOT decided + +An open question is a decision that has not been made yet. Record it rather +than carrying it in your head — the conversation ends, and the question is +expensive to rediscover. + +```bash +mxcli brain capture "Do approvers see rejected orders? +The spec is silent. Affects the overview page and the access rules." \ + --open -a @Sales.Order -p app.mpr +``` + +A question's **anchors are not checked**. It may name something that does not +exist — often the question is precisely whether it should — so the staleness +rule that keeps decisions honest does not apply to it. + +`--open` combines with `--slice`: a question about a slice's scope is filed with +that slice, and is counted apart from its requirements. An unanswered question +is not outstanding scope, so it never inflates the slice. + +Answering it turns it into a decision, in place: + +```bash +mxcli brain resolve "Yes, for 30 days +Agreed with the product owner; drives the overview filter and the access rule." +``` + +The entry keeps its id and its position, and the question survives as the +answer's context. From that moment its anchors **are** checked, like any other +decision. + +`mxcli brain check` and `mxcli lint` both report unanswered questions until +someone resolves one. That is deliberate: a question nobody answers is the one +kind of entry that gets more expensive the longer it sits. + ## Write the anchor, not the name `@Sales.Order.Status` is what makes an entry **routable** (its module decides @@ -211,6 +263,8 @@ cap: the cap is what stops the store becoming a file nobody reads. | `mxcli brain promote [--to ]` | Writes it into its shard. The human step | | `mxcli brain drop ` | Removes it from the queue or from its shard | | `mxcli brain capture "" --slice [-a @Anchor]…` | Queues a **requirement** of that slice | +| `mxcli brain capture "" --open [-a @Anchor]…` | Queues an **open question**; its anchors are not checked | +| `mxcli brain resolve ""` | Answers a question, turning it into a decision in place | | `mxcli brain plan` | The roadmap: each slice's requirements counted against the model | | `mxcli brain check [--changed]` | Anchors still resolve, entries in the right shard, plus slice progress | | `mxcli brain show []` | Entries, lines and headroom per shard | diff --git a/CLAUDE.md b/CLAUDE.md index 9384aa2e80..af2151e101 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -717,7 +717,7 @@ go build -o bin/mxcli ./cmd/mxcli | **Marketplace drift** | `mxcli marketplace diff -p app.mpr [--to V] [--json]` | Which elements of an installed marketplace module have been edited locally, and what an upgrade would overwrite | | **Model repair** | `mxcli fix widgets`, `mxcli fix design-properties` | Runs `mx update-widgets` / `mx rename-design-properties` and **persists** the result without their MPR v2 → v1 collapse (harvest: let the tool convert, read the units back, restore v2, write the changed ones through mxcli's writer). Clears CE0463 / CE6087 after a headless install — measured 203 → 0 errors on a vanilla 11.12.1 app | | **Diagnostics** | `mxcli diag [--bundle]` | Session logs, version info, bug report bundles | -| **Project brain** | `mxcli brain init\|capture\|staged\|promote\|drop\|check\|show\|plan` | Opt-in store in `docs/brain/` for what mxcli **cannot** compute (why a pattern was chosen here, which marketplace version broke what). Sharded by module — an entry's first anchor names its file — so a session loads `project.md` plus the modules it is touching, not the whole store. Also holds the **plan**: requirements grouped into slices, whose anchors point *forward*, so `brain plan` reports progress **derived from the model** rather than from a status column. An agent captures to a git-ignored queue; a person promotes | +| **Project brain** | `mxcli brain init\|capture\|staged\|promote\|drop\|check\|show\|plan\|resolve` | Opt-in store in `docs/brain/` for what mxcli **cannot** compute (why a pattern was chosen here, which marketplace version broke what). Sharded by module — an entry's first anchor names its file — so a session loads `project.md` plus the modules it is touching, not the whole store. Also holds the **plan**: requirements grouped into slices, whose anchors point *forward*, so `brain plan` reports progress **derived from the model** rather than from a status column. An agent captures to a git-ignored queue; a person promotes | | **New project** | `mxcli new --version X.Y.Z [--output-dir dir] [--theme none] [--layout none]` | Downloads mxbuild, creates blank project, applies default styling, scaffolds a project-owned layout, runs init, installs Linux mxcli for devcontainer | | **Default styling** | `mxcli theme list\|show\|apply\|remove` | Applies a theme (signal/ledger/console) — files under `theme/` only, the model is never touched | | **Project themes** | `mxcli theme create [--from ]` | Scaffolds a theme the project owns into `theme/mxcli-themes/`; `--from ` seeds the palette from `--mxt-*` declarations | @@ -814,7 +814,7 @@ Full syntax tables for all MDL statements (microflows, pages, security, navigati ## Current Implementation Status **Implemented:** -- Project brain (`mxcli brain init/capture/staged/promote/drop/check/show`): an **opt-in** store in `docs/brain/` for the project knowledge mxcli cannot compute. The governing rule is that anything derivable from the model is answered by a command and never written down — a note that transcribes the model disagrees with it silently. Records shard by **anchor scope**: an entry's first anchor names its file (`@Sales.Order` → `modules/Sales.md`), an anchorless entry is cross-cutting (`project.md`), and there is no index to maintain because the module prefix *is* the file name. That is what makes the cap per-shard rather than a project-wide budget, and lets a session load `project.md` plus the modules it is touching. `check` answers two independent questions: each anchor is **resolved / not found / not indexable** — only the middle one fails, and the third exists because the catalog's `objects` view covers the describable types only, so a scheduled event would otherwise read as *missing* (separated with `FindDocumentUnit`, which cannot miss a kind because it never asks what kind anything is). Misfiling is a **second axis, not a fourth state**: every anchor can resolve and the entry still be in the wrong file, and it is only decided when something resolved — judging it on an all-not-indexable entry reintroduced the same false staleness through the other axis (caught by a test, with the guard stubbed as the control). An agent `capture`s to a git-ignored queue and a person `promote`s; the queue is deliberately **not** sharded, because routing it would force the file decision before a human has looked at the entry. `mxcli lint` prints the unpromoted-queue count, because a report only `brain check` prints is a report nothing demands. Sizes are computed by `brain show` and never written into a committed file. A second record kind, **requirement**, lives in `plan/.md` and inverts the anchor's meaning: a decision's anchor points backward (not resolving = stale, fails), a requirement's points forward (not resolving = not built yet, passes). Measured: filed as an ordinary entry, one unbuilt requirement takes `brain check` to exit 1 — which is why it is a separate kind rather than more entries in the same files. That inversion is also what makes `brain plan` a real progress report: a requirement is *built* when its anchors resolve, so creating the microflow it names moves the count with the plan file untouched (measured 0/1 → 1/0). A status written beside a requirement is therefore refused by the skill, not just discouraged. Slices are ordered by name (`01-accounts`), span modules by design (so misfiling does not apply), and carry a generous cap that enforces the slicing discipline — a slice too long to read should be split. `bootstrap-app` asks for requirements at the interview and records them by default. Package: `cmd/mxcli/brain/`. See `docs-site/src/tools/project-brain.md` and `docs/11-proposals/PROPOSAL_project_brain.md` +- Project brain (`mxcli brain init/capture/staged/promote/drop/check/show`): an **opt-in** store in `docs/brain/` for the project knowledge mxcli cannot compute. The governing rule is that anything derivable from the model is answered by a command and never written down — a note that transcribes the model disagrees with it silently. Records shard by **anchor scope**: an entry's first anchor names its file (`@Sales.Order` → `modules/Sales.md`), an anchorless entry is cross-cutting (`project.md`), and there is no index to maintain because the module prefix *is* the file name. That is what makes the cap per-shard rather than a project-wide budget, and lets a session load `project.md` plus the modules it is touching. `check` answers two independent questions: each anchor is **resolved / not found / not indexable** — only the middle one fails, and the third exists because the catalog's `objects` view covers the describable types only, so a scheduled event would otherwise read as *missing* (separated with `FindDocumentUnit`, which cannot miss a kind because it never asks what kind anything is). Misfiling is a **second axis, not a fourth state**: every anchor can resolve and the entry still be in the wrong file, and it is only decided when something resolved — judging it on an all-not-indexable entry reintroduced the same false staleness through the other axis (caught by a test, with the guard stubbed as the control). An agent `capture`s to a git-ignored queue and a person `promote`s; the queue is deliberately **not** sharded, because routing it would force the file decision before a human has looked at the entry. `mxcli lint` prints the unpromoted-queue count, because a report only `brain check` prints is a report nothing demands. Sizes are computed by `brain show` and never written into a committed file. A second record kind, **requirement**, lives in `plan/.md` and inverts the anchor's meaning: a decision's anchor points backward (not resolving = stale, fails), a requirement's points forward (not resolving = not built yet, passes). Measured: filed as an ordinary entry, one unbuilt requirement takes `brain check` to exit 1 — which is why it is a separate kind rather than more entries in the same files. That inversion is also what makes `brain plan` a real progress report: a requirement is *built* when its anchors resolve, so creating the microflow it names moves the count with the plan file untouched (measured 0/1 → 1/0). A status written beside a requirement is therefore refused by the skill, not just discouraged. Slices are ordered by name (`01-accounts`), span modules by design (so misfiling does not apply), and carry a generous cap that enforces the slicing discipline — a slice too long to read should be split. A third kind, **open question** (`--open`), records what is *not* decided; its anchors are deliberately **not** checked, since the question is often whether the thing should exist at all — measured, the identical anchor exits 1 as a decision and 0 as a question. `brain resolve` converts one into a decision in place, keeping its id and position and starting to check its anchors, which is the transition the kind exists for. Unanswered questions are reported by `brain check` and by `mxcli lint`. The skill also gives capture a **trigger** rather than good intentions — a correction you have had to make twice — because the decisions half otherwise under-fills while the plan half fills at bootstrap. `bootstrap-app` asks for requirements at the interview and records them by default. Package: `cmd/mxcli/brain/`. See `docs-site/src/tools/project-brain.md` and `docs/11-proposals/PROPOSAL_project_brain.md` - Default styling + runtime theme switching (`mxcli theme list/show/create/apply/remove/switcher`, `mxcli new --theme`): three embedded themes (**signal** light-first, **ledger** light-first, **console** dark-first), each a palette in `theme/web/custom-variables.scss` + a shared Atlas wiring partial + a theme partial imported from `theme/web/main.scss` (which compiles last), plus vendored fonts. **No model changes**, so it hot-applies under `run --local --watch` and cannot affect a build. Generated regions are digest-fenced: a block carrying local edits is refused rather than overwritten. Applying a theme removes the previous one. `--variant auto` (default) ships both palettes — the app follows `prefers-color-scheme` before first paint and honours a `theme-light`/`theme-dark` class on ``; `light`/`dark` bakes one. `theme switcher install` is the only part that writes to the model (JS actions + a nanoflow for a toggle button). A project can add its own themes under `theme/mxcli-themes//` (committed, not compiled); `theme create [--from ]` scaffolds one from an existing theme, renaming the identifiers built from the name and optionally seeding the palette from `--mxt-*` declarations in a design artifact. A local theme shadows a built-in of the same name. Package: `cmd/mxcli/theme/`. See `docs/11-proposals/PROPOSAL_default_styling.md` - MPR v1/v2 reading and writing - Idempotent writes (ADR-0008): a unit whose new content is semantically equal to what is stored is **not written**, so re-running an MDL script against an in-sync project leaves the `.mpr` and `mprcontents/` byte-identical and Studio Pro shows no version-control changes. Comparison is on a canonical form (element `$ID`s normalised away — a rebuild mints them randomly, so byte comparison would skip nothing); `Microflows$Microflow.StableId` is carried from the stored document rather than re-minted, because the build derives every client-callable microflow's operation id from it. When a write **does** land, `canon.TransplantIDs` matches the rebuild against the stored document and reuses its element `$ID`s (rewriting every pointer in the same pass), so a changed document's diff is the change rather than a wholesale replacement — measured on #910's nanoflow: 1 of 37 identities survived an argument edit before, 37 of 37 after, and a change plus its revert returns to the original bytes. Inserting or deleting an activity mints IDs only for the genuinely new elements. One policy in `modelsdk/canon`, called from both engines' write choke points. `MXCLI_ALWAYS_WRITE=1` disables elision (not preservation) for bisecting — which means it no longer changes the resulting bytes, only the mtimes. The executor's output distinguishes the two: `Unchanged nanoflow: …` where the write was skipped. See `docs-site/src/internals/idempotent-writes.md` diff --git a/cmd/mxcli/brain/brain_test.go b/cmd/mxcli/brain/brain_test.go index 02c54bda63..bb0b7d51f2 100644 --- a/cmd/mxcli/brain/brain_test.go +++ b/cmd/mxcli/brain/brain_test.go @@ -634,3 +634,196 @@ func TestPlanSlicesGetMoreRoomThanDecisions(t *testing.T) { t.Fatal("a slice holds source material and is not loaded every session; it needs more room than a decision shard") } } + +func mustQuestion(t *testing.T, text, slice string, anchors ...string) Entry { + t.Helper() + e, err := NewQuestion(text, anchors, slice, day) + if err != nil { + t.Fatalf("NewQuestion(%q): %v", text, err) + } + return e +} + +// A question's anchors are not checked, and the control is the identical +// anchor recorded as a decision. Often the question IS whether the thing should +// exist, so the staleness rule that keeps decisions honest would report every +// question as a defect. +func TestOpenQuestionIsNotCheckedButTheSameDecisionIs(t *testing.T) { + dir := t.TempDir() + s := NewStore(dir) + if _, err := s.Init(); err != nil { + t.Fatal(err) + } + r := stubResolver{} // resolves nothing + + q := mustQuestion(t, "Should approvers see rejected orders?", "", "@Sales.ACT_Nope") + if err := s.Promote(q, "Sales"); err != nil { + t.Fatal(err) + } + rep, err := Check(s, r, []string{"Sales"}) + if err != nil { + t.Fatal(err) + } + if rep.Failed() { + t.Errorf("an open question must not fail the check: %+v", rep) + } + if len(rep.Open) != 1 || rep.Open[0].EntryID != q.ID { + t.Fatalf("the question must still be reported: %+v", rep.Open) + } + if len(rep.Findings) != 0 { + t.Errorf("a question's anchors must not be resolved at all: %+v", rep.Findings) + } + + // Control: the same anchor as a settled decision still fails. + d := mustEntry(t, "Approvers see rejected orders", "@Sales.ACT_Nope") + if err := s.Promote(d, "Sales"); err != nil { + t.Fatal(err) + } + rep, err = Check(s, r, []string{"Sales"}) + if err != nil { + t.Fatal(err) + } + if !rep.Failed() { + t.Error("a decision anchored at nothing must still fail — otherwise --open disables the check for everything") + } +} + +// Resolution is the transition the whole kind exists for: the entry becomes a +// decision in place, keeps its identity, and its anchors start being checked. +func TestResolvingAQuestionMakesItACheckedDecision(t *testing.T) { + dir := t.TempDir() + s := NewStore(dir) + if _, err := s.Init(); err != nil { + t.Fatal(err) + } + q := mustQuestion(t, "Should approvers see rejected orders?", "", "@Sales.Order") + if err := s.Promote(q, "Sales"); err != nil { + t.Fatal(err) + } + resolved, err := q.Resolve("Yes, for 30 days", day) + if err != nil { + t.Fatal(err) + } + if resolved.ID != q.ID { + t.Errorf("id changed on resolution (%s -> %s); an answered question is the same knowledge", q.ID, resolved.ID) + } + if resolved.Open { + t.Error("a resolved question is no longer open") + } + if !strings.Contains(resolved.Body, "Should approvers see rejected orders?") { + t.Errorf("the question must survive as the answer's context: %q", resolved.Body) + } + if err := s.Replace("Sales", resolved); err != nil { + t.Fatal(err) + } + + // Now it IS checked: a dead anchor on the answer fails. + rep, err := Check(s, stubResolver{}, []string{"Sales"}) + if err != nil { + t.Fatal(err) + } + if !rep.Failed() { + t.Error("once answered, the entry's anchors must be checked like any other decision") + } + if len(rep.Open) != 0 { + t.Errorf("it is no longer an open question: %+v", rep.Open) + } +} + +func TestResolveRefusesSomethingThatIsNotAQuestion(t *testing.T) { + if _, err := mustEntry(t, "A settled decision").Resolve("an answer", day); err == nil { + t.Fatal("resolving a decision must be refused") + } + if _, err := mustQuestion(t, "A question", "").Resolve("", day); err == nil { + t.Fatal("an empty answer must be refused") + } +} + +func TestReplaceKeepsPositionInTheShard(t *testing.T) { + dir := t.TempDir() + s := NewStore(dir) + if _, err := s.Init(); err != nil { + t.Fatal(err) + } + first := mustEntry(t, "First decision", "@Sales.A") + q := mustQuestion(t, "A question?", "", "@Sales.B") + last := mustEntry(t, "Last decision", "@Sales.C") + for _, e := range []Entry{first, q, last} { + if err := s.Promote(e, "Sales"); err != nil { + t.Fatal(err) + } + } + resolved, err := q.Resolve("An answer", day) + if err != nil { + t.Fatal(err) + } + if err := s.Replace("Sales", resolved); err != nil { + t.Fatal(err) + } + entries, _, err := s.LoadShard("Sales") + if err != nil { + t.Fatal(err) + } + if len(entries) != 3 || entries[1].ID != q.ID || entries[1].Title != "An answer" { + t.Fatalf("a resolved question must stay where it was, not move to the end: %+v", entries) + } +} + +// A question filed against a slice is not scope until it is answered, so it +// must not inflate the slice's outstanding work. +func TestSliceQuestionsAreCountedApartFromRequirements(t *testing.T) { + dir := t.TempDir() + s := NewStore(dir) + if _, err := s.Init(); err != nil { + t.Fatal(err) + } + req := mustRequirement(t, "Orders can be approved", "02-approvals", "@Sales.ACT_Approve") + q := mustQuestion(t, "Do approvers see rejected orders?", "02-approvals", "@Sales.Order") + for _, e := range []Entry{req, q} { + if err := s.Promote(e, e.Shard()); err != nil { + t.Fatal(err) + } + } + rep, err := Check(s, stubResolver{}, []string{PlanShard("02-approvals")}) + if err != nil { + t.Fatal(err) + } + got := rep.Slices[0] + if got.Questions != 1 { + t.Errorf("questions = %d, want 1: %+v", got.Questions, got) + } + if got.Planned != 1 || got.Total() != 1 { + t.Errorf("an unanswered question is not outstanding scope: %+v", got) + } + if len(rep.Open) != 1 { + t.Errorf("a slice's questions must still be reported: %+v", rep.Open) + } +} + +func TestOpenMarkerRoundTrips(t *testing.T) { + in := []Entry{ + mustQuestion(t, "Still open?", "", "@Sales.Order"), + mustEntry(t, "Settled", "@Sales.Order"), + } + out, malformed, err := ParseShard("Sales", RenderShard("Sales", in)) + if err != nil || len(malformed) != 0 { + t.Fatalf("err=%v malformed=%v", err, malformed) + } + if !out[0].Open || out[1].Open { + t.Fatalf("the OPEN marker did not round-trip: %+v", out) + } +} + +// Entries written before questions existed carry no marker and must read back +// as settled decisions, not as questions. +func TestEntryWithoutTheMarkerIsNotOpen(t *testing.T) { + content := "# Sales\n\n" + shardMarker + "\n\n## An older entry\n\n" + + "Anchors: `@Sales.Order` · id `abc123` · 2026-08-01\n" + out, malformed, err := ParseShard("Sales", content) + if err != nil || len(malformed) != 0 { + t.Fatalf("err=%v malformed=%v", err, malformed) + } + if len(out) != 1 || out[0].Open { + t.Fatalf("an entry with no marker is a settled decision: %+v", out) + } +} diff --git a/cmd/mxcli/brain/check.go b/cmd/mxcli/brain/check.go index 17ca24410c..0b06f969e9 100644 --- a/cmd/mxcli/brain/check.go +++ b/cmd/mxcli/brain/check.go @@ -67,6 +67,13 @@ type AnchorFinding struct { Kind string } +// OpenQuestion is something the project has not decided yet. +type OpenQuestion struct { + Shard string + EntryID string + Title string +} + // MisfiledFinding is an entry sitting in a shard none of its anchors belong to. type MisfiledFinding struct { Shard string @@ -85,12 +92,18 @@ type SliceProgress struct { // Planned is requirements with at least one anchor that does not resolve // yet. Not a failure: that is what a requirement is until it is built. Planned int + // Questions is open questions filed against this slice — scope that is not + // settled. They are not requirements and are not counted as either built + // or planned; counting an unanswered question as outstanding work would + // overstate the slice. + Questions int // Unanchored is requirements with no anchor at all. They cannot be // measured, and are counted apart rather than silently called planned. Unanchored int } -// Total is every requirement in the slice. +// Total is every requirement in the slice. Open questions are excluded: they +// are not scope until they are answered. func (p SliceProgress) Total() int { return p.Built + p.Planned + p.Unanchored } // Report is what `brain check` prints and exits on. @@ -103,6 +116,7 @@ type Report struct { Misfiled []MisfiledFinding Malformed []string // entry blocks whose metadata line could not be read Slices []SliceProgress + Open []OpenQuestion } // Failed reports whether the check should exit non-zero. @@ -143,10 +157,19 @@ func Check(s *Store, r Resolver, shards []string) (Report, error) { rep.Anchors += progress.anchors rep.ResolvedN += progress.resolved rep.Slices = append(rep.Slices, progress.SliceProgress) + rep.Open = append(rep.Open, progress.open...) continue } for _, e := range entries { rep.Entries++ + if e.Open { + // A question's anchors are not checked. It may name something + // that does not exist — often the question IS whether it + // should — so the staleness rule that keeps decisions honest + // would report every question as a defect. + rep.Open = append(rep.Open, OpenQuestion{Shard: shard, EntryID: e.ID, Title: e.Title}) + continue + } var resolvedModules []string for _, a := range e.ParsedAnchors() { rep.Anchors++ @@ -178,6 +201,7 @@ func Check(s *Store, r Resolver, shards []string) (Report, error) { type sliceCounts struct { SliceProgress anchors, resolved int + open []OpenQuestion } // checkSlice counts a slice's requirements against the model. It records no @@ -192,6 +216,11 @@ type sliceCounts struct { func checkSlice(r Resolver, shard string, entries []Entry) (sliceCounts, error) { out := sliceCounts{SliceProgress: SliceProgress{Slice: SliceOf(shard)}} for _, e := range entries { + if e.Open { + out.Questions++ + out.open = append(out.open, OpenQuestion{Shard: shard, EntryID: e.ID, Title: e.Title}) + continue + } anchors := e.ParsedAnchors() if len(anchors) == 0 { out.Unanchored++ diff --git a/cmd/mxcli/brain/entry.go b/cmd/mxcli/brain/entry.go index 320b89714f..feff2de6e0 100644 --- a/cmd/mxcli/brain/entry.go +++ b/cmd/mxcli/brain/entry.go @@ -39,7 +39,17 @@ type Entry struct { // existed still reads correctly. Use EntryKind rather than this field. Kind Kind `json:"kind,omitempty"` // Slice is the deliverable a requirement belongs to. Empty for a decision. - Slice string `json:"slice,omitempty"` + Slice string `json:"slice,omitempty"` + // Open marks an unresolved question: a decision that has not been made + // yet. It lives beside the decisions it will join rather than in a file of + // its own, because the moment you need to see it is when you are reading + // what that module already decided. + // + // Unlike Kind for a requirement — which is implied by the file — this has + // to travel with the entry, since a question shares its shard with settled + // decisions. There is still only ONE copy of the fact, so nothing can + // drift; what the store forbids is two copies, not one in an odd place. + Open bool `json:"open,omitempty"` Title string `json:"title"` Body string `json:"body,omitempty"` Anchors []string `json:"anchors,omitempty"` @@ -129,6 +139,52 @@ func NewRequirement(text string, anchors []string, slice string, now time.Time) // user's choice rather than a field mxcli maintains. var sliceName = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_-]*$`) +// NewQuestion builds an open question: something undecided, recorded so it is +// not silently forgotten and rediscovered expensively later. +// +// A question's anchors are not checked for staleness. It may name something +// that does not exist — the question is often precisely whether it should — so +// the rule that keeps decisions honest would report every question as a defect. +func NewQuestion(text string, anchors []string, slice string, now time.Time) (Entry, error) { + var ( + e Entry + err error + ) + if slice != "" { + e, err = NewRequirement(text, anchors, slice, now) + } else { + e, err = NewEntry(text, anchors, now) + } + if err != nil { + return Entry{}, err + } + e.Open = true + return e, nil +} + +// Resolve turns a question into the decision it was always going to become, +// keeping the entry's id so that anything referring to it still resolves, and +// keeping the question itself as the answer's context. +// +// The id therefore stops matching the title it was derived from. That is +// deliberate: an answered question is the same piece of knowledge as the +// question, and re-minting the id would make it a different one. +func (e Entry) Resolve(answer string, now time.Time) (Entry, error) { + if !e.Open { + return Entry{}, fmt.Errorf("entry %s is not an open question", e.ID) + } + title, body := splitTitle(answer) + if title == "" { + return Entry{}, fmt.Errorf("an answer needs at least one line") + } + out := e + out.Open = false + out.Title = title + out.Date = now.Format("2006-01-02") + out.Body = strings.TrimSpace("Resolves: " + e.Title + "\n\n" + strings.TrimSpace(body+"\n\n"+e.Body)) + return out, nil +} + // Shard is where this entry belongs. A requirement goes to its slice; a // decision to the module of its first anchor. func (e Entry) Shard() string { diff --git a/cmd/mxcli/brain/shard.go b/cmd/mxcli/brain/shard.go index ca89fd2b17..6c870776dd 100644 --- a/cmd/mxcli/brain/shard.go +++ b/cmd/mxcli/brain/shard.go @@ -22,7 +22,7 @@ const shardMarker = "" // metaLine matches an entry's one metadata line. The separator is a middle dot // so that a title or body containing a hyphen cannot be mistaken for it. -var metaLine = regexp.MustCompile("^Anchors: (.*?) · id `([0-9a-f]{6})` · (\\d{4}-\\d{2}-\\d{2})\\s*$") +var metaLine = regexp.MustCompile("^Anchors: (.*?) · id `([0-9a-f]{6})` · (\\d{4}-\\d{2}-\\d{2})( · OPEN)?\\s*$") // anchorRef matches one backticked anchor inside the metadata line. var anchorRef = regexp.MustCompile("`(@[A-Za-z_][A-Za-z0-9_.]*)`") @@ -45,7 +45,11 @@ func RenderShard(shard string, entries []Entry) string { func renderEntry(e Entry) string { var b strings.Builder fmt.Fprintf(&b, "## %s\n\n", e.Title) - fmt.Fprintf(&b, "Anchors: %s · id `%s` · %s\n", renderAnchors(e.Anchors), e.ID, e.Date) + open := "" + if e.Open { + open = " · OPEN" + } + fmt.Fprintf(&b, "Anchors: %s · id `%s` · %s%s\n", renderAnchors(e.Anchors), e.ID, e.Date, open) if e.Body != "" { fmt.Fprintf(&b, "\n%s\n", e.Body) } @@ -125,6 +129,7 @@ func parseEntry(block string) (Entry, bool) { e.Anchors = append(e.Anchors, a[1]) } e.ID, e.Date = m[2], m[3] + e.Open = m[4] != "" metaAt = i break } diff --git a/cmd/mxcli/brain/store.go b/cmd/mxcli/brain/store.go index 1637c2eff5..cd584ed502 100644 --- a/cmd/mxcli/brain/store.go +++ b/cmd/mxcli/brain/store.go @@ -189,6 +189,47 @@ func (s *Store) Promote(e Entry, shard string) error { return s.SaveShard(shard, next) } +// Replace swaps an entry for a new one with the same id, in place. Used by +// resolution, where a question becomes the decision it was heading towards and +// should not move, jump to the end of the file, or change identity. +func (s *Store) Replace(shard string, e Entry) error { + entries, _, err := s.LoadShard(shard) + if err != nil { + return err + } + found := false + for i := range entries { + if entries[i].ID == e.ID { + entries[i], found = e, true + break + } + } + if !found { + return fmt.Errorf("%s does not carry entry %s", shard, e.ID) + } + return s.SaveShard(shard, entries) +} + +// Find returns the entry with the given id and the shard holding it. +func (s *Store) Find(id string) (Entry, string, error) { + shards, err := s.ListShards() + if err != nil { + return Entry{}, "", err + } + for _, sh := range shards { + entries, _, err := s.LoadShard(sh) + if err != nil { + return Entry{}, "", err + } + for _, e := range entries { + if e.ID == id { + return e, sh, nil + } + } + } + return Entry{}, "", nil +} + // Drop removes an entry by id, reporting which shard it came from and whether // that emptied the shard. func (s *Store) Drop(id string) (shard string, deletedFile bool, err error) { @@ -221,6 +262,29 @@ func (s *Store) Drop(id string) (shard string, deletedFile bool, err error) { return "", false, nil } +// OpenQuestions lists the unanswered questions across the store. It needs no +// resolver and no catalog — just the files — so a caller that only wants to +// nag about them pays a few small reads rather than a model load. +func (s *Store) OpenQuestions() ([]OpenQuestion, error) { + shards, err := s.ListShards() + if err != nil { + return nil, err + } + var out []OpenQuestion + for _, sh := range shards { + entries, _, err := s.LoadShard(sh) + if err != nil { + return nil, err + } + for _, e := range entries { + if e.Open { + out = append(out, OpenQuestion{Shard: sh, EntryID: e.ID, Title: e.Title}) + } + } + } + return out, nil +} + // Usage computes size and headroom for every shard. Nothing here is cached or // written down (A6). func (s *Store) Usage() ([]Usage, error) { diff --git a/cmd/mxcli/cmd_brain.go b/cmd/mxcli/cmd_brain.go index d710aea52d..7decb0a692 100644 --- a/cmd/mxcli/cmd_brain.go +++ b/cmd/mxcli/cmd_brain.go @@ -93,9 +93,13 @@ var brainCaptureCmd = &cobra.Command{ e brain.Entry err error ) - if slice != "" { + open, _ := cmd.Flags().GetBool("open") + switch { + case open: + e, err = brain.NewQuestion(args[0], anchors, slice, time.Now()) + case slice != "": e, err = brain.NewRequirement(args[0], anchors, slice, time.Now()) - } else { + default: e, err = brain.NewEntry(args[0], anchors, time.Now()) } if err != nil { @@ -109,7 +113,11 @@ var brainCaptureCmd = &cobra.Command{ fmt.Printf("Already queued as %s — not added again.\n", e.ID) return } - fmt.Printf("Queued %s -> would promote into %s\n", e.ID, shardLabel(e.Shard())) + what := "Queued" + if e.Open { + what = "Queued open question" + } + fmt.Printf("%s %s -> would promote into %s\n", what, e.ID, shardLabel(e.Shard())) fmt.Println("Review with 'mxcli brain staged'; commit it with 'mxcli brain promote " + e.ID + "'.") }, } @@ -229,6 +237,39 @@ var brainShowCmd = &cobra.Command{ }, } +var brainResolveCmd = &cobra.Command{ + Use: "resolve ", + Short: "Answer an open question, turning it into a decision", + Long: `Answer an open question. + +The entry becomes an ordinary decision in place: same id, same position in the +file, with the question kept as the answer's context. It does not move and does +not get a new identity, because an answered question is the same piece of +knowledge as the question — anything referring to it still resolves. + +This is the step that stops questions accumulating. A question nobody answers +is reported by 'brain check' and by 'mxcli lint' until someone does.`, + Args: cobra.ExactArgs(2), + Run: func(cmd *cobra.Command, args []string) { + store := brain.NewStore(brainProjectDir(cmd)) + e, shard, err := store.Find(args[0]) + if err != nil { + brainFatal(err) + } + if shard == "" { + brainFatal(fmt.Errorf("no committed entry with id %s", args[0])) + } + resolved, err := e.Resolve(args[1], time.Now()) + if err != nil { + brainFatal(err) + } + if err := store.Replace(shard, resolved); err != nil { + brainFatal(err) + } + fmt.Printf("Resolved %s in %s\n", resolved.ID, shardLabel(shard)) + }, +} + var brainPlanCmd = &cobra.Command{ Use: "plan", Short: "The roadmap: each slice's requirements counted against the model", @@ -388,8 +429,18 @@ func printBrainReport(rep brain.Report) { fmt.Println() printBrainPlan(rep.Slices) } + for _, q := range rep.Open { + fmt.Printf("OPEN %s: %q (%s)\n", q.EntryID, q.Title, shardLabel(q.Shard)) + } fmt.Printf("\n%d entries, %d anchors, %d resolved, across %d shard(s).\n", rep.Entries, rep.Anchors, rep.ResolvedN, len(rep.Shards)) + if n := len(rep.Open); n > 0 { + noun := "questions" + if n == 1 { + noun = "question" + } + fmt.Printf("%d open %s — answer one with 'mxcli brain resolve \"\"'.\n", n, noun) + } if !rep.Failed() { fmt.Println("OK") } @@ -582,6 +633,8 @@ func init() { } brainCaptureCmd.Flags().StringSliceP("anchor", "a", nil, "Anchor into the model (@Module, @Module.Element, @Module.Entity.Attribute); repeatable") + brainCaptureCmd.Flags().Bool("open", false, + "Record this as an OPEN QUESTION — something not decided yet, so its anchors are not checked") brainCaptureCmd.Flags().StringP("slice", "s", "", "Record this as a requirement of the named slice (plan/.md) instead of a decision") brainPromoteCmd.Flags().String("to", "", @@ -590,7 +643,8 @@ func init() { brainCheckCmd.Flags().Bool("ci", false, "Machine-friendly output for CI") brainPlanCmd.Flags().StringP("project", "p", "", "Path to the .mpr file") + brainResolveCmd.Flags().StringP("project", "p", "", "Path to the .mpr file") brainCmd.AddCommand(brainInitCmd, brainCaptureCmd, brainStagedCmd, - brainPromoteCmd, brainDropCmd, brainShowCmd, brainCheckCmd, brainPlanCmd) + brainPromoteCmd, brainDropCmd, brainShowCmd, brainCheckCmd, brainPlanCmd, brainResolveCmd) rootCmd.AddCommand(brainCmd) } diff --git a/cmd/mxcli/cmd_lint.go b/cmd/mxcli/cmd_lint.go index c39241791b..404e4de13e 100644 --- a/cmd/mxcli/cmd_lint.go +++ b/cmd/mxcli/cmd_lint.go @@ -379,14 +379,33 @@ func reportBrainGap(projectDir string) { return } staged, err := brain.NewQueue(projectDir).Load() - if err != nil || len(staged) == 0 { + if err != nil { return } - noun := "entries" - if len(staged) == 1 { - noun = "entry" + // An unanswered question is the same shape of gap as an unpromoted entry: + // something a person has to act on, that no other command would mention. + open, err := brain.NewStore(projectDir).OpenQuestions() + if err != nil { + return + } + if len(staged) == 0 && len(open) == 0 { + return + } + var parts []string + if len(staged) > 0 { + parts = append(parts, fmt.Sprintf("%d staged %s not yet promoted ('mxcli brain staged')", + len(staged), plural(len(staged), "entry", "entries"))) + } + if len(open) > 0 { + parts = append(parts, fmt.Sprintf("%d open %s ('mxcli brain check')", + len(open), plural(len(open), "question", "questions"))) + } + fmt.Fprintf(os.Stderr, "\nProject brain: %s.\n", strings.Join(parts, "; ")) +} + +func plural(n int, one, many string) string { + if n == 1 { + return one } - fmt.Fprintf(os.Stderr, - "\nProject brain: %d staged %s not yet promoted — 'mxcli brain staged' to review.\n", - len(staged), noun) + return many } diff --git a/docs-site/src/tools/project-brain.md b/docs-site/src/tools/project-brain.md index 546fc67859..b3c3f603d7 100644 --- a/docs-site/src/tools/project-brain.md +++ b/docs-site/src/tools/project-brain.md @@ -4,6 +4,8 @@ - **Decisions** — why a pattern was chosen here, which marketplace version broke what, what a recurring mxbuild error means in *this* app. +- **Open questions** — what is not decided yet, recorded so it is not silently + forgotten and rediscovered expensively later. - **The plan** — the requirements being built from, grouped into slices, when the source is a specification document, a prototype or a conversation. @@ -171,6 +173,51 @@ implement it. Misfiling is not checked for slices: a slice spans modules by design. +## Open questions + +An open question is a decision that has not been made yet. + +```bash +mxcli brain capture "Do approvers see rejected orders?" --open -a @Sales.Order -p app.mpr +``` + +Its **anchors are not checked**. A question may name something that does not +exist — often the question is precisely whether it should — so the staleness +rule that keeps decisions honest would report every question as a defect. +Measured: the identical anchor takes `brain check` to exit 1 as a decision and +exit 0 as a question. + +`--open` combines with `--slice`. A question about a slice's scope is filed with +that slice and counted apart from its requirements: an unanswered question is +not outstanding scope, so it never inflates the slice's numbers. + +Answering it converts it in place: + +```bash +mxcli brain resolve "Yes, for 30 days +Agreed with the product owner; drives the overview filter and the access rule." +``` + +The entry keeps its **id** and its position in the file, and the question +survives as the answer's context. From that moment its anchors are checked like +any other decision — which is the whole point of the transition, and is asserted +by a test. + +Both `brain check` and `mxcli lint` report unanswered questions until one is +resolved. A question nobody answers is the one kind of entry that gets more +expensive the longer it sits. + +## When to capture a decision + +Capture needs a trigger, not good intentions. The reliable one: + +> **You have had to correct the same thing twice.** + +The second correction is the signal — it will happen a third time, to someone +else. The other trigger is choosing between real alternatives where the losing +one would look reasonable to the next person: record the choice *and* what ruled +the other out, or it gets re-litigated. + ## Size ```bash @@ -209,6 +256,8 @@ stale the next time anyone promotes. | `brain promote [--to ]` | Writes it into its shard | | `brain drop ` | Removes it from the queue or from its shard | | `brain capture "" --slice [-a @Anchor]…` | Queues a **requirement** of that slice | +| `brain capture "" --open [-a @Anchor]…` | Queues an **open question**; anchors not checked | +| `brain resolve ""` | Answers it, turning it into a decision in place | | `brain plan` | Each slice's requirements counted against the model | | `brain check [--changed]` | Anchors resolve, entries filed correctly, plus slice progress | | `brain show []` | Entries, lines and headroom per shard |