From 3921be1d00c445d834f97653fc69919fe30cd029 Mon Sep 17 00:00:00 2001 From: Ryan Dombrowski Date: Tue, 21 Jul 2026 19:27:31 -0400 Subject: [PATCH] scenario: hotel reservations ships ready (transactional-review) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The scenario that validates the selection vocabulary — atomic with the intent that governs it, per the established no-intent-without-scenario discipline. The shelf completes: six ready, zero planned. Intent transactional-review + five floor rules: options in a collection with an explicit action (component-choice); at least two alternatives per list (required-composition min:2 — the reason `list` exists); every option states attributes (selectable-card ⊃ metadata-list); metadata carries items; every option is named. Exact counts, upper bounds, and exactly-one-primary are inexpressible in current rule types and remain prose guidance, stated in the rules' own rationales. The worked example is deliberately NOT a hotel — ex.plan-comparison, pricing plans — so the vocabulary's domain neutrality is exercised before hotel copy exists. Fixtures (Spark, ollama:gpt-oss:latest, published dspack-gen 0.1.2): - fixture-015 clean: FIRST live pass on the deepest composite yet (card → list → selectable-cards → metadata-lists): three named, badged hotels, 3-item metadata each, one "Book now" action. - fixture-016 argues-back: asked for a single no-options recommendation, the model ships a bare apology text node; rule.review-presents-options catches a surface with nothing to choose (three findings) and the repair ships two named hotels compared on their attributes. First --require-repair attempt, unforced. Break-it: review-without-options (recordedCatch → argues-back). studio-shell's planned-reveal test becomes "six ready, no planned placeholders" (the planned-entry machinery keeps its code path); permalinks' not-ready test uses an invented id now that none remains. Fail-first: pre-contract recording failed with "intent 'transactional-review' is not registered"; all four hotel-reservations.spec.ts tests failed on the pre-scenario build. Gates: pnpm -r typecheck/test green; contracts build+drift clean; local Playwright 106 passed / 4 skipped; production config vs local deploy artifact 86 passed; one live agent run produced a clean three-option comparison, audit passed. Co-Authored-By: Claude Fable 5 --- docs/AUDIT.md | 2 +- docs/IMPLEMENTATION_LOG.md | 52 ++ docs/NAV-REFACTOR.md | 6 +- e2e/hotel-reservations.spec.ts | 64 ++ e2e/permalinks.spec.ts | 3 +- e2e/studio-shell.spec.ts | 30 +- packages/contracts/astryx.dspack.json | 247 ++++++ packages/replay/fixtures/fixture-015.json | 846 ++++++++++++++++++++ packages/replay/fixtures/fixture-016.json | 864 +++++++++++++++++++++ packages/scenarios/src/break-conditions.ts | 18 +- packages/scenarios/src/registry.ts | 43 +- playwright.production.config.ts | 1 + 12 files changed, 2148 insertions(+), 28 deletions(-) create mode 100644 e2e/hotel-reservations.spec.ts create mode 100644 packages/replay/fixtures/fixture-015.json create mode 100644 packages/replay/fixtures/fixture-016.json diff --git a/docs/AUDIT.md b/docs/AUDIT.md index 367076c..0688a33 100644 --- a/docs/AUDIT.md +++ b/docs/AUDIT.md @@ -18,7 +18,7 @@ followed it. Statuses: Complete, Partial, Missing, Intentionally changed. | X-ray and provenance (FM-4) | Complete | Canvas click-to-trace, provenance card (node, catalog entry, creating/updating events, rules), reverse violet wash for the playhead delivery. This pass added a keyboard path: "trace" buttons in the inspector components tab. | | Governed repair / "argues back" (FM-1) | Complete | fixture-001 carries two real repairs (`rule.destructive-requires-alertdialog`, `rule.alertdialog-action-label-specific`); gate ticker + gates tab show findings and verbatim repair messages. | | Break-it mode (FM-8) | Complete | 7 conditions across 6 failure kinds through the ordinary RunView. This pass added offline behavior: three conditions replay their recorded catch when the local agent is absent (the deployed static site); live-only conditions say so plainly. | -| Scenarios | Partial by plan | 5 ready (project-deletion, appointment-booking, recipe-creator, support-triage, onboarding), 1 planned (hotel-reservations) presented honestly with its blockers. support-triage and onboarding shipped 2026-07-21 with owner-authored governance (record-collection, data-collection) and live gpt-oss recordings (fixtures 010-014). hotel-reservations needs new design-system components (plan §17, open question 2) — deferred, not a defect. | +| Scenarios | Complete | 6 ready (project-deletion, appointment-booking, recipe-creator, support-triage, onboarding, hotel-reservations); no planned placeholders remain. support-triage, onboarding, and hotel-reservations shipped 2026-07-21 with owner-authored governance (record-collection, data-collection, transactional-review) and live gpt-oss recordings (fixtures 010-016). hotel-reservations rode the selection vocabulary (list, selectable-card, metadata-list) that plan §17 deferred; the planned-entry shelf treatment stands by for the next expansion. | | Recipe shared-state experience (FM-6) | Complete | Real dishes, per-serving amounts, constraint swaps; this pass added numbered cooking instructions that the constraint swaps rewrite, on both the deterministic and generated paths, and re-recorded fixture-006 (live `gpt-oss`, 2026-07-11) with the model laying out an instructions table and the labeled enhancement seeding responder content into it. | | Live BYO-key mode | Intentionally changed | The plan's "browser BYO key" became "no credentials in the browser": modelRef selects `scripted` or local `ollama:*`; hosted keys live only in the agent's env. Stricter than the plan; documented in the UI. | | HITL (FM-7) | Partial | Appointment booking pauses for slot selection and validates round-trips; the agent's question is not itself a governed AlertDialog. Post-MVP. | diff --git a/docs/IMPLEMENTATION_LOG.md b/docs/IMPLEMENTATION_LOG.md index 76d6a3d..939a189 100644 --- a/docs/IMPLEMENTATION_LOG.md +++ b/docs/IMPLEMENTATION_LOG.md @@ -1001,3 +1001,55 @@ five onboarding.spec.ts tests fail on the old registry. `studio-shell.spec.ts`'s planned-reveal test retargeted `scenario-onboarding` → `scenario-hotel-reservations` (the last planned entry) — a retarget, not a new assertion; it passes on both builds. + +## 2026-07-21 — selection vocabulary + hotel reservations: the shelf completes + +Two PRs, per the approved platform design: the vocabulary first (inert +until governed), then the intent and the scenario that validates it, +atomically. + +Vocabulary (PR A): `list`, `selectable-card`, `metadata-list` — each a +faithful projection of a real upstream Astryx component, none +hotel-specific. metadata-list is data-driven ({label,value} items, +grammar-required) because sub-components carry no props under +grammar-constrained generation; the one deliberate upstream divergence +is declared as `x-drift` on the prop, and drift-check now reports +x-drift props as acknowledged divergences. Renderers landed in BOTH +design systems (Dialog stays the sole shadcn placeholder, 11 of 12). +The record-collection table rule's "widen to a choice set" caveat was +revised honestly: choice sets are inexpressible (component-choice +require is conjunctive); requireOneOf noted as candidate dspack-gen +work. + +Governance + scenario (PR B): intent `transactional-review` with five +floor rules (options-in-a-collection with an explicit action; ≥2 +alternatives per list; every option states attributes; metadata carries +items; every option is named). Exact counts, upper bounds, and +"exactly one primary action per option" are inexpressible in current +rule types and remain prose guidance, stated in the rules' own +rationales. The worked example is deliberately NOT a hotel — +`ex.plan-comparison`, pricing plans — so the vocabulary's domain +neutrality is exercised before any hotel copy exists. + +Recording (Spark, ollama:gpt-oss:latest, published dspack-gen 0.1.2): +the very first live run on the deepest composite yet (card → list → +selectable-cards → metadata-lists) passed clean with populated +attributes — the 0.1.2 required-props grammar mechanism carrying over. +fixture-015 (clean): three named, badged hotels, each with a 3-item +metadata list, one "Book now" action, first pass. fixture-016 +(argues-back): asked for a single no-options recommendation, the model +shipped a bare apology text node; rule.review-presents-options caught a +surface with nothing to choose (three findings — list, selectable-card, +button all absent) and the repair shipped two named hotels compared on +their attributes. First --require-repair attempt, unforced. + +Break-it: `review-without-options` (recordedCatch → argues-back). The +planned-entry shelf machinery keeps its code path but nothing renders +it: studio-shell's planned-reveal test became "six ready, no planned +placeholders", and permalinks' not-ready-link test now uses an invented +id. Docs: AUDIT Scenarios row goes Complete (6 ready / 0 planned). + +Fail-first evidence: pre-contract, recording with the new intent failed +with getIntent's "intent 'transactional-review' is not registered"; all +four hotel-reservations.spec.ts tests and the rewritten studio-shell +assertion failed on the pre-scenario build. diff --git a/docs/NAV-REFACTOR.md b/docs/NAV-REFACTOR.md index 1da0976..a67d374 100644 --- a/docs/NAV-REFACTOR.md +++ b/docs/NAV-REFACTOR.md @@ -69,9 +69,9 @@ session file, or run it live and download one."* ## 2. Current architecture - **Scenario** = `scenarioId` (React state in `Studio`), canonical ids from - `packages/scenarios/src/registry.ts`: ready `recipe-creator`, + `packages/scenarios/src/registry.ts`: all six ready — `recipe-creator`, `appointment-booking`, `project-deletion`, `support-triage`, - `onboarding`; planned (not selectable) `hotel-reservations`. + `onboarding`, `hotel-reservations`. No planned entries remain. - **Operation** = `view` state: `"replay" | "live" | "break" | "canvas"`. X-ray and Fork/diff are not top-level views: they are controls inside `RunView` and operate on whatever run is loaded — already @@ -158,7 +158,7 @@ local agent · **gap** = no support yet. Cells marked † change with this work. | *(any scenario)* | — | — | `malformed-import`: **universal** (client-side validator; mislabeled project-deletion → `scenarioIndependent`†) | — | — | — | | `support-triage` | **recorded** ×2 (fixtures 010/011) | **live-only** (generation: `scripted` deterministic or `ollama:*`; zero agent code, same as project-deletion) | `records-as-prose` **recorded** (catch=argues-back) | **universal** via fixture-011 final surface | **universal** | **universal** | | `onboarding` | **recorded** ×3 (fixtures 012/013/014) | **live-only** (generation: `scripted` deterministic or `ollama:*`; zero agent code) | `ask-without-a-form` **recorded** (catch=argues-back) | **universal** via fixture-012 final surface | **universal** | **universal** | -| `hotel-reservations` (planned) | **gap** (not selectable; shelf states needs) | **gap** | **gap** | **gap** | n/a until a run loads | n/a | +| `hotel-reservations` | **recorded** ×2 (fixtures 015/016) | **live-only** (generation: `scripted` deterministic or `ollama:*`; zero agent code) | `review-without-options` **recorded** (catch=argues-back) | **universal** via fixture-015 final surface | **universal** | **universal** | Unused asset worth noting: `packages/replay/fixtures/fixture-004.json` ("Scheduling, generated live", intent `scheduling`, 12 events) is bundled but diff --git a/e2e/hotel-reservations.spec.ts b/e2e/hotel-reservations.spec.ts new file mode 100644 index 0000000..facc7d4 --- /dev/null +++ b/e2e/hotel-reservations.spec.ts @@ -0,0 +1,64 @@ +/** + * Hotel reservations: the transactional-review scenario, replay-first, + * against the static export. Both fixtures are mode:"live" recordings + * (gpt-oss via Ollama) — nothing here is hand-scripted content. + * + * The governance story: a review presents named options as selectable cards + * in a list, compared on metadata attributes, with one explicit committing + * action — a platform vocabulary (plan pickers, product comparison) that + * this hotel scenario validates, not the other way round. + */ +import { expect, test, type Page } from "@playwright/test"; + +const scrubber = (page: Page) => page.getByTestId("scrubber"); + +async function scrubToEnd(page: Page) { + await scrubber(page).focus(); + await page.keyboard.press("End"); +} + +test.beforeEach(async ({ page }) => { + await page.goto("/"); +}); + +test("hotel-reservations is a ready, selectable scenario on the shelf", async ({ page }) => { + const shelf = page.getByTestId("scenario-hotel-reservations"); + await expect(shelf).not.toContainText("(planned)"); + await shelf.click(); + await expect(page.getByTestId("fixture-argues-back")).toBeVisible(); + await expect(page.getByTestId("fixture-clean")).toBeVisible(); +}); + +test("clean replay: three named options, attributes as metadata, one action", async ({ page }) => { + await page.getByTestId("scenario-hotel-reservations").click(); + await page.getByTestId("fixture-clean").click(); + await expect(page.getByTestId("fixture-meta")).toContainText("live run"); + await scrubToEnd(page); + await expect(page.getByTestId("audit-outcome")).toContainText("passed"); + const canvas = page.locator("[data-canvas]"); + // SelectableCard labels surface as accessible names (checkbox semantics). + await expect(canvas.getByRole("checkbox", { name: "Hotel Belém" })).toBeVisible(); + await expect(canvas).toContainText("Best value"); + await expect(canvas).toContainText("Book now"); +}); + +test("repair replay: a bare apology caught by the review rules, options ship", async ({ page }) => { + await page.getByTestId("scenario-hotel-reservations").click(); + await page.getByTestId("fixture-argues-back").click(); + await scrubToEnd(page); + const ticker = page.getByTestId("gate-ticker"); + await expect(ticker).toContainText(/attempt 0:\s*S1✓\s*S2✓\s*S3✗\s*→ repair/); + await expect(ticker).toContainText(/attempt 1:\s*S1✓\s*S2✓\s*S3✓/); + await expect(ticker).toContainText("outcome passed"); + // The design system won: the apology became a comparison. + const canvas = page.locator("[data-canvas]"); + await expect(canvas.getByRole("checkbox", { name: "Ritz-Carlton Lisboa" })).toBeVisible(); + await expect(canvas).toContainText("Book Hotel"); +}); + +test("permalink: a hotel-reservations deep link resolves", async ({ page }) => { + await page.goto("/#s=hotel-reservations&f=clean"); + await page.reload(); + await expect(page.getByTestId("fixture-meta")).toContainText("live run"); + await expect(page.getByTestId("link-error")).toHaveCount(0); +}); diff --git a/e2e/permalinks.spec.ts b/e2e/permalinks.spec.ts index ee258fb..c8d8da4 100644 --- a/e2e/permalinks.spec.ts +++ b/e2e/permalinks.spec.ts @@ -92,7 +92,8 @@ test("malformed and unknown links fail clearly and fall back", async ({ page }) await expect(page.getByTestId("link-error")).toContainText("did not parse"); await expect(page.getByTestId("fixture-meta")).toBeVisible(); // default view still works - await page.goto("/#s=hotel-reservations"); + // No planned scenarios remain, so an invented id exercises the same path. + await page.goto("/#s=not-a-scenario"); await page.reload(); await expect(page.getByTestId("link-error")).toContainText("unknown or not-yet-ready"); diff --git a/e2e/studio-shell.spec.ts b/e2e/studio-shell.spec.ts index 344899b..ba9b574 100644 --- a/e2e/studio-shell.spec.ts +++ b/e2e/studio-shell.spec.ts @@ -40,21 +40,25 @@ test("the pipeline diagram names the honest pipeline in reading order", async ({ await expect(figure.locator("figcaption")).toContainText("One pipeline, inspectable at every joint"); }); -test("planned scenarios reveal what they need, by keyboard", async ({ page }) => { +test("the shelf carries six ready scenarios and no planned placeholders", async ({ page }) => { await page.goto("/"); - // hotel-reservations is the last remaining planned scenario. - const planned = page.getByTestId("scenario-hotel-reservations"); - await expect(planned).toContainText("(planned)"); - await planned.focus(); - await page.keyboard.press("Enter"); - await expect(page.getByTestId("planned-needs")).toContainText(/planned, not built/i); - await expect(page.getByTestId("planned-needs")).toContainText(/needs/i); - // Revealing never selects the scenario: the ready experience is untouched - // (recipe-creator is the default, so its recording stays on screen). - await expect(page.getByTestId("fixture-generated-cooked")).toBeVisible(); - // Tapping again dismisses the reveal. - await planned.click(); + // The example-expansion milestone completed: every scenario is ready. The + // planned-entry treatment (dimmed chip revealing its needs) stands by in + // the shelf code for the next expansion, but nothing renders it today. + for (const id of [ + "recipe-creator", + "appointment-booking", + "project-deletion", + "support-triage", + "onboarding", + "hotel-reservations", + ]) { + await expect(page.getByTestId(`scenario-${id}`)).not.toContainText("(planned)"); + } await expect(page.getByTestId("planned-needs")).toHaveCount(0); + // The last-flipped scenario actually selects. + await page.getByTestId("scenario-hotel-reservations").click(); + await expect(page.getByTestId("fixture-argues-back")).toBeVisible(); }); test("restyle themes the active scenario's own surface", async ({ page }) => { diff --git a/packages/contracts/astryx.dspack.json b/packages/contracts/astryx.dspack.json index 8694c66..7fe0f7f 100644 --- a/packages/contracts/astryx.dspack.json +++ b/packages/contracts/astryx.dspack.json @@ -568,6 +568,11 @@ "id": "data-collection", "name": "Data collection", "description": "Surfaces that ask the user for information and submit it — signup, contact, profile edits, an RSVP. The ask is stated, the fields are labeled, and the submit action is explicit: at minimum, something to fill in and a way to send it." + }, + { + "id": "transactional-review", + "name": "Transactional review", + "description": "Surfaces that present alternatives for a consequential choice — plans, offers, bookings, products — compared on stated attributes and committed with an explicit action. Options are named, comparable, and at least two; the commitment is a labeled action." } ], "rules": [ @@ -765,6 +770,128 @@ "structure" ], "x-source": "https://github.com/facebook/astryx/blob/v0.1.4/packages/core/src/TextInput/TextInput.doc.mjs" + }, + { + "id": "rule.review-presents-options", + "type": "component-choice", + "severity": "must", + "appliesTo": { + "intents": [ + "transactional-review" + ] + }, + "require": [ + "list", + "selectable-card", + "button" + ], + "rationale": "SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4", + "examples": [ + "ex.plan-comparison" + ], + "tags": [ + "structure" + ], + "x-source": "https://github.com/facebook/astryx/blob/v0.1.4/packages/core/src/SelectableCard/SelectableCard.doc.mjs" + }, + { + "id": "rule.review-offers-alternatives", + "type": "required-composition", + "severity": "must", + "appliesTo": { + "intents": [ + "transactional-review" + ] + }, + "component": "list", + "requiredSubComponents": [ + { + "id": "selectable-card", + "min": 2 + } + ], + "rationale": "A choice of one is not a comparison: a review's option list carries at least two selectable cards. (Under this intent, lists are option groups; the floor applies to every list on the surface. Exact counts and upper bounds are inexpressible in current rule types and remain guidance.)", + "examples": [ + "ex.plan-comparison" + ], + "tags": [ + "structure" + ], + "x-source": "https://github.com/facebook/astryx/blob/v0.1.4/packages/core/src/List/List.doc.mjs" + }, + { + "id": "rule.option-states-attributes", + "type": "required-composition", + "severity": "must", + "appliesTo": { + "intents": [ + "transactional-review" + ] + }, + "component": "selectable-card", + "requiredSubComponents": [ + { + "id": "metadata-list", + "min": 1 + } + ], + "rationale": "\"MetadataList displays key-value pairs for object attributes\" — an option you can compare states what it offers as structured attributes, not free prose. Every selectable card under this intent carries a metadata list. — Astryx MetadataList docs, v0.1.4", + "examples": [ + "ex.plan-comparison" + ], + "tags": [ + "structure" + ], + "x-source": "https://github.com/facebook/astryx/blob/v0.1.4/packages/core/src/MetadataList/MetadataList.doc.mjs" + }, + { + "id": "rule.metadata-carries-items", + "type": "required-props", + "severity": "must", + "appliesTo": { + "intents": [ + "transactional-review" + ] + }, + "component": "metadata-list", + "requiredProps": [ + { + "prop": "items" + } + ], + "rationale": "The catalog's MetadataList is data-driven: attributes are `{ label, value }` records. A metadata list without its items renders an empty shell, attributes nowhere to be seen — the same class of failure rule.table-carries-data guards against for tables.", + "examples": [ + "ex.plan-comparison" + ], + "tags": [ + "structure" + ], + "x-source": "https://github.com/facebook/astryx/blob/v0.1.4/packages/core/src/MetadataList/MetadataList.doc.mjs" + }, + { + "id": "rule.option-carries-label", + "type": "required-props", + "severity": "must", + "appliesTo": { + "intents": [ + "transactional-review" + ] + }, + "component": "selectable-card", + "requiredProps": [ + { + "prop": "label" + } + ], + "rationale": "Every option is named: SelectableCard's label is how an alternative is identified, compared, and announced. An unnamed option cannot be chosen deliberately. — Astryx SelectableCard docs, v0.1.4", + "examples": [ + "ex.plan-comparison" + ], + "tags": [ + "accessibility", + "structure" + ], + "x-source": "https://github.com/facebook/astryx/blob/v0.1.4/packages/core/src/SelectableCard/SelectableCard.doc.mjs" } ], "examples": [ @@ -1192,6 +1319,126 @@ ] } } + }, + { + "id": "ex.plan-comparison", + "intent": "transactional-review", + "name": "Plan comparison", + "prompt": "compare the available plans and choose one", + "description": "Two named plans as selectable cards in a list, each stating its attributes as a metadata list, with one explicit committing action. Deliberately not a hotel: the intent is domain-neutral, and the scenario that validates it comes from a different domain.", + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "transactional-review", + "root": { + "component": "card", + "id": "compare-card", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3" + }, + "text": "Choose a plan" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body" + }, + "text": "Two plans, compared on what they include. Pick one to continue." + }, + { + "component": "list", + "id": "plans", + "props": { + "density": "balanced" + }, + "children": [ + { + "component": "selectable-card", + "id": "plan-basic", + "props": { + "label": "Basic", + "isSelected": false + }, + "children": [ + { + "component": "metadata-list", + "id": "basic-attrs", + "props": { + "items": [ + { + "label": "Price", + "value": "$5/mo" + }, + { + "label": "Storage", + "value": "50 GB" + }, + { + "label": "Support", + "value": "Community" + } + ] + } + } + ] + }, + { + "component": "selectable-card", + "id": "plan-pro", + "props": { + "label": "Pro", + "isSelected": true, + "variant": "blue" + }, + "children": [ + { + "component": "metadata-list", + "id": "pro-attrs", + "props": { + "items": [ + { + "label": "Price", + "value": "$12/mo" + }, + { + "label": "Storage", + "value": "1 TB" + }, + { + "label": "Support", + "value": "Priority" + } + ] + } + }, + { + "component": "badge", + "id": "pro-badge", + "props": { + "variant": "info", + "label": "Most popular" + } + } + ] + } + ] + }, + { + "component": "button", + "id": "choose", + "props": { + "label": "Choose plan", + "variant": "primary" + } + } + ] + } + } } ] } diff --git a/packages/replay/fixtures/fixture-015.json b/packages/replay/fixtures/fixture-015.json new file mode 100644 index 0000000..0d4e9ee --- /dev/null +++ b/packages/replay/fixtures/fixture-015.json @@ -0,0 +1,846 @@ +{ + "replayFixture": "0.1", + "id": "fixture-015", + "name": "Clean hotel comparison", + "recordedAt": "2026-07-21T23:16:46.966Z", + "mode": "live", + "adapterId": "ollama:gpt-oss:latest", + "intent": "transactional-review", + "prompt": "Find a hotel in Lisbon for two nights in September.", + "events": [ + { + "atMs": 2, + "event": { + "type": "RUN_STARTED", + "threadId": "studio", + "runId": "fixture-015" + } + }, + { + "atMs": 2, + "event": { + "type": "CUSTOM", + "name": "dspack.run.start", + "value": { + "intent": "transactional-review", + "prompt": "Find a hotel in Lisbon for two nights in September.", + "adapterId": "ollama:gpt-oss:latest", + "ruleIds": [ + "rule.alertdialog-carries-content", + "rule.alertdialog-action-label-specific", + "rule.button-carries-label", + "rule.input-carries-label", + "rule.dialog-no-nested-overlays", + "rule.review-presents-options", + "rule.review-offers-alternatives", + "rule.option-states-attributes", + "rule.metadata-carries-items", + "rule.option-carries-label" + ] + } + } + }, + { + "atMs": 97472, + "event": { + "type": "STEP_STARTED", + "stepName": "attempt-0" + } + }, + { + "atMs": 97472, + "event": { + "type": "CUSTOM", + "name": "dspack.gates", + "value": { + "index": 0, + "model": "gpt-oss:latest", + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "transactional-review", + "root": { + "component": "card", + "id": "hotel-search", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3", + "as": "p" + }, + "text": "Find a hotel in Lisbon" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body", + "as": "p" + }, + "text": "Choose from the options below for your two-night stay in September." + }, + { + "component": "list", + "id": "hotels", + "props": { + "density": "balanced" + }, + "children": [ + { + "component": "selectable-card", + "props": { + "label": "Hotel Belém", + "isSelected": false + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$120" + }, + { + "label": "Rating", + "value": "4.5/5" + }, + { + "label": "Distance to city center", + "value": "0.3 km" + } + ] + } + }, + { + "component": "badge", + "id": "belém-badge", + "props": { + "variant": "info", + "label": "Best value" + } + } + ] + }, + { + "component": "selectable-card", + "props": { + "label": "Lisbon Marriott", + "isSelected": true, + "variant": "blue" + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$200" + }, + { + "label": "Rating", + "value": "4.8/5" + }, + { + "label": "Distance to city center", + "value": "0.1 km" + } + ] + } + }, + { + "component": "badge", + "id": "marriott-badge", + "props": { + "variant": "info", + "label": "Most popular" + } + } + ] + }, + { + "component": "selectable-card", + "props": { + "label": "Lisbon Boutique Hotel", + "isSelected": false + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$150" + }, + { + "label": "Rating", + "value": "4.6/5" + }, + { + "label": "Distance to city center", + "value": "0.2 km" + } + ] + } + }, + { + "component": "badge", + "id": "boutique-badge", + "props": { + "variant": "info", + "label": "Boutique" + } + } + ] + } + ] + }, + { + "component": "button", + "id": "book-now", + "props": { + "label": "Book now", + "variant": "primary" + } + } + ] + } + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + } + }, + { + "atMs": 97472, + "event": { + "type": "STEP_FINISHED", + "stepName": "attempt-0" + } + }, + { + "atMs": 97554, + "event": { + "type": "CUSTOM", + "name": "dspack.emit", + "value": { + "validations": [ + { + "a2uiVersion": "0.9.1", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + }, + { + "a2uiVersion": "1.0", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + } + ], + "warnings": [ + { + "code": "surface-prop-dropped", + "message": "$.root.children[0]: prop 'as' on 'text' has no A2UI projection; dropped." + }, + { + "code": "surface-prop-dropped", + "message": "$.root.children[1]: prop 'as' on 'text' has no A2UI projection; dropped." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[1]: node id 'selectablecard' already used; emitted as 'selectablecard_2'." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[1].children[0]: node id 'metadatalist' already used; emitted as 'metadatalist_2'." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[2]: node id 'selectablecard' already used; emitted as 'selectablecard_3'." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[2].children[0]: node id 'metadatalist' already used; emitted as 'metadatalist_3'." + }, + { + "code": "surface-synthesized-action", + "message": "$.root.children[3]: A2UI requires a declarative action on Button; synthesized event 'book_now'." + }, + { + "code": "surface-synthesized-wrap", + "message": "$.root: 4 children wrapped in a synthesized Column ('root_col') — the target slot takes a single child." + } + ] + } + } + }, + { + "atMs": 97554, + "event": { + "type": "TOOL_CALL_START", + "toolCallId": "fixture-015-a2ui-0", + "toolCallName": "generate_a2ui" + } + }, + { + "atMs": 97554, + "event": { + "type": "TOOL_CALL_ARGS", + "toolCallId": "fixture-015-a2ui-0", + "delta": "{\"surfaceId\":\"transactional_review\"}" + } + }, + { + "atMs": 97554, + "event": { + "type": "TOOL_CALL_END", + "toolCallId": "fixture-015-a2ui-0" + } + }, + { + "atMs": 97554, + "event": { + "type": "TOOL_CALL_RESULT", + "messageId": "fixture-015-a2ui-0-result", + "toolCallId": "fixture-015-a2ui-0", + "content": "{\"a2ui_operations\":[{\"version\":\"v0.9\",\"createSurface\":{\"surfaceId\":\"transactional_review\",\"catalogId\":\"https://aestheticfunction.github.io/dspack-studio/catalogs/astryx/v0_9_1/catalog.json\",\"theme\":{\"agentDisplayName\":\"Astryx via dspack\"}}},{\"version\":\"v0.9\",\"updateComponents\":{\"surfaceId\":\"transactional_review\",\"components\":[{\"id\":\"root\",\"component\":\"Card\",\"child\":\"root_col\"},{\"id\":\"title\",\"component\":\"Text\",\"variant\":\"h2\",\"text\":\"Find a hotel in Lisbon\"},{\"id\":\"intro\",\"component\":\"Text\",\"variant\":\"body\",\"text\":\"Choose from the options below for your two-night stay in September.\"},{\"id\":\"hotels\",\"component\":\"List\",\"density\":\"balanced\",\"children\":[\"selectablecard\",\"selectablecard_2\",\"selectablecard_3\"]},{\"id\":\"selectablecard\",\"component\":\"SelectableCard\",\"label\":\"Hotel Belém\",\"isSelected\":false,\"children\":[\"metadatalist\",\"bel_m_badge\"]},{\"id\":\"metadatalist\",\"component\":\"MetadataList\",\"items\":[{\"label\":\"Price per night\",\"value\":\"$120\"},{\"label\":\"Rating\",\"value\":\"4.5/5\"},{\"label\":\"Distance to city center\",\"value\":\"0.3 km\"}]},{\"id\":\"bel_m_badge\",\"component\":\"Badge\",\"variant\":\"info\",\"label\":\"Best value\"},{\"id\":\"selectablecard_2\",\"component\":\"SelectableCard\",\"label\":\"Lisbon Marriott\",\"isSelected\":true,\"variant\":\"blue\",\"children\":[\"metadatalist_2\",\"marriott_badge\"]},{\"id\":\"metadatalist_2\",\"component\":\"MetadataList\",\"items\":[{\"label\":\"Price per night\",\"value\":\"$200\"},{\"label\":\"Rating\",\"value\":\"4.8/5\"},{\"label\":\"Distance to city center\",\"value\":\"0.1 km\"}]},{\"id\":\"marriott_badge\",\"component\":\"Badge\",\"variant\":\"info\",\"label\":\"Most popular\"},{\"id\":\"selectablecard_3\",\"component\":\"SelectableCard\",\"label\":\"Lisbon Boutique Hotel\",\"isSelected\":false,\"children\":[\"metadatalist_3\",\"boutique_badge\"]},{\"id\":\"metadatalist_3\",\"component\":\"MetadataList\",\"items\":[{\"label\":\"Price per night\",\"value\":\"$150\"},{\"label\":\"Rating\",\"value\":\"4.6/5\"},{\"label\":\"Distance to city center\",\"value\":\"0.2 km\"}]},{\"id\":\"boutique_badge\",\"component\":\"Badge\",\"variant\":\"info\",\"label\":\"Boutique\"},{\"id\":\"book_now\",\"component\":\"Button\",\"label\":\"Book now\",\"variant\":\"primary\",\"action\":{\"event\":{\"name\":\"book_now\",\"context\":{}}}},{\"id\":\"root_col\",\"component\":\"Column\",\"children\":[\"title\",\"intro\",\"hotels\",\"book_now\"]}]}}]}" + } + }, + { + "atMs": 97554, + "event": { + "type": "CUSTOM", + "name": "dspack.audit", + "value": { + "outcome": "passed", + "exitCode": 0, + "report": { + "reportVersion": "1", + "createdAt": "2026-07-21T23:15:09.413Z", + "request": { + "prompt": "Find a hotel in Lisbon for two nights in September.", + "intent": "transactional-review", + "contract": { + "name": "Astryx", + "dspack": "0.4", + "sha256": "20692f4acc622e6ad220d3ae27d00dd5b610b2aead113cd8e03bea656cb55081" + } + }, + "generation": { + "adapterId": "ollama:gpt-oss:latest", + "schemaSha256": "cf0056e411fa9e7c4ad6f69fe03d84909a9fc9176e232dcdee7665049552a53d", + "maxRepairs": 2, + "ruleSteering": true, + "repairTemplate": "standard" + }, + "attempts": [ + { + "index": 0, + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "transactional-review", + "root": { + "component": "card", + "id": "hotel-search", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3", + "as": "p" + }, + "text": "Find a hotel in Lisbon" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body", + "as": "p" + }, + "text": "Choose from the options below for your two-night stay in September." + }, + { + "component": "list", + "id": "hotels", + "props": { + "density": "balanced" + }, + "children": [ + { + "component": "selectable-card", + "props": { + "label": "Hotel Belém", + "isSelected": false + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$120" + }, + { + "label": "Rating", + "value": "4.5/5" + }, + { + "label": "Distance to city center", + "value": "0.3 km" + } + ] + } + }, + { + "component": "badge", + "id": "belém-badge", + "props": { + "variant": "info", + "label": "Best value" + } + } + ] + }, + { + "component": "selectable-card", + "props": { + "label": "Lisbon Marriott", + "isSelected": true, + "variant": "blue" + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$200" + }, + { + "label": "Rating", + "value": "4.8/5" + }, + { + "label": "Distance to city center", + "value": "0.1 km" + } + ] + } + }, + { + "component": "badge", + "id": "marriott-badge", + "props": { + "variant": "info", + "label": "Most popular" + } + } + ] + }, + { + "component": "selectable-card", + "props": { + "label": "Lisbon Boutique Hotel", + "isSelected": false + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$150" + }, + { + "label": "Rating", + "value": "4.6/5" + }, + { + "label": "Distance to city center", + "value": "0.2 km" + } + ] + } + }, + { + "component": "badge", + "id": "boutique-badge", + "props": { + "variant": "info", + "label": "Boutique" + } + } + ] + } + ] + }, + { + "component": "button", + "id": "book-now", + "props": { + "label": "Book now", + "variant": "primary" + } + } + ] + } + }, + "model": "gpt-oss:latest", + "usage": { + "inputTokens": 2660, + "outputTokens": 433 + }, + "meta": { + "provider": "ollama", + "total_duration": 97402082614, + "load_duration": 328866152, + "prompt_eval_duration": 71913000, + "eval_duration": 27888293000 + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + ], + "repairMessages": [], + "outcome": "passed", + "emitted": { + "target": "a2ui", + "surfaceMessages": { + "messages": [ + { + "version": "v0.9", + "createSurface": { + "surfaceId": "transactional_review", + "catalogId": "https://aestheticfunction.github.io/dspack-studio/catalogs/astryx/v0_9_1/catalog.json", + "theme": { + "agentDisplayName": "Astryx via dspack" + } + } + }, + { + "version": "v0.9", + "updateComponents": { + "surfaceId": "transactional_review", + "components": [ + { + "id": "root", + "component": "Card", + "child": "root_col" + }, + { + "id": "title", + "component": "Text", + "variant": "h2", + "text": "Find a hotel in Lisbon" + }, + { + "id": "intro", + "component": "Text", + "variant": "body", + "text": "Choose from the options below for your two-night stay in September." + }, + { + "id": "hotels", + "component": "List", + "density": "balanced", + "children": [ + "selectablecard", + "selectablecard_2", + "selectablecard_3" + ] + }, + { + "id": "selectablecard", + "component": "SelectableCard", + "label": "Hotel Belém", + "isSelected": false, + "children": [ + "metadatalist", + "bel_m_badge" + ] + }, + { + "id": "metadatalist", + "component": "MetadataList", + "items": [ + { + "label": "Price per night", + "value": "$120" + }, + { + "label": "Rating", + "value": "4.5/5" + }, + { + "label": "Distance to city center", + "value": "0.3 km" + } + ] + }, + { + "id": "bel_m_badge", + "component": "Badge", + "variant": "info", + "label": "Best value" + }, + { + "id": "selectablecard_2", + "component": "SelectableCard", + "label": "Lisbon Marriott", + "isSelected": true, + "variant": "blue", + "children": [ + "metadatalist_2", + "marriott_badge" + ] + }, + { + "id": "metadatalist_2", + "component": "MetadataList", + "items": [ + { + "label": "Price per night", + "value": "$200" + }, + { + "label": "Rating", + "value": "4.8/5" + }, + { + "label": "Distance to city center", + "value": "0.1 km" + } + ] + }, + { + "id": "marriott_badge", + "component": "Badge", + "variant": "info", + "label": "Most popular" + }, + { + "id": "selectablecard_3", + "component": "SelectableCard", + "label": "Lisbon Boutique Hotel", + "isSelected": false, + "children": [ + "metadatalist_3", + "boutique_badge" + ] + }, + { + "id": "metadatalist_3", + "component": "MetadataList", + "items": [ + { + "label": "Price per night", + "value": "$150" + }, + { + "label": "Rating", + "value": "4.6/5" + }, + { + "label": "Distance to city center", + "value": "0.2 km" + } + ] + }, + { + "id": "boutique_badge", + "component": "Badge", + "variant": "info", + "label": "Boutique" + }, + { + "id": "book_now", + "component": "Button", + "label": "Book now", + "variant": "primary", + "action": { + "event": { + "name": "book_now", + "context": {} + } + } + }, + { + "id": "root_col", + "component": "Column", + "children": [ + "title", + "intro", + "hotels", + "book_now" + ] + } + ] + } + } + ] + }, + "warnings": [ + { + "code": "surface-prop-dropped", + "message": "$.root.children[0]: prop 'as' on 'text' has no A2UI projection; dropped." + }, + { + "code": "surface-prop-dropped", + "message": "$.root.children[1]: prop 'as' on 'text' has no A2UI projection; dropped." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[1]: node id 'selectablecard' already used; emitted as 'selectablecard_2'." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[1].children[0]: node id 'metadatalist' already used; emitted as 'metadatalist_2'." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[2]: node id 'selectablecard' already used; emitted as 'selectablecard_3'." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[2].children[0]: node id 'metadatalist' already used; emitted as 'metadatalist_3'." + }, + { + "code": "surface-synthesized-action", + "message": "$.root.children[3]: A2UI requires a declarative action on Button; synthesized event 'book_now'." + }, + { + "code": "surface-synthesized-wrap", + "message": "$.root: 4 children wrapped in a synthesized Column ('root_col') — the target slot takes a single child." + } + ], + "validations": [ + { + "a2uiVersion": "0.9.1", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + }, + { + "a2uiVersion": "1.0", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + } + ] + }, + "timings": { + "totalMs": 97553 + } + } + } + } + }, + { + "atMs": 97554, + "event": { + "type": "RUN_FINISHED", + "threadId": "studio", + "runId": "fixture-015" + } + } + ] +} diff --git a/packages/replay/fixtures/fixture-016.json b/packages/replay/fixtures/fixture-016.json new file mode 100644 index 0000000..720471c --- /dev/null +++ b/packages/replay/fixtures/fixture-016.json @@ -0,0 +1,864 @@ +{ + "replayFixture": "0.1", + "id": "fixture-016", + "name": "Comparisons argue back", + "recordedAt": "2026-07-21T23:18:25.598Z", + "mode": "live", + "adapterId": "ollama:gpt-oss:latest", + "intent": "transactional-review", + "prompt": "Just tell me which Lisbon hotel is best in a short paragraph. No cards, no lists, no options — just your one recommendation.", + "events": [ + { + "atMs": 2, + "event": { + "type": "RUN_STARTED", + "threadId": "studio", + "runId": "fixture-016" + } + }, + { + "atMs": 2, + "event": { + "type": "CUSTOM", + "name": "dspack.run.start", + "value": { + "intent": "transactional-review", + "prompt": "Just tell me which Lisbon hotel is best in a short paragraph. No cards, no lists, no options — just your one recommendation.", + "adapterId": "ollama:gpt-oss:latest", + "ruleIds": [ + "rule.alertdialog-carries-content", + "rule.alertdialog-action-label-specific", + "rule.button-carries-label", + "rule.input-carries-label", + "rule.dialog-no-nested-overlays", + "rule.review-presents-options", + "rule.review-offers-alternatives", + "rule.option-states-attributes", + "rule.metadata-carries-items", + "rule.option-carries-label" + ] + } + } + }, + { + "atMs": 16035, + "event": { + "type": "STEP_STARTED", + "stepName": "attempt-0" + } + }, + { + "atMs": 16035, + "event": { + "type": "CUSTOM", + "name": "dspack.gates", + "value": { + "index": 0, + "model": "gpt-oss:latest", + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "transactional-review", + "root": { + "component": "text", + "id": "refusal", + "props": { + "type": "body" + }, + "text": "I’m sorry, but I can’t comply with that request." + } + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "FAIL" + } + ], + "findings": [ + { + "ruleId": "rule.review-presents-options", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'list' does not appear in the surface.", + "rationale": "SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.plan-comparison" + ] + }, + { + "ruleId": "rule.review-presents-options", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'selectable-card' does not appear in the surface.", + "rationale": "SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.plan-comparison" + ] + }, + { + "ruleId": "rule.review-presents-options", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'button' does not appear in the surface.", + "rationale": "SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.plan-comparison" + ] + } + ] + } + } + }, + { + "atMs": 16035, + "event": { + "type": "STEP_FINISHED", + "stepName": "attempt-0" + } + }, + { + "atMs": 16035, + "event": { + "type": "CUSTOM", + "name": "dspack.repair", + "value": { + "index": 0, + "message": "Your surface violates 3 governance rule finding(s) of the \"Astryx\" design system:\n\nViolation 1: [rule.review-presents-options / must]\n At: $.root (component: surface)\n Required component 'list' does not appear in the surface.\n Why: SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4\nViolation 2: [rule.review-presents-options / must]\n At: $.root (component: surface)\n Required component 'selectable-card' does not appear in the surface.\n Why: SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4\nViolation 3: [rule.review-presents-options / must]\n At: $.root (component: surface)\n Required component 'button' does not appear in the surface.\n Why: SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4\n\nA correct example (ex.plan-comparison): {\"dspackSurface\":\"0.1\",\"system\":\"Astryx\",\"intent\":\"transactional-review\",\"root\":{\"component\":\"card\",\"id\":\"compare-card\",\"children\":[{\"component\":\"text\",\"id\":\"title\",\"props\":{\"type\":\"display-3\"},\"text\":\"Choose a plan\"},{\"component\":\"text\",\"id\":\"intro\",\"props\":{\"type\":\"body\"},\"text\":\"Two plans, compared on what they include. Pick one to continue.\"},{\"component\":\"list\",\"id\":\"plans\",\"props\":{\"density\":\"balanced\"},\"children\":[{\"component\":\"selectable-card\",\"id\":\"plan-basic\",\"props\":{\"label\":\"Basic\",\"isSelected\":false},\"children\":[{\"component\":\"metadata-list\",\"id\":\"basic-attrs\",\"props\":{\"items\":[{\"label\":\"Price\",\"value\":\"$5/mo\"},{\"label\":\"Storage\",\"value\":\"50 GB\"},{\"label\":\"Support\",\"value\":\"Community\"}]}}]},{\"component\":\"selectable-card\",\"id\":\"plan-pro\",\"props\":{\"label\":\"Pro\",\"isSelected\":true,\"variant\":\"blue\"},\"children\":[{\"component\":\"metadata-list\",\"id\":\"pro-attrs\",\"props\":{\"items\":[{\"label\":\"Price\",\"value\":\"$12/mo\"},{\"label\":\"Storage\",\"value\":\"1 TB\"},{\"label\":\"Support\",\"value\":\"Priority\"}]}},{\"component\":\"badge\",\"id\":\"pro-badge\",\"props\":{\"variant\":\"info\",\"label\":\"Most popular\"}}]}]},{\"component\":\"button\",\"id\":\"choose\",\"props\":{\"label\":\"Choose plan\",\"variant\":\"primary\"}}]}}\n\nProduce a corrected dspack surface document that fixes every violation above.\nDo not change parts of the surface that were not flagged.\nRespond with a single JSON object conforming to the provided schema. No commentary." + } + } + }, + { + "atMs": 97993, + "event": { + "type": "STEP_STARTED", + "stepName": "attempt-1" + } + }, + { + "atMs": 97993, + "event": { + "type": "CUSTOM", + "name": "dspack.gates", + "value": { + "index": 1, + "model": "gpt-oss:latest", + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "transactional-review", + "root": { + "component": "card", + "id": "compare-card", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3" + }, + "text": "Choose a Lisbon Hotel" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body" + }, + "text": "Compare two top hotels in Lisbon. Pick one to book." + }, + { + "component": "list", + "id": "hotels", + "props": { + "density": "balanced" + }, + "children": [ + { + "component": "selectable-card", + "props": { + "label": "Ritz-Carlton Lisboa", + "isSelected": false + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$350" + }, + { + "label": "Rating", + "value": "4.8/5" + }, + { + "label": "Location", + "value": "Downtown" + } + ] + } + } + ] + }, + { + "component": "selectable-card", + "props": { + "label": "Mandarin Oriental, Lisbon", + "isSelected": true, + "variant": "blue" + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$280" + }, + { + "label": "Rating", + "value": "4.7/5" + }, + { + "label": "Location", + "value": "Alfama" + } + ] + } + } + ] + } + ] + }, + { + "component": "button", + "id": "choose", + "props": { + "label": "Book Hotel", + "variant": "primary" + } + } + ] + } + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + } + }, + { + "atMs": 97993, + "event": { + "type": "STEP_FINISHED", + "stepName": "attempt-1" + } + }, + { + "atMs": 98078, + "event": { + "type": "CUSTOM", + "name": "dspack.emit", + "value": { + "validations": [ + { + "a2uiVersion": "0.9.1", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + }, + { + "a2uiVersion": "1.0", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + } + ], + "warnings": [ + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[1]: node id 'selectablecard' already used; emitted as 'selectablecard_2'." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[1].children[0]: node id 'metadatalist' already used; emitted as 'metadatalist_2'." + }, + { + "code": "surface-synthesized-action", + "message": "$.root.children[3]: A2UI requires a declarative action on Button; synthesized event 'choose'." + }, + { + "code": "surface-synthesized-wrap", + "message": "$.root: 4 children wrapped in a synthesized Column ('root_col') — the target slot takes a single child." + } + ] + } + } + }, + { + "atMs": 98078, + "event": { + "type": "TOOL_CALL_START", + "toolCallId": "fixture-016-a2ui-0", + "toolCallName": "generate_a2ui" + } + }, + { + "atMs": 98078, + "event": { + "type": "TOOL_CALL_ARGS", + "toolCallId": "fixture-016-a2ui-0", + "delta": "{\"surfaceId\":\"transactional_review\"}" + } + }, + { + "atMs": 98078, + "event": { + "type": "TOOL_CALL_END", + "toolCallId": "fixture-016-a2ui-0" + } + }, + { + "atMs": 98078, + "event": { + "type": "TOOL_CALL_RESULT", + "messageId": "fixture-016-a2ui-0-result", + "toolCallId": "fixture-016-a2ui-0", + "content": "{\"a2ui_operations\":[{\"version\":\"v0.9\",\"createSurface\":{\"surfaceId\":\"transactional_review\",\"catalogId\":\"https://aestheticfunction.github.io/dspack-studio/catalogs/astryx/v0_9_1/catalog.json\",\"theme\":{\"agentDisplayName\":\"Astryx via dspack\"}}},{\"version\":\"v0.9\",\"updateComponents\":{\"surfaceId\":\"transactional_review\",\"components\":[{\"id\":\"root\",\"component\":\"Card\",\"child\":\"root_col\"},{\"id\":\"title\",\"component\":\"Text\",\"variant\":\"h2\",\"text\":\"Choose a Lisbon Hotel\"},{\"id\":\"intro\",\"component\":\"Text\",\"variant\":\"body\",\"text\":\"Compare two top hotels in Lisbon. Pick one to book.\"},{\"id\":\"hotels\",\"component\":\"List\",\"density\":\"balanced\",\"children\":[\"selectablecard\",\"selectablecard_2\"]},{\"id\":\"selectablecard\",\"component\":\"SelectableCard\",\"label\":\"Ritz-Carlton Lisboa\",\"isSelected\":false,\"children\":[\"metadatalist\"]},{\"id\":\"metadatalist\",\"component\":\"MetadataList\",\"items\":[{\"label\":\"Price per night\",\"value\":\"$350\"},{\"label\":\"Rating\",\"value\":\"4.8/5\"},{\"label\":\"Location\",\"value\":\"Downtown\"}]},{\"id\":\"selectablecard_2\",\"component\":\"SelectableCard\",\"label\":\"Mandarin Oriental, Lisbon\",\"isSelected\":true,\"variant\":\"blue\",\"children\":[\"metadatalist_2\"]},{\"id\":\"metadatalist_2\",\"component\":\"MetadataList\",\"items\":[{\"label\":\"Price per night\",\"value\":\"$280\"},{\"label\":\"Rating\",\"value\":\"4.7/5\"},{\"label\":\"Location\",\"value\":\"Alfama\"}]},{\"id\":\"choose\",\"component\":\"Button\",\"label\":\"Book Hotel\",\"variant\":\"primary\",\"action\":{\"event\":{\"name\":\"choose\",\"context\":{}}}},{\"id\":\"root_col\",\"component\":\"Column\",\"children\":[\"title\",\"intro\",\"hotels\",\"choose\"]}]}}]}" + } + }, + { + "atMs": 98078, + "event": { + "type": "CUSTOM", + "name": "dspack.audit", + "value": { + "outcome": "passed", + "exitCode": 0, + "report": { + "reportVersion": "1", + "createdAt": "2026-07-21T23:16:47.519Z", + "request": { + "prompt": "Just tell me which Lisbon hotel is best in a short paragraph. No cards, no lists, no options — just your one recommendation.", + "intent": "transactional-review", + "contract": { + "name": "Astryx", + "dspack": "0.4", + "sha256": "20692f4acc622e6ad220d3ae27d00dd5b610b2aead113cd8e03bea656cb55081" + } + }, + "generation": { + "adapterId": "ollama:gpt-oss:latest", + "schemaSha256": "cf0056e411fa9e7c4ad6f69fe03d84909a9fc9176e232dcdee7665049552a53d", + "maxRepairs": 2, + "ruleSteering": true, + "repairTemplate": "standard" + }, + "attempts": [ + { + "index": 0, + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "transactional-review", + "root": { + "component": "text", + "id": "refusal", + "props": { + "type": "body" + }, + "text": "I’m sorry, but I can’t comply with that request." + } + }, + "model": "gpt-oss:latest", + "usage": { + "inputTokens": 2240, + "outputTokens": 56 + }, + "meta": { + "provider": "ollama", + "total_duration": 15969289995, + "load_duration": 343409205, + "prompt_eval_duration": 75980000, + "eval_duration": 3522120000 + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "FAIL" + } + ], + "findings": [ + { + "ruleId": "rule.review-presents-options", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'list' does not appear in the surface.", + "rationale": "SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.plan-comparison" + ] + }, + { + "ruleId": "rule.review-presents-options", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'selectable-card' does not appear in the surface.", + "rationale": "SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.plan-comparison" + ] + }, + { + "ruleId": "rule.review-presents-options", + "type": "component-choice", + "requirement": "must", + "level": "error", + "message": "Required component 'button' does not appear in the surface.", + "rationale": "SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4", + "location": { + "path": "$.root", + "component": "surface" + }, + "exampleIds": [ + "ex.plan-comparison" + ] + } + ] + }, + { + "index": 1, + "surface": { + "dspackSurface": "0.1", + "system": "Astryx", + "intent": "transactional-review", + "root": { + "component": "card", + "id": "compare-card", + "children": [ + { + "component": "text", + "id": "title", + "props": { + "type": "display-3" + }, + "text": "Choose a Lisbon Hotel" + }, + { + "component": "text", + "id": "intro", + "props": { + "type": "body" + }, + "text": "Compare two top hotels in Lisbon. Pick one to book." + }, + { + "component": "list", + "id": "hotels", + "props": { + "density": "balanced" + }, + "children": [ + { + "component": "selectable-card", + "props": { + "label": "Ritz-Carlton Lisboa", + "isSelected": false + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$350" + }, + { + "label": "Rating", + "value": "4.8/5" + }, + { + "label": "Location", + "value": "Downtown" + } + ] + } + } + ] + }, + { + "component": "selectable-card", + "props": { + "label": "Mandarin Oriental, Lisbon", + "isSelected": true, + "variant": "blue" + }, + "children": [ + { + "component": "metadata-list", + "props": { + "items": [ + { + "label": "Price per night", + "value": "$280" + }, + { + "label": "Rating", + "value": "4.7/5" + }, + { + "label": "Location", + "value": "Alfama" + } + ] + } + } + ] + } + ] + }, + { + "component": "button", + "id": "choose", + "props": { + "label": "Book Hotel", + "variant": "primary" + } + } + ] + } + }, + "model": "gpt-oss:latest", + "usage": { + "inputTokens": 3478, + "outputTokens": 280 + }, + "meta": { + "provider": "ollama", + "total_duration": 81919952749, + "load_duration": 405094360, + "prompt_eval_duration": 64183000, + "eval_duration": 18296019000 + }, + "gates": [ + { + "gate": "S1", + "name": "surface-schema", + "status": "PASS" + }, + { + "gate": "S2", + "name": "contract-vocabulary", + "status": "PASS" + }, + { + "gate": "S3", + "name": "governance", + "status": "PASS" + } + ], + "findings": [] + } + ], + "repairMessages": [ + "Your surface violates 3 governance rule finding(s) of the \"Astryx\" design system:\n\nViolation 1: [rule.review-presents-options / must]\n At: $.root (component: surface)\n Required component 'list' does not appear in the surface.\n Why: SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4\nViolation 2: [rule.review-presents-options / must]\n At: $.root (component: surface)\n Required component 'selectable-card' does not appear in the surface.\n Why: SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4\nViolation 3: [rule.review-presents-options / must]\n At: $.root (component: surface)\n Required component 'button' does not appear in the surface.\n Why: SelectableCard is for \"plan pickers, filter chips, or option grids\"; List renders \"ordered or unordered groups of related content\" — \"lists imply a meaningful collection\"; Button \"triggers an action when clicked … any interaction that needs a clear call to action\". A review surface presents its options as a collection of named selectable cards with an explicit committing action — not prose, not a bare recommendation. — Astryx SelectableCard, List, and Button docs, v0.1.4\n\nA correct example (ex.plan-comparison): {\"dspackSurface\":\"0.1\",\"system\":\"Astryx\",\"intent\":\"transactional-review\",\"root\":{\"component\":\"card\",\"id\":\"compare-card\",\"children\":[{\"component\":\"text\",\"id\":\"title\",\"props\":{\"type\":\"display-3\"},\"text\":\"Choose a plan\"},{\"component\":\"text\",\"id\":\"intro\",\"props\":{\"type\":\"body\"},\"text\":\"Two plans, compared on what they include. Pick one to continue.\"},{\"component\":\"list\",\"id\":\"plans\",\"props\":{\"density\":\"balanced\"},\"children\":[{\"component\":\"selectable-card\",\"id\":\"plan-basic\",\"props\":{\"label\":\"Basic\",\"isSelected\":false},\"children\":[{\"component\":\"metadata-list\",\"id\":\"basic-attrs\",\"props\":{\"items\":[{\"label\":\"Price\",\"value\":\"$5/mo\"},{\"label\":\"Storage\",\"value\":\"50 GB\"},{\"label\":\"Support\",\"value\":\"Community\"}]}}]},{\"component\":\"selectable-card\",\"id\":\"plan-pro\",\"props\":{\"label\":\"Pro\",\"isSelected\":true,\"variant\":\"blue\"},\"children\":[{\"component\":\"metadata-list\",\"id\":\"pro-attrs\",\"props\":{\"items\":[{\"label\":\"Price\",\"value\":\"$12/mo\"},{\"label\":\"Storage\",\"value\":\"1 TB\"},{\"label\":\"Support\",\"value\":\"Priority\"}]}},{\"component\":\"badge\",\"id\":\"pro-badge\",\"props\":{\"variant\":\"info\",\"label\":\"Most popular\"}}]}]},{\"component\":\"button\",\"id\":\"choose\",\"props\":{\"label\":\"Choose plan\",\"variant\":\"primary\"}}]}}\n\nProduce a corrected dspack surface document that fixes every violation above.\nDo not change parts of the surface that were not flagged.\nRespond with a single JSON object conforming to the provided schema. No commentary." + ], + "outcome": "passed", + "emitted": { + "target": "a2ui", + "surfaceMessages": { + "messages": [ + { + "version": "v0.9", + "createSurface": { + "surfaceId": "transactional_review", + "catalogId": "https://aestheticfunction.github.io/dspack-studio/catalogs/astryx/v0_9_1/catalog.json", + "theme": { + "agentDisplayName": "Astryx via dspack" + } + } + }, + { + "version": "v0.9", + "updateComponents": { + "surfaceId": "transactional_review", + "components": [ + { + "id": "root", + "component": "Card", + "child": "root_col" + }, + { + "id": "title", + "component": "Text", + "variant": "h2", + "text": "Choose a Lisbon Hotel" + }, + { + "id": "intro", + "component": "Text", + "variant": "body", + "text": "Compare two top hotels in Lisbon. Pick one to book." + }, + { + "id": "hotels", + "component": "List", + "density": "balanced", + "children": [ + "selectablecard", + "selectablecard_2" + ] + }, + { + "id": "selectablecard", + "component": "SelectableCard", + "label": "Ritz-Carlton Lisboa", + "isSelected": false, + "children": [ + "metadatalist" + ] + }, + { + "id": "metadatalist", + "component": "MetadataList", + "items": [ + { + "label": "Price per night", + "value": "$350" + }, + { + "label": "Rating", + "value": "4.8/5" + }, + { + "label": "Location", + "value": "Downtown" + } + ] + }, + { + "id": "selectablecard_2", + "component": "SelectableCard", + "label": "Mandarin Oriental, Lisbon", + "isSelected": true, + "variant": "blue", + "children": [ + "metadatalist_2" + ] + }, + { + "id": "metadatalist_2", + "component": "MetadataList", + "items": [ + { + "label": "Price per night", + "value": "$280" + }, + { + "label": "Rating", + "value": "4.7/5" + }, + { + "label": "Location", + "value": "Alfama" + } + ] + }, + { + "id": "choose", + "component": "Button", + "label": "Book Hotel", + "variant": "primary", + "action": { + "event": { + "name": "choose", + "context": {} + } + } + }, + { + "id": "root_col", + "component": "Column", + "children": [ + "title", + "intro", + "hotels", + "choose" + ] + } + ] + } + } + ] + }, + "warnings": [ + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[1]: node id 'selectablecard' already used; emitted as 'selectablecard_2'." + }, + { + "code": "surface-id-deduplicated", + "message": "$.root.children[2].children[1].children[0]: node id 'metadatalist' already used; emitted as 'metadatalist_2'." + }, + { + "code": "surface-synthesized-action", + "message": "$.root.children[3]: A2UI requires a declarative action on Button; synthesized event 'choose'." + }, + { + "code": "surface-synthesized-wrap", + "message": "$.root: 4 children wrapped in a synthesized Column ('root_col') — the target slot takes a single child." + } + ], + "validations": [ + { + "a2uiVersion": "0.9.1", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + }, + { + "a2uiVersion": "1.0", + "gates": [ + { + "gate": "A1", + "name": "schema-compile + no-external-ref", + "pass": true + }, + { + "gate": "A2", + "name": "catalog-shape", + "pass": true + }, + { + "gate": "A3", + "name": "instance", + "pass": true + } + ] + } + ] + }, + "timings": { + "totalMs": 98078 + } + } + } + } + }, + { + "atMs": 98078, + "event": { + "type": "RUN_FINISHED", + "threadId": "studio", + "runId": "fixture-016" + } + } + ] +} diff --git a/packages/scenarios/src/break-conditions.ts b/packages/scenarios/src/break-conditions.ts index 842df81..d9828c5 100644 --- a/packages/scenarios/src/break-conditions.ts +++ b/packages/scenarios/src/break-conditions.ts @@ -12,7 +12,7 @@ export interface BreakCondition { id: string; label: string; /** The scenario this condition belongs to; absent when scenarioIndependent. */ - scenarioId?: "project-deletion" | "appointment-booking" | "recipe-creator" | "support-triage" | "onboarding"; + scenarioId?: "project-deletion" | "appointment-booking" | "recipe-creator" | "support-triage" | "onboarding" | "hotel-reservations"; /** * True for conditions that belong to every scenario: pure client-side * demonstrations that never start a run or read scenario state. @@ -155,6 +155,22 @@ export const breakConditions: BreakCondition[] = [ note: "In this recorded real run the model shipped a text-only welcome with nothing to fill in; the form rule caught it and the repaired surface added a labeled email field and an explicit action.", }, }, + { + id: "review-without-options", + label: "a review with nothing to choose", + scenarioId: "hotel-reservations", + intent: "transactional-review", + kind: "governed-repair", + expected: + "S3 fails with rule.review-presents-options (the SelectableCard, List, and Button docs' own rationale, verbatim), a repair message is sent, and the repaired surface ships named options compared on their attributes.", + prompt: + "Just tell me which Lisbon hotel is best in a short paragraph. No cards, no lists, no options — just your one recommendation.", + recordedCatch: { + scenarioId: "hotel-reservations", + fixtureKey: "argues-back", + note: "In this recorded real run the model shipped a bare apology instead of options; the review rules caught a surface with nothing to choose and the repaired surface shipped two named hotels compared on their attributes.", + }, + }, { id: "malformed-import", label: "malformed session import", diff --git a/packages/scenarios/src/registry.ts b/packages/scenarios/src/registry.ts index bc4add5..53cdb1f 100644 --- a/packages/scenarios/src/registry.ts +++ b/packages/scenarios/src/registry.ts @@ -4,10 +4,11 @@ * (usually contract expansion — new intents/rules/examples are owner-authored * governance content, per the project plan). * - * Order is intentional: ready scenarios first (recipe co-editing leads as the - * default, then booking, then project deletion, then support triage, then - * signup/onboarding), planned ones grouped after. readyScenarios[0] is the - * studio's default scenario. + * Order is intentional: recipe co-editing leads as the default, then booking, + * project deletion, support triage, signup/onboarding, and hotel + * reservations. Every scenario is ready; the shelf's planned-entry treatment + * stands by for the next expansion. readyScenarios[0] is the studio's + * default scenario. */ import type { Scenario } from "./types"; import fixture001 from "@dspack-studio/replay/fixtures/fixture-001.json"; @@ -23,6 +24,8 @@ import fixture011 from "@dspack-studio/replay/fixtures/fixture-011.json"; import fixture012 from "@dspack-studio/replay/fixtures/fixture-012.json"; import fixture013 from "@dspack-studio/replay/fixtures/fixture-013.json"; import fixture014 from "@dspack-studio/replay/fixtures/fixture-014.json"; +import fixture015 from "@dspack-studio/replay/fixtures/fixture-015.json"; +import fixture016 from "@dspack-studio/replay/fixtures/fixture-016.json"; export const scenarios: Scenario[] = [ { @@ -199,12 +202,34 @@ export const scenarios: Scenario[] = [ { id: "hotel-reservations", name: "Hotel reservations", - tagline: "Search, compare, reserve, once the contract carries the vocabulary for it.", + tagline: "Search, compare, reserve: options as selectable cards, attributes as metadata, one explicit action.", intent: "transactional-review", - status: "planned", - needs: ["new design-system components for search, compare, and reserve", "a recorded real run to replay"], - seedPrompts: ["Find a hotel in Lisbon for two nights in September."], - fixtures: [], + status: "ready", + seedPrompts: [ + "Find a hotel in Lisbon for two nights in September.", + "Just tell me which Lisbon hotel is best in a short paragraph. No cards, no lists, no options — just your one recommendation.", + ], + breakItPrompts: [ + { + ruleId: "rule.review-presents-options", + prompt: + "Just tell me which Lisbon hotel is best in a short paragraph. No cards, no lists, no options — just your one recommendation.", + }, + ], + fixtures: [ + { + key: "argues-back", + label: "the interface argues back", + blurb: "One governed repair: asked for a single no-options recommendation, the model ships a bare apology; the review rules catch a surface with nothing to choose, and the repaired surface ships two named hotels compared on their attributes.", + fixture: fixture016, + }, + { + key: "clean", + label: "clean first pass", + blurb: "No violations: one attempt, straight through the gates to three named hotels — each a selectable card stating its attributes as metadata, badged, with one explicit booking action.", + fixture: fixture015, + }, + ], }, ]; diff --git a/playwright.production.config.ts b/playwright.production.config.ts index f266d6b..c837abb 100644 --- a/playwright.production.config.ts +++ b/playwright.production.config.ts @@ -32,6 +32,7 @@ export default defineConfig({ "replay.spec.ts", "support-triage.spec.ts", "onboarding.spec.ts", + "hotel-reservations.spec.ts", "alive.spec.ts", "permalinks.spec.ts", "tour-xray-wire.spec.ts",