You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(agents): two steps, data first: the user confirms the data before any plot - #12131
Owner decision of 2026-10-11: "Use with my data" runs in two steps. Step 1 ends with .confirm() once the preview and the bindings fit; only then does .create_plot() appear. The agents service enforces it: POST /v1/sessions/{sid}/dataset/confirm stores the confirmed dataset id, the pipeline answers not_ready with the new reason data_unconfirmed without it (before any model call), a new paste, a transcription's .use() or a binding change (the PUT route or the root's set_bindings tool) drops the confirmation, and a library switch keeps it. The session block tells the root the state; the root asks the user to confirm in the data panel and cannot confirm itself. The eval harness and the development fixture confirm on the developer's behalf.
The stream's done event now carries the session's bindings and data_confirmed after a run, so the page syncs its dropdowns and its step after a turn in which the root changed the bindings (a pre-existing gap the confirmation made matter); the BFF relays both fields and the new route.
Frontend: a one-line step header 1 data · 2 plot, the step-1 line with .confirm(), the confirmed line ✓ data fits · N rows · M columns with .create_plot(), readable text for data_unconfirmed, the new agent_data_confirmed analytics event; the mock BFF mirrors the route, the gate and the sync. Docs: the design doc's new section "Two steps: data first, then the plot" (the owner's rule and the planned data operations: rename, drop, set type, aggregate, sort), the request flow, docs/reference/api.md, agents/README.md, docs/reference/plausible.md.
Plan
N/A (owner decision in the session, 2026-10-11: "klingt gut so umsetzen" on the two-step design; the design section in docs/concepts/agent-network.md is the written form)
Browser against the branch's own service and API with the real judge model and the deployed renderer: paste a 6-row table → step 1 with .confirm() → ✓ data fits · 6 rows · 2 columns and .create_plot() → a plot shipped (needs_attention for long labels, a plot matter) → a binding change dropped the confirmation and showed step 1 again with .confirm() disabled until the value role is bound
Mock-driven browser check by the frontend delegate at desktop and phone width, light and dark: confirm, plot, binding change after the plot, refine while unconfirmed (data_unconfirmed readable), free text after a confirmation back to step 1
After the merge: deploy-agents, deploy-api and deploy-app green; the two-step flow once in the production chat
Rollout note
The three deploy triggers (deploy-agents, deploy-api, deploy-app) fire on the same merge commit and land within about five minutes of each other. In that window either order degrades gracefully for the admin-only chat: an old app gets not_ready on create_plot and recovers on reload; a new app's confirm call fails with 404 until the API revision is live, and the page shows the error under the button.
The session keeps the confirmed dataset id; POST /v1/sessions/{sid}/dataset/confirm
sets it, the pipeline answers not_ready: data_unconfirmed without it, a new dataset
or a binding change (the PUT route or the root's set_bindings tool) drops it, and a
library switch keeps it. The session block tells the root the state, the root asks
the user to confirm in the data panel, the eval harness and the development fixture
confirm on the developer's behalf. The design doc gets the section "Two steps: data
first, then the plot" with the planned data operations (owner decision of 2026-10-11).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
…state
After a run the stream's closing event carries the session's bindings and
whether the data is confirmed, so the page shows the bindings the root's
set_bindings tool chose and returns to step 1 when the confirmation dropped;
the BFF relays both fields.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
…nd the done sync
The data panel ends step 1 with .confirm() once the bindings are complete; the
confirmed line and .create_plot() follow. A new parse, a transcription or a
binding change takes the panel back to step 1; a library switch keeps the
confirmation. The done event's bindings and confirmation state sync the
dropdowns and the step after a turn in which the root changed the bindings.
A one-line header shows 1 data · 2 plot. The mock mirrors the confirm route,
the gate and the sync, and plays the root's set_bindings on a "swap" message.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Confirmation can become stale under concurrent state changes, and the enforcement cannot be deployed safely across the independently released services.
… bindings
A delayed confirm that read the old bindings and writes after a binding
change stores a fingerprint the session no longer matches, so the new
bindings stay unconfirmed; the explicit resets on a binding change go,
the fingerprint covers them (Copilot round 1 on #12131). The design doc
marks the done event's sync fields optional and names the pre-run
refusals that omit them.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Confirm data in the shared development fixture seeding path
agents/main.py:554
This confirms only the HTTP route’s _fixture_seed. The DevFixturePlugin used by adk web agents still calls agents/anyplot/dev_fixture.py:seed_delta, which stores the dataset and bindings without CONFIRMED; its first fixture plot will now return data_unconfirmed. Put the confirmation in the shared fixture seeding path (and assert view.data_confirmed in test_dev_fixture.py) so both advertised development-fixture entry points continue to run.
…ndings
The bindings are a set of role-to-column assignments, so they are sorted
before hashing; the same assignments in another order keep the confirmation
(Copilot round 2 on #12131).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
)
## Summary
- **The assistant changes your data in step 1.** The root agent's new
`prepare_data` tool chooses from a fixed set of pandas operations
(`rename`, `drop_columns`, `drop_rows`, `set_type`, `aggregate`, `sort`,
`pivot_longer`, `pivot_wider`, at most five per call) that the agents
service applies to the stored canonical CSV and stores as the next
dataset version (the last five kept for `POST .../dataset/undo`). The
model never writes data rows; `ToolSafety` validates the operations
against the same models first. The turn's `done` event carries the new
version with one change line per operation, unconfirmed, and the BFF
relays a `done` of up to 512 KiB so a wide dataset's change list always
reaches the page.
- **Step-1 summary and the page.** After every stored parse the page
sends the action `summarize_data`, so the assistant's
two-to-four-sentence description of the data (the profile now carries
`mean` and `median`) is the first turn in the thread; data requests in
words go through the composer before the confirmation. A `done` with
`dataset` replaces the preview and the bindings, takes the page back to
step 1 and shows a `# changed` block with the change lines and
`.undo()`. Two enum-only analytics events:
`agent_data_prepared{changes}`, `agent_data_undo{version}`.
- **Run queue.** A turn that starts while the data is unconfirmed cannot
render (the pipeline answers `not_ready`, nothing can confirm during a
turn), so it takes the slot but no start in the one-a-minute window: the
summary, a data request and the plot after the confirmation no longer
wait a minute for each other, while whole pipeline runs keep the minute
rule.
- **Prep eval.** `python -m agents.evals.prep` drives 17 German and
English data requests through `/v1` with the real model and scores the
root's `prepare_data` calls by exact operation match (80 % gate). Three
runs on 2026-10-11 with Claude Haiku 5.5 on `eu`: 16/17 before the
review fixes, 17/17 after them, 17/17 after the merge with #12132 (about
$0.03 each).
Built with a workflow (implementer, two adversarial verifiers, fix pass)
plus two delegates (frontend; the merge follow-up for #12132's parser
units). The verifiers' findings that hold are all fixed in `4fa98ae85`:
the `done` event cap, the `MM/DD/YYYY` swap scoped to parser-flagged
ambiguous columns, `12,5` never read as 125, JSON numbers as condition
values with validation lines on refusals, the long-format size cap,
`set_type text` kept through the re-parse (`text_columns`), the idle
sweep per session, the parser's per-column time flag carried through the
operations.
Left as designed, for the owner's eye: a rename's `name` is exempt from
the link rule like a condition's `value` (a column may be renamed to
what the user's table holds); `set_type datetime` without a format uses
pandas' per-cell inference; a `year` period re-parses as an integer
column. Two parser follow-ups found after the merge are a Todoist task
(`claude` label): thousands commas behind a currency (`$1,200` reads as
1.2 since #12132's unit rule, and a numeric column cannot be re-read),
and the unit staying out of the column name after `set_type number` on a
mixed column.
## Plan
Design: [Two steps: data first, then the
plot](docs/concepts/agent-network.md#two-steps-data-first-then-the-plot).
Order agreed with the owner on 2026-10-11: (1) the confirm gate
(#12131), (2) this PR, (3) the chat-first data layout, (4) the `.save()`
export.
## Test plan
- [x] `ruff check`, `ruff format --check`, `mypy api core agents`,
`pytest tests/unit tests/integration` (8,981 passed), `tools.changelog
check`
- [x] `cd app && yarn lint && yarn fm:check && yarn type-check && yarn
test` (805 passed) `&& yarn build`
- [x] Live prep eval 17/17 on the branch head (report in the session
scratchpad; reports are gitignored)
- [x] Browser walk-through against the mock BFF (desktop 1280 and phone
390, light and dark): paste → summary as the first turn → "das ist
prozent" → `# changed · version 2` with the change line → `.undo()` → `#
undone · back to version 1` → "gruppier nach x" → `.confirm()` →
`.create_plot()` renders
- [ ] After the deploy: paste the owner's CSV in production, ask "das
ist Prozent, nicht Anzahl" and "gruppier nach Monat", undo, confirm,
create the plot
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.confirm()once the preview and the bindings fit; only then does.create_plot()appear. The agents service enforces it:POST /v1/sessions/{sid}/dataset/confirmstores the confirmed dataset id, the pipeline answersnot_readywith the new reasondata_unconfirmedwithout it (before any model call), a new paste, a transcription's.use()or a binding change (thePUTroute or the root'sset_bindingstool) drops the confirmation, and a library switch keeps it. The session block tells the root the state; the root asks the user to confirm in the data panel and cannot confirm itself. The eval harness and the development fixture confirm on the developer's behalf.doneevent now carries the session'sbindingsanddata_confirmedafter a run, so the page syncs its dropdowns and its step after a turn in which the root changed the bindings (a pre-existing gap the confirmation made matter); the BFF relays both fields and the new route.1 data · 2 plot, the step-1 line with.confirm(), the confirmed line✓ data fits · N rows · M columnswith.create_plot(), readable text fordata_unconfirmed, the newagent_data_confirmedanalytics event; the mock BFF mirrors the route, the gate and the sync. Docs: the design doc's new section "Two steps: data first, then the plot" (the owner's rule and the planned data operations: rename, drop, set type, aggregate, sort), the request flow,docs/reference/api.md,agents/README.md,docs/reference/plausible.md.Plan
N/A (owner decision in the session, 2026-10-11: "klingt gut so umsetzen" on the two-step design; the design section in
docs/concepts/agent-network.mdis the written form)Test plan
ruff,mypy api core agents,pytest tests/unit/agents tests/unit/api/test_agent_router.py(4,325 passed),tools.changelog checkyarn lint,yarn fm:check,yarn type-check,yarn test(782 passed),yarn build.confirm()→✓ data fits · 6 rows · 2 columnsand.create_plot()→ a plot shipped (needs_attention for long labels, a plot matter) → a binding change dropped the confirmation and showed step 1 again with.confirm()disabled until the value role is bounddata_unconfirmedreadable), free text after a confirmation back to step 1deploy-agents,deploy-apianddeploy-appgreen; the two-step flow once in the production chatRollout note
The three deploy triggers (deploy-agents, deploy-api, deploy-app) fire on the same merge commit and land within about five minutes of each other. In that window either order degrades gracefully for the admin-only chat: an old app gets
not_readyon create_plot and recovers on reload; a new app's confirm call fails with 404 until the API revision is live, and the page shows the error under the button.