Skip to content

fix(agents): report the round's changes and keep the user's earlier wishes - #12170

Merged
MarkusNeusinger merged 3 commits into
mainfrom
fix/pipeline-changes-user-wishes
Oct 14, 2026
Merged

MarkusNeusinger merged 3 commits into
mainfrom
fix/pipeline-changes-user-wishes

Conversation

@MarkusNeusinger

Copy link
Copy Markdown
Owner

Summary

Fixes two bugs from the owner's live test of 2026-10-14 (Vienna weather data, line-timeseries, matplotlib).

  • The reported changes were the proofread's, not the round's. finish() took the changes from the shipped candidate's plan. In the self-review flow that plan is the proofread's, so its "already present" replaced the real changes. The root then told the user that "possibly nothing changed", although the plot was changed correctly.
    • note_changes() now collects the notes of every adapter call that changed the code text. A proofread or retry that changed nothing adds nothing.
    • Each candidate keeps a snapshot, and finish() reports the shipped candidate's notes. Ship-best of round 1 reports round 1's notes.
  • An earlier explicit wish was undone by the review loop. The user had set the title "Wien – Tageshöchsttemperatur 2025". Two changes later, a marker request rewrote it to "…Januar bis April 2025": the adapter's title rule, the self-review's SC-04 row and the reviewer's DQ-03 all pull a title toward the data range.
    • Each stored version now keeps the change requests of its lineage (CodeVersion.requests).
    • A change run passes them as user_requests to every adapter call (adaptation, retry, proofread, repair) and to the reviewer, fenced under "Earlier requests (the user's standing decisions, oldest first)".
    • The prompts say these win over the rubric and the checklist, are never a defect or a deduction, and are reverted only when the current request asks. The shared catalogue rubric is untouched.
    • Caps: 8 requests and 2,400 characters, the most recent first.

Plan

N/A

Test plan

  • 11 new tests: changes per round, ship-best, standing requests through adaptation, proofread and review, caps, and a fence canary. pytest tests/unit tests/integration: 10,067 passed. ruff, ruff format, mypy and the changelog check pass.
  • After merge: re-run the live sequence (title, second line, marker) and check that the title survives and the reply names the changes.
  • The prompt changes can only be verified on real model runs, so they get the live re-run above.

…requests

Two bugs from the owner's live test of 2026-10-14.

PlotResult.changes came from the plan of the attempt that produced the
shipped candidate, which in the live flow is the self-review. A proofread
that changed nothing wrote "already present" notes, which replaced the
round's real changes, and the root told the user the change might not have
run. Changes are now collected per adapter call whose plan changed the code
(an adaptation that changed nothing counts only while nothing else has),
snapshotted on each Candidate, so ship-best of an earlier round reports
that round's changes and a shipped repair keeps the user's request too.

A later change silently rewrote a title the user had chosen two turns
earlier: neither the adapter (whose title rule asks for a title describing
the data) nor the self-review checklist nor the catalogue reviewer (SC-04,
DQ-03) knew it was the user's choice. Each CodeVersion now keeps the change
requests it carries out; a change of it passes them as user_requests to
every adapter call and to the reviewer, fenced as user text, and the
adapter, reviewer and catalogue-reviewer prompts treat them and the current
request as standing decisions that are never defects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Copilot AI balanced review requested due to automatic review settings October 14, 2026 11:24
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Request precedence is ambiguous, and cumulative change reporting can silently omit later edits.

2 open findings
What changed in this PR

Fixes agent change reporting and preserves prior user requests across later plot revisions.

Changes:

  • Accumulates change notes per shipped candidate.
  • Propagates fenced request history through adapter and reviewer calls.
  • Adds regression tests and documentation.
File Description
agents/​anyplot/​pipeline.py Tracks change notes and standing requests.
agents/​anyplot/​services.py Stores request lineage per version.
agents/​anyplot/​schemas.py Defines request limits and fields.
agents/​anyplot/​prompts/​adapter.md Adds standing-request precedence guidance.
agents/​anyplot/​prompts/​reviewer.md Protects explicit user choices during review.
agents/​anyplot/​prompts/​reviewer_catalogue.md Exempts user choices from rubric deductions.
tests/​unit/​agents/​runtime/​test_pipeline.py Tests caps, fencing, and request rendering.
tests/​unit/​agents/​runtime/​test_catalogue_flow.py Tests reporting and request propagation.
agents/​README.md Documents runtime behavior.
docs/​concepts/​agent-network.md Records the new pipeline rules.
changelog.d/​pipeline-changes-user-wishes.md Adds the changelog fragment.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread agents/anyplot/pipeline.py Resolved
Comment thread agents/anyplot/prompts/adapter.md Resolved
…ge change translates standing wording

Copilot review of #12170:
- More notes than PlotResult holds keep the first ones (the request) and
  the latest (a later repair), not only the earliest five.
- The adapter and both reviewer prompts agree: a plot-language change
  translates the wording the user set and keeps its meaning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

No-op calls can report nonexistent changes, and conflicting retained requests lack explicit precedence.

0 open findings

2 resolved since last review
Previously missed (4)

In code that hasn't changed since last review

Medium severity Exclude no-op retries from reported changes

agents/​anyplot/​pipeline.py:634

A no-op initial adaptation or check retry still enters plan.changes whenever run.changes is empty. That makes PlotResult.changes report work that did not alter the code, contrary to the PR's stated rule that a no-op retry adds nothing. Return for every changed=False call; the successful no-edit case can remain an empty change list.

Medium severity Define precedence for conflicting earlier requests

agents/​anyplot/​prompts/​adapter.md:75

After a newer request supersedes an older one, both are stored and arrive here as “earlier requests” on the next run (for example, “make it red” followed by “make it blue”). The prompt currently requires both to stay exactly as set, which is impossible and can make later adaptations revert the newest choice. Define that later-numbered earlier requests override conflicting older ones.

Medium severity Prioritize newer requests over superseded history

agents/​anyplot/​prompts/​reviewer.md:15

The stored history can contain superseded requests, such as an old red-line request followed by a newer blue-line request. Treating every earlier choice as simultaneously correct gives the reviewer contradictory requirements and may reject the current blue render for not being red. State that later-numbered earlier requests take precedence.

Medium severity Give current and later requests precedence

agents/​anyplot/​prompts/​reviewer_catalogue.md:5

Conflicting requests are retained in the lineage, so a later run can provide both an obsolete choice and its replacement as earlier requests. Saying that either is correct leaves the scorer unable to evaluate the render consistently. Give the later-numbered request precedence, with the current request highest.

🧠 Review effort: Balanced

@codecov

codecov Bot commented Oct 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@MarkusNeusinger
MarkusNeusinger merged commit a437d88 into main Oct 14, 2026
16 checks passed
@MarkusNeusinger
MarkusNeusinger deleted the fix/pipeline-changes-user-wishes branch October 14, 2026 12:00
MarkusNeusinger added a commit that referenced this pull request Oct 14, 2026
Brings in #12169 (frame, generator, coroutine, traceback and code-object
attributes in ESCAPE_ATTRIBUTES, both profiles) and #12170 (the round's
changes and the user's standing requests). No conflicts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GdMSqLR5ww4ji74EUSmk9M
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants