fix: resolve 8 bugs found in full-surface self-test (chat, terminal, sandbox, plan mode) - #18
Open
Wyn2004 wants to merge 11 commits into
Open
fix: resolve 8 bugs found in full-surface self-test (chat, terminal, sandbox, plan mode)#18Wyn2004 wants to merge 11 commits into
Wyn2004 wants to merge 11 commits into
Conversation
Wyn2004
force-pushed
the
fix/evoflux-qa-selftest-bugs
branch
from
September 4, 2026 00:14
06274be to
4faf26f
Compare
isEmptyIdleSession() judged a session empty from live agentStreams alone, ignoring that loadSession() might still be fetching its history. Right after mount/switch this window makes any session look empty, so New Chat's early-return guard silently swallowed the click even when the session had real history. Now isSessionLoading also forces "not empty".
The sandbox's default deny-list only matched the literal .env/.env.* filenames, so an agent blocked from writing .env could trivially bypass it by renaming to an equivalent secret file (env.local, secrets.json, credentials.yaml, .aws/credentials, id_rsa, etc.) with identical content. Widen DEFAULT_DENIED_PATTERNS to cover common secret/credential and SSH-key naming conventions. Filename denylists can't be exhaustive, so this narrows the obvious gap rather than closing the bug class.
…r (BUG-002) The composer's model picker showed the raw internal "__PROVIDER_MODEL__" sentinel instead of "Default" for any session whose lead agent has no per-agent model override, since shortModelName() had no placeholder awareness. Add a normalizeModelId() helper and a single shared PROVIDER_MODEL_PLACEHOLDER constant in model-settings.ts, apply it at SessionPillsRow's effectiveModel computation (covers button label, aria-label, provider icon, and raw-id caption in one place), and harden BlockRenderer.tsx/AssistantTurnFooter.tsx's local shortModelName copies against the same raw value.
…hts (BUG-008) The Accept/Reject/Revise controls were mounted as a separate floating strip above the chat composer instead of inside PlanReviewPanel's own footer. When SidePanel falls back to its mobileOverlay full-width sheet (a width-driven threshold in getResponsiveSidePanelLayout, not a height one as originally suspected), that sheet renders in its own stacking context on top of the entire app, permanently covering the separately rendered action bar underneath — with no CSS fix possible without folding it into the same DOM subtree. Move the action bar into PlanReviewPanel's own shrink-0 footer, below its existing min-h-0 flex-1 overflow-y-auto content region, matching the header/content/footer shape already used by ChangeSetReviewPanel and ChangesReviewPanel. Verified with document.elementFromPoint against a real plan-review session at 1000x600/768/900 (previously unreachable) and 1280x768.
Wyn2004
force-pushed
the
fix/evoflux-qa-selftest-bugs
branch
from
September 4, 2026 05:20
4faf26f to
7600fc4
Compare
An unknown/uncataloged model on the silent-thinking path was handed straight to _disable_fields() as a raw string where a _ModelContract was expected, crashing provider model listing with AttributeError: 'str' object has no attribute 'is_effort_control' the moment a model outside the catalog showed up (e.g. a newly released Gemini variant). Build the contract via _model_contract() first, matching every other call site of _disable_fields().
…ng thought_signature Gemini 3 rejects a replayed functionCall part with no thoughtSignature (HTTP 400: "Function call is missing a thought_signature"), which happens for any call captured before this field existed, from a different provider/model, or a Gemini response that simply didn't return one for that call — reproduced live on a real long-running session switched to gemini-3.1-flash-lite-preview. Google documents "skip_thought_signature_validator" as the exact sentinel to place in thoughtSignature when a real signature genuinely isn't available (https://ai.google.dev/gemini-api/docs/generate-content/thought-signatures), for precisely this scenario: injecting function calls executed deterministically by the client, or replaying a trace from a model that doesn't produce signatures.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Ran an exhaustive self-test pass across EvoFlux's full feature surface (~21 feature clusters, ~300 individual features/endpoints) to find and fix real, reproducible bugs before they hit users. This PR bundles the confirmed, fixed findings from that pass — one commit per bug (plus one unrelated formatting cleanup commit). Full technical reports (root-cause traces, raw logs, more screenshots) live in a local
report_bugs/folder outside this repo's tracked history; screenshots referenced below are hosted on thepr-evidence/qa-selftest-bugsbranch, following the existingpr-evidence-hostconvention (see #16) — that branch is not meant to be merged into a product branch.The most severe: Plan mode did not actually block destructive tool calls (BUG-007).
permission_mode="plan"is EvoFlux's core "review before execution" safety feature, but a Pydantic dict-copy timing bug meant the interception flag never reached the tool executor. Also found: a sandbox denylist an agent could trivially rename its way around, and a race condition that made "New Chat" silently do nothing.Note: one originally-found bug (a missing
@xterm/addon-webgldependency crashing the Terminal panel) was fixed independently by another commit already onmain(915e5074) while this branch was in flight — dropped from this PR during rebase since it's already resolved, no action needed here.Bugs fixed
BUG-001 — "New Chat" silently does nothing right after page load
Description: Clicking "New Chat" immediately after opening the app (before the previous session's history has finished loading) does nothing — no new session appears, no error either.
Use case: Anyone opening EvoFlux and immediately trying to start a fresh conversation before the UI has fully settled — a very ordinary first action.
How to reproduce: Reload the app, and within the first ~1s (while the last session's history is still fetching) click "New Chat". Nothing happens. Wait a few seconds and click again — it works fine.
Root cause:
isEmptyIdleSession()decided whether the current session was "empty enough to reuse" by looking only at live in-memoryagentStreams, with no awareness thatloadSession()might still be fetching that session's real history in the background. Right after mount/switch, this made every session look empty, so "New Chat"'s guard silently swallowed the click on a session that actually had content.Fix: The emptiness check now also treats
isSessionLoadingas "not empty".BUG-002 — Model picker shows a raw internal placeholder instead of "Default"
Description: The composer's model picker button shows the literal text
__PROVIDER_MODEL__(visually truncated to "_PROVIDER...") instead of "Default", whenever the active agent has no per-agent model override — the normal state for a fresh install.Use case: Any new EvoFlux user, or anyone starting a new Coding session with an agent that hasn't been given a specific model — i.e. most first-time and default usage.
How to reproduce: Ensure the lead agent has no model override configured. Start a new Coding session, don't send a message yet, and look at the model picker button in the composer.
Root cause:
shortModelName(), used by the composer's model picker, had no special case for the internal"__PROVIDER_MODEL__"sentinel (which means "use the provider's default"), so it displayed the raw value verbatim.Fix: Added a shared
normalizeModelId()helper (and a single exportedPROVIDER_MODEL_PLACEHOLDERconstant, replacing a private duplicate that already existed in the Settings → Agents screen) and applied it where the composer resolves its displayed model.BUG-004 — Sandbox's secret-file protection is trivially bypassed by renaming (security)
Description: EvoFlux's sandbox is meant to block an agent from writing files like
.envthat typically hold secrets. In practice, the block only matches the exact names.env/.env.*— any other filename holding the same kind of secret (env.local,secrets.json,credentials.yaml, an SSH private key, etc.) is not blocked at all.Use case: Every EvoFlux user relying on the sandbox's file-write protection to keep an agent from creating or overwriting credential files — which is the entire point of that setting.
How to reproduce: Ask an agent to write a
.envfile with some content; observe it gets blocked. Then ask it to write the same content toenv.localinstead — it succeeds. In the real test run below, the agent hit the.envblock and chose to rename toenv.localon its own, without being told to.Root cause: The default
denied_patternslist only ever contained["**/.env", "**/.env.*"]. Additionally, this default is only applied when a fresh config is first created — an already-existingsandbox.yaml(i.e. every real install predating this fix) keeps whatever was seeded on first run forever, with no upgrade path.Fix: Two commits — (1) widened the default pattern list from 2 to 14 entries covering common secret/credential/SSH-key filenames; (2) added a one-time migration that upgrades an existing
sandbox.yamlto the new list, but only when it exactly matches the old 2-pattern default (so any user customization, however small, is left untouched). Verified this machine's own real local config got migrated correctly as a result.Real transcript — agent bypassing the block unprompted:
BUG-005 — Same placeholder leak as BUG-002, in a different picker
Description: The same
__PROVIDER_MODEL__raw-token leak as BUG-002, found in a different UI element: the model picker inside the "choose which agent to spawn" dialog.Use case: Anyone using multi-agent delegation where the Lead asks the user to pick which specialist to spawn, and that specialist has no model override.
How to reproduce: Trigger the agent-spawn picker (Lead delegating to a specialist without a model override) and look at the model label shown in that dialog.
Root cause: Same as BUG-002 — a different call site (
AskUserQuestionModal.tsx) reading the model id directly without normalizing the placeholder, found while auditing every place BUG-002's root cause could recur.Fix: Reused the
normalizeModelId()helper added for BUG-002 at this call site — a 3-line change. (No separate screenshot — same visual defect as BUG-002, at a different call site.)BUG-006 — Rejecting a tool-call request doesn't stop the agent from repeating it
Description: When a user clicks "Reject" on a permission prompt (in
askmode), the action is correctly blocked — but the agent gets no usable signal that it was rejected, and simply tries the exact same action again, producing an identical-looking prompt, repeatedly, with no explanation.Use case: Anyone running in
askpermission mode (the mode whose entire purpose is "the user stays in control") who explicitly declines an action — they end up looking at what appears to be the same unresponsive dialog over and over.How to reproduce: Set a session to
askpermission mode, ask the agent to write a file, and click "Reject" on the resulting prompt. Within a few seconds, an apparently identical prompt reappears for the same action (confirmed via distinct request IDs — the agent really is issuing a new call each time, not a stuck UI).Root cause: Deeper than expected —
PermissionRejectedErrorwas raised inside a hook that runs before the only code path that turns tool exceptions into a message the model can see. The rejection error was being silently dropped entirely: the model received zero signal, not merely an unclear one.Fix: (1) The agent's turn loop now always converts a hook-raised exception into a real error message back to the model, instead of discarding it; (2) that message is now explicit and directive ("the user declined this; do not retry without a materially different approach or asking first"). A further idea from testing — auto-detecting repeated identical rejections and surfacing a clearer UI prompt after N tries — was intentionally left as an open recommendation rather than built, since it's a UX/product-scope decision, not a one-line fix.
Real transcript — same prompt reappearing after two separate rejections (different request IDs, agent genuinely retrying):
BUG-007 — Plan mode doesn't actually prevent real changes (the most severe finding)
Description: Setting a session's permission mode to
planis supposed to make the agent record its intended file edits/commands and present them for approval before touching anything. Instead, the agent's edits and shell commands ran for real immediately, with no recording and no approval step ever appearing — 100% reproducibly, independent of which model was used.Use case: Anyone who turns on Plan mode specifically because they want to review changes before an agent touches their codebase — the exact scenario Plan mode exists for was silently not happening.
How to reproduce: Set a Coding session's permission mode to
plan. Ask the agent to make a small file edit. Watch it get written to disk immediately (git diffon the workspace shows a real, uncommitted change) while the composer still displays "Plan mode" as active, with no plan-review panel ever appearing.Root cause: The per-run configuration object (
RunConfig, a Pydantic model) copies itsmetadatadictionary into a new object as soon as it's constructed. The code that flips on the "we're in plan mode" flag did so by mutating the original dictionary — 38 lines after that copy had already been made — so the flag never reached the part of the code that checks it. The interception logic itself was correct; it was simply never being told plan mode was active.Fix: Moved the flag-setting code to run before the config object is constructed, so the flag lands in the copy that's actually used. Verified live end-to-end afterward: an edit request under Plan mode is now correctly recorded rather than executed, a plan-review panel appears with the proposed change, and the file is only written after clicking "Accept & execute". Also checked the other permission modes (
accept-edits,bypass,auto,ask) for the same class of bug — none of them relied on this same after-the-fact mutation, so none needed a change.BUG-008 — Plan Review's approve/reject buttons are invisible at common window widths
Description: Found while verifying the BUG-007 fix: the "Reject / Revise / Accept & execute" buttons on the new Plan Review panel are completely covered and unclickable whenever the browser window is narrower than roughly 1058px — a common width for anyone not using a full, wide, maximized window.
Use case: Anyone reviewing a plan in a moderately narrow window (a laptop that isn't full-screen, a split-screen layout, an external monitor at a modest resolution) — they'd see a plan with no way to act on it.
How to reproduce: Narrow the browser window below ~1058px wide (height doesn't matter — reproduces at 600, 768, and 900px tall, all at that width). Trigger a Plan Review (see BUG-007's repro). The action buttons are not visible/clickable; at a wider window the same panel works fine.
Root cause: Below that width, the side panel switches from docking in-flow next to the chat to a full-width overlay sheet — which sits in its own layer on top of the entire app, including a separately-mounted action-bar component that isn't part of the same panel. The overlay simply covers it.
Fix: Moved the action bar to live inside the Plan Review panel's own layout (as its footer, below the scrollable content) instead of being a separate, independently-positioned component — so it can never be covered by anything again, at any window size. Verified working again at several widths/heights after the change.
BUG-009 — Listing a provider's models crashes if any model is outside EvoFlux's internal catalog
Description:
can_disable()(used when deciding whether a model's "thinking"/reasoning effort can be turned off) has a fallback path for a model EvoFlux's internal catalog doesn't know about yet. That path passed the model's raw name (a string) into a helper that expects a structured model-contract object, crashing withAttributeError: 'str' object has no attribute 'is_effort_control'.Use case: Anyone browsing a provider's model list (e.g. the model picker, or Settings → Providers) at a moment when that provider has released a model newer than EvoFlux's bundled catalog — increasingly likely for a fast-moving provider like Google Gemini, which shipped several new model IDs during this very testing session.
How to reproduce: Call
can_disable(provider_id, model)for any model absent from the provider's thinking catalog (e.g. a freshly released preview model) — it raisesAttributeErrorinstead of returning a sensible default, which surfaces as a 500 when listing that provider's models.Root cause: The "unknown model" branch called
_disable_fields(resolved, model), handing it the raw model-id string where every other call site passes a_ModelContractbuilt by_model_contract(...).Fix: Build the
_ModelContractvia_model_contract()before calling_disable_fields(), matching the pattern used everywhere else in the module. No screenshot — this is a backend crash caught by a new regression test (test_an_unknown_model_still_answers_can_disable), not a UI-visible defect.BUG-010 — Gemini 3.x rejects the whole turn when a replayed function call has no
thought_signatureDescription: Discovered from a real crash log on a live, long-running user session after switching it to
gemini-3.1-flash-lite-preview. Gemini 3.x requires every replayedfunctionCallpart in conversation history to carry athoughtSignature; when one is missing, Gemini rejects the entire request with HTTP 400, not just that one call — the turn, and the session on that model, becomes unusable.Use case: Any long-running Coding/Work session with tool-call history that gets switched to a Gemini 3.x model — very easy to hit, since it's exactly what happened here: a session with history predating this field, or from a different model, then switched to Gemini 3.
How to reproduce: Have a session with a tool call in history whose
thought_signatureisnull(an old session, or one created on a non-Gemini model). Switch it to a Gemini 3.x model and send a message.Root cause:
_convert_messages_to_gemini()passedthought_signature=tc.function.thought_signaturestraight through — when that'sNone, Gemini received a literalnullinstead of the sentinel it requires for this exact scenario. Confirmed this is a common integration gap (not unique to EvoFlux) via other tools hitting the identical error: Cline #7974, Continue #8785, Roo-Code, Dify.Fix: Google documents an exact sentinel value,
"skip_thought_signature_validator", for precisely this case — "transferring a trace from a different model that does not include thought signatures" (official docs). Substitute it whenever the real signature is missing.Real crash, from a live session (not synthetic):
After the fix, the new regression test (
test_convert_assistant_tool_call_missing_thought_signature_gets_sentinel) passes alongside the rest of the provider's suite:What changed (files)
Backend
app/agent/mode/team/member.py— hoist plan-mode metadata writes beforeRunConfig(...)construction (BUG-007).app/agent/sandbox_config.py— widenDEFAULT_DENIED_PATTERNS, plus a one-time migration for existing configs (BUG-004).app/agent/permission.py,app/agent/agent_loop/core.py— rejected tool calls always produce a real, directiveToolMessage(BUG-006).Frontend
web/src/stores/useTeamStore/index.ts—isEmptyIdleSession()accounts forisSessionLoading(BUG-001).web/src/lib/model-settings.ts(+SessionPillsRow.tsx,BlockRenderer.tsx,AskUserQuestionModal.tsx) — sharednormalizeModelId()helper (BUG-002, BUG-005).web/src/components/PlanReviewPanel.tsx(+TeamChatView/index.tsx,chat/ChatPanels.tsx) — fold the action bar into the panel's own footer (BUG-008).chore: appliedruff formatto files with pre-existing formatting drift, unrelated to any bug.app/agent/providers/thinking.py— build a_ModelContractbefore calling_disable_fields()(BUG-009).app/agent/providers/googlegenai/googlegenai.py— substitute Google's documented sentinel for a missingthought_signature(BUG-010).Checklist
report_bugs/folder, not in this repo)ruff check/ruff format --check/ty checkclean against latestmain(ty checkshows the same pre-existingapp/workflow/diagnostics asmainitself, none new)pytestrun against this branch rebased onto latestmain— every failure reproduces identically on a cleanmaincheckout (confirmed by direct comparison), none introduced by this branchlint/typecheckclean against latestmain(2 pre-existing lint errors confirmed present onmainitself, untouched files),test:unit472/472 passing across 121 filesbuildsucceedsmainafter significant upstream churn during this PR's development; one originally-fixed bug (BUG-003) was dropped as redundant aftermainfixed the same issue independentlyty check, which flagged the exact type mismatch this fix resolvesNotes for the reviewer
sandbox.yamlwhen it's an exact match for the old default — any customization at all is left alone.mainis moving fast — this branch was rebased once already during review to resolve conflicts with ~35 intervening commits. A second rebase may be needed at merge time depending on further upstream activity.CLAUDE.mdanddocuments/codebase-guide/still describe a removed "AIM" mode (dropped from the product on 2026-08-06) — a stale-documentation issue unrelated to any of these bugs.