Skip to content

test(chat): align operation fixture and recovery smoke - #5418

Merged
huangruiteng merged 5 commits into
loopx-project:mainfrom
LIHUA919:codex/personal-workspace-elapsed-smoke
Oct 2, 2026
Merged

huangruiteng merged 5 commits into
loopx-project:mainfrom
LIHUA919:codex/personal-workspace-elapsed-smoke

Conversation

@LIHUA919

@LIHUA919 LIHUA919 commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Goal and gap

The standalone operation fixture emits prepared and delivered before confirmed, while its Python assertion expected only four later stages. In exact-head CI, the Personal Workspace browser smoke also timed out waiting for a recovered Todo card. That scenario armed its transient preview failure after reopening Goal Chat, allowing recovery to race the injection; its 5-second card deadline was tight for the history retry's 3-second backoff under CI load.

Change

  • Assert the complete six-stage operation fixture lifecycle.
  • Arm the transient preview failure before reloading, verify it was exercised, and give recovery up to 15 seconds to display the card after retry.
  • Preserve the existing product UI, persisted state and authority behavior. Signed merge d701822be integrates current main 850268bff; the contribution diff is limited to the two test files.

Validation

  • tests/test_chat_operation_actions.py: 41 passed on the integrated source.
  • Chat bundle build and verification passed.
  • Focused chat-todo-proposal real browser scenario passed in development with coverage and in the packaged Chat build.
  • The complete development browser suite was attempted: 16 scenarios passed, then team-evidence timed out waiting for its active-mode label; that scenario passed when run alone. This separate load-sensitive failure remains unclassified, so the complete suite is not claimed green.
  • node --check and git diff --check passed. Exact-head remote CI passed the four Python shards and dashboard acceptance, but aggregate pytest failed while reporting coverage for a disposable semantic-probe fixture (No source for code). #5440 owns that narrow repair. CI is red; maintainer review and merge remain pending.

中文

独立操作夹具会先输出 prepared、delivered,原 Python 断言却只接受后四个阶段。上一精确版本 CI 还在恢复后的 Todo 卡片可见性检查处超时:浏览器场景直到重新打开 Goal Chat 才启用一次性预览失败,可能与恢复完成竞争;在 CI 负载下,5 秒等待也不足以覆盖历史重试的 3 秒退避。

本 PR 校正六阶段断言,并在重载前启用预览失败,确认故障确实被触发,给恢复重试后的卡片最多 15 秒显示。产品界面、持久化和权限行为未修改。签署合并 d701822be 纳入最新 main 850268bff,相对主干仅改动两个测试文件。

整合源码上的 Python 文件测试 41 项通过;Chat 打包构建与校验通过;chat-todo-proposal 在带覆盖率的开发浏览器和打包浏览器均通过。完整开发浏览器套件曾运行,前 16 个场景通过,随后 team-evidence 等待运行状态标签超时;该场景单独运行通过,因此不能宣称完整套件已通过。语法和差异检查通过。精确版本远端 CI 的四个 Python 分片与 dashboard acceptance 已通过,但覆盖率汇总读取已清理的语义探针临时源码时失败(No source for code);独立的 #5440 负责该修复。CI 仍为红色,维护者评审与合并待完成。

Signed-off-by: Lihua <1017343802@qq.com>
@LIHUA919
LIHUA919 requested a review from huangruiteng as a code owner October 1, 2026 13:36
Signed-off-by: Lihua <1017343802@qq.com>
@LIHUA919 LIHUA919 changed the title test(workspace): make elapsed view-switch smoke deterministic test(workspace): stabilize packaged browser and fixture smokes Oct 1, 2026
Signed-off-by: Lihua <1017343802@qq.com>
Signed-off-by: Lihua <1017343802@qq.com>
Signed-off-by: Lihua <1017343802@qq.com>
@LIHUA919 LIHUA919 changed the title test(chat): align isolated operation fixture stages test(chat): align operation fixture and recovery smoke Oct 2, 2026
@Duang777

Duang777 commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

Exact-head CI attribution for d701822beb6892cdf7d9062b24bcc5ccc6779730 on current main@850268bffc6f7123e88f578d3743f8f6da5c5c73:

  • Run 36958279407 passed all four Python shards, Windows PowerShell, Dashboard acceptance, every TypeScript job, and all other direct checks.
  • The only direct failure was the pytest coverage aggregate. Coverage combined all four shard files, then stopped on No source for code: /tmp/.../test_cli_inspects_disjoint_ind0/loopx/a.py.
  • test(ci): isolate semantic probe fixture coverage #5375 fixes that inherited coverage leak by removing the outer pytest-cov environment from the synthetic subprocess. Its earlier exact-head run 36825867638 passed every job.
  • Current-main run 36914904063 independently fails the stale four-stage operation-fixture assertion that this PR updates. The same main run also hit the unrelated Windows process-cleanup WinError 5 seen intermittently on other branches.

The clean order is to review and merge #5418 first, then sync #5375 so its coverage fix runs with the six-stage assertion. #5409 and #5389 can sync after those baseline fixes land. No merge action was taken.

@Duang777

Duang777 commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Updated CI triage for d701822beb6892cdf7d9062b24bcc5ccc6779730:

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking finding. No product behavior, authority or protocol change. The remaining uncertainty is the reported load-sensitive browser race: the immutable base also passed the focused browser scenario in this run, so I do not claim to have reproduced that timing failure or qualified the entire browser suite.

动机

本次按 pull_request_review_execution_contract_v2(policy 13)独立评审完整 base-to-head:850268bffc6f7123e88f578d3743f8f6da5c5c73 → d701822beb6892cdf7d9062b24bcc5ccc6779730。目标是修复已有验收的陈旧期望和故障注入竞争,不是新增恢复机制。实际独立 fixture 已输出六阶段;基线的四阶段断言失败,因此修正期望有明确仓库价值。相比新增 runner 或重写生产恢复 owner,延续两个已有测试是更小、完整且可逆的修复。

改动思路

失败开关在 reload 前启用,避免恢复完成后才注入;卡片等待上限从 5 秒调整到 15 秒,以容纳已有历史重试,而非修改产品退避或承诺新的性能指标。新增断言要求开关已被消费,防止场景“绿色但没有测试失败”。Python 测试继续用 -S 执行独立 fixture,不引入测试框架依赖。

具体改动

  • test_browser_operation_fixture_needs_no_test_framework(tests/test_chat_operation_actions.py:81)校验 prepared → delivered → confirmed → waiting → unknown → reconciled,保留 Goal、executor 和未知结果/对账断言。浏览器 confirmed-operations.mjs 也实际消费前两阶段,并非为适配偶然输出添加字段。
  • chatTodoProposalScenario.run(examples/personal-workspace-browser/chat-todo-proposal.mjs:124–150)仅改变故障注入位置、等待预算及消费检查;仍检查唯一幂等键、原 Goal、用户确认及后续 reload 不重复 apply。
  • 已沿生产 projectRecoveredTurnProposals 和 completed-history effect 检查恢复路径:读取已有 action,失败返回未完成并由既有有限退避重试;preview 不授予 apply 权限。模拟 API 只提供输入/503,恢复与展示执行真实前端代码。

独立本地验证:

  • 精确 head 的 uv run --extra test python -m pytest -q tests/test_chat_operation_actions.py:41 passed;同命令不可变 base:40 passed / 1 failed,仅上述陈旧四阶段断言失败。
  • 原生 chat-todo-proposal 浏览器验收:base、head 开发版均通过;head 打包版通过。head npm run build:chat 和 uv run --extra test python scripts/chat_bundle.py verify --source 通过。
  • 独立负向探针只在内存中绕过 fixture 的故障分支、保留未消费开关,产品代码未改;真实 Chromium 场景准确在 Recovery did not exercise the transient preview failure 断言失败。这证明新增检查能拒绝“故障未发生”的伪通过,不证明历史负载问题已复现。
  • 改动 Python 的 Ruff、JavaScript 语法检查和 whole-diff whitespace 检查通过。

对主干的风险

贡献 diff 两文件 +8/-3,都是现有测试;无生产状态分类、默认开关、通用控制面措辞、guidance/obligation 或 actor authority 变更。未新增并行决策 owner,不需要语言迁移。长期重复恢复、离开 Goal 后返回及下一次 reload 的既有幂等/确认断言保留,用户操作路径和权限保持不变。

浏览器证据边界是“真实开发/打包前端 + 隔离合成 API/stream”,不是真实模型、部署后端或已安装版本验收。未跑完整 pytest、完整浏览器套件或 PostgreSQL;没有读取、等待或用远端 CI 代替本地评审。作者描述的全套浏览器失败和覆盖率问题不宣称已由本 PR 修好。

耐久性/重复扫描:检查了两个 immutable revision 的 fixture、相邻恢复场景、现有 runner、package smoke 命令和 Python workflow 调用,并查看作者近期 30 个 PR(包括 #5440 的独立 coverage 边界)。本改动复用既有生产验收,没有新建同形 smoke 或临时工具。future-facing pass 考虑了是否需要按请求绑定故障计数器或 TS 抽象;当前局部场景已有明确时序和消费断言,没有足以支持额外共用机制的缺口,暂不扩展。

我的整体评价

APPROVE。明确的陈旧断言修复,加上能经负向探针验证的既有恢复场景加固,构成完整测试维护结果;不把更大的等待值等同于产品性能修复。不替整个恢复路线、全套 CI 或上层路线图宣布完成。批准后应单独读回 effective blocking reviews,只有逐项独立确认旧 finding 已解决且拥有撤销权限时才撤销;本结论不授权 merge。

English verdict

English verdict: APPROVE

APPROVE — independently reviewed exact head d701822beb6892cdf7d9062b24bcc5ccc6779730. The stale lifecycle expectation is reproduced at the immutable base and corrected at head. Native development and packaged browser acceptance pass; an independent guard mutation demonstrates that the new assertion rejects an unexercised transient failure. The base browser also passed, so the historical timing failure is not claimed reproduced. This is existing-test maintenance, not a production latency, authority, protocol or default-policy change. Synthetic API/stream boundaries and unrun whole-suite/backend validation are explicitly retained. No merge authorization.

@huangruiteng
huangruiteng merged commit 553623e into loopx-project:main Oct 2, 2026
27 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants