Skip to content

Fix audit findings: result integrity, model units, cleanup and bounded context - #23

Merged
zaynullinmi merged 3 commits into
mainfrom
fix/audit-result-safety-and-performance
Oct 20, 2026
Merged

zaynullinmi merged 3 commits into
mainfrom
fix/audit-result-safety-and-performance

Conversation

@debug23win

@debug23win debug23win commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

Large tool outputs could turn failures into success and hide rollback state, and Python failures could commit partial model edits. Incorrect density conversion also understated mass, and cleanup and Undo relied on unsafe assumptions.

This PR preserves execution semantics across JSON completion, truncation, history aging, MCP and API chat. It rolls back logical failures before committing, and adds bounded archive pages that can retrieve data beyond the previous 30k ceiling.

Changes:

  • Units and types: correct density and derived Length/Area/Volume units, total compound wall/floor thickness, and type parameter preservation.
  • Cleanup and unloading: use native Purge Unused candidates, rehearse deletion cascades, keep placed zero-area rooms, and refuse to unload families that have instances or dependencies.
  • Undo: validate Undo counts, give unscoped actions their own ids, invalidate history after external edits, check document identity between steps, and wait for Redo confirmation.
  • Context size: reveal individual API tools, avoid ambiguous Russian prewarm matches, page and filter the complete catalog, and stop repeating the Codex driving rules on resumed prompts.
  • UI and benchmark: fit PNGs to the real base64 budget, throttle streaming markdown, respect benchmark round budgets, and distinguish benchmark execution profiles.
  • Receipts and diagnostics: return change receipts with paged changed IDs and scoped post-write readback. Record authorization, provider and readback phase timings, journal write health, and persistent SPDS refresh errors.
  • Docs: update both READMEs, and document evidence, limits and acceptance steps in docs/audit-fixes-2026-10-20.md.

Follow-up review fixes (6f104f2, e2d0aad)

A review of this change found 28 issues; all are addressed:

  • Tool discovery:
    • A group named only by a keyword loads whole, because a Russian request gives the English descriptions nothing to rank by; before this fix only the first 12 tools in registration order loaded.
    • Every keyword group in a prompt is loaded, and ranked matches beyond 12 are listed by name.
    • Matching accepts Russian verb prefixes and punctuated keys.
    • Pre-loading uses the user's text only, once per group per conversation.
  • Live SPDS: the updater writes its DataStorage only when the refresh state changes, which avoids worksharing ownership conflicts.
  • Headless results: escaped Cyrillic is re-encoded before the budget is measured.
  • Archive cache: it is now a real LRU; it used to evict the newest entry.
  • Aging:
    • Failures stay JSON, so aged results keep ok:false.
    • Aged summaries are bounded and never larger than the original.
    • Pages survive completion, and the system prompt describes the JSON aged summary.
  • Catalog: the default call returns an overview with counts, a nonzero offset requires a single list, and the response always fits the result cap.
  • Undo:
    • "steps": null undoes the whole last action.
    • An unconfirmed Redo of the user's step is reported.
    • Empty receipts are no longer archived.
    • Readback timing is recorded.
  • Cleanup and unloading: one purge pass, batch rehearsal, and bounded output. unload_family checks type-parameter references.
  • Names: an exact-case match wins, and system types accept the Family: Type form.
  • Calculations:
    • derive_parameters fails once on a dimension mismatch, and the description of factor is clarified.
    • In calculate_weight, material density takes precedence over the fallback, and parameter write failures are reported.
  • Native density fixture: uses 1234 kg/m³, which is independent of the default densities.

Validation:

  • 430 unit tests pass, 52 of them new. The 14 tests for discovery, keyword matching, the headless budget and the archive cache fail on the previous revision.
  • MCP transport and subscription CLI checks pass with isolated clients and processes, and no paid model calls.
  • 51 ModPlus adapter checks pass.
  • Builds for Revit 2025, 2026 and 2027 succeed, and the native acceptance harness compiles. Existing compiler warnings remain.

Native acceptance is still pending in live Revit:

  • Four fixtures: 300 mm multilayer total width, derived Length units, 1234 kg for a 1 m³ solid, and preservation of placed unbounded rooms.
  • Python rollback.
  • Painted and nested purge references.
  • Unload cascades.
  • Undo interleaved with tab switching and manual edits.
  • SPDS recovery.

Scope: this implements the confirmed recommendations. It does not claim that all 168 raw report entries were independently verified or resolved.

🤖 Generated with Claude Code

https://claude.ai/code/session_01RCGB6xWx3ZtmWZRQEJtN6B

debug23win and others added 3 commits October 20, 2026 05:18
…and archive cache

- Discovery: a group named only by a keyword (a Russian request gives
  English descriptions nothing to rank by) is loaded whole instead of its
  first 12 tools in registration order; every keyword group in the prompt
  is loaded; ranked matches beyond 12 are listed by name.
- Keyword matching accepts common Russian verb prefixes (образмерь,
  замаркируй, сгруппируй, заармируй), splits keys like the text so
  "технико-эконом" and "СП 1.13130" match, treats quotes, guillemets and
  brackets as separators, and restores English welded/wire mesh triggers.
- Live SPDS updater writes its DataStorage only when the refresh state
  changes, so ordinary edits no longer modify it on every transaction
  (ownership conflicts in workshared models).
- Headless results are re-encoded without \uXXXX escapes before the budget
  is measured; the test now uses the production serializer.
- Archive hot cache is a real LRU; Dictionary slot reuse made it evict the
  newest entry.

14 new regression tests, each failing on the previous revision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RCGB6xWx3ZtmWZRQEJtN6B
…nup and model tools

- Exceptions and repeated-error notes stay one JSON value, so aged results
  keep their failure state; text "Error…" results count as failures.
- Aged summaries copy fewer, shorter fields and never replace a result with
  something larger.
- get_full_result pages carry completion fields up front, so they are not
  re-capped; the test now runs completion then cap.
- System prompt describes the JSON aged summary and next_offset paging.
- post_write_readback timing passes the task id from the Revit thread.
- Pre-loading uses the user's text only and pre-loads each group once per
  conversation, avoiding repeated prompt-cache rewrites.
- Catalog: overview with family type counts by default, paging first, limit
  halved until the response fits, offset limited to a single list.
- Benchmark API runs that reach the round budget are marked budget-stopped.
- undo_last: "steps": null undoes the whole action; an unconfirmed Redo of
  the user's step is reported as such; no empty change archives.
- clean_model: one purge pass, one batch rehearsal with per-candidate
  fallback only on cascades/errors, bounded failure and cascade samples.
- unload_family refuses when types reference the family via a parameter.
- NameResolve prefers an exact-case match and accepts "Family: Type" for
  system types.
- derive_parameters fails once on a dimension mismatch, bounds failures,
  documents factor; calculate_weight lets material density win over the
  fallback and reports parameter write failures.
- Native density case uses 1234 kg/m3 converted by Revit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RCGB6xWx3ZtmWZRQEJtN6B
@zaynullinmi
zaynullinmi merged commit e9013b7 into main Oct 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants