Skip to content

perf: persist Claude cost report memo across launches - #3284

Merged
steipete merged 3 commits into
steipete:mainfrom
eggyrooch-blip:perf/claude-report-memo-disk
Sep 8, 2026
Merged

perf: persist Claude cost report memo across launches#3284
steipete merged 3 commits into
steipete:mainfrom
eggyrooch-blip:perf/claude-report-memo-disk

Conversation

@eggyrooch-blip

@eggyrooch-blip eggyrooch-blip commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Persist the Claude local-cost report memo beside claude-v6.json, allowing an unchanged transcript inventory to reuse its report after relaunch without decoding the full transcript cache. This addresses repeated cold-start work on large Claude histories, investigated alongside #3247.

The saved envelope now has separate format and report-semantics revisions. Missing or mismatched semantics reject the saved report before the early return, so changed bundled pricing, aliases, or aggregation can invalidate it. Source identities, cache/pricing stamps, requested window, cancellation handling, and atomic writes retain current main's protections. Missing or corrupt sidecars fall back to the normal cache path.

Maintainer repair of @eggyrooch-blip's contribution; human commit credit remains sunke sunke@keep.com. Added regressions cover poisoned old/missing/future revisions, corrupt sidecars, cold pricing replacement, cancellation preserving sidecar bytes, and complete report-field round trips. Changelog context is collected in the batch's separate notes PR.

Built CLI proof on 1d690b69c7cd96e5f05d08cebb018fefb5e0fbc2 used 1,250 compact synthetic Claude events in 25 files, isolated child-process storage, and a seeded pricing catalog. Every row below is a new process running CodexBarCLI cost --provider claude --days 1 --format json:

Scenario Tokens USD
First process creates cache and memo 1,375,000 12.6250
Second process, transcripts and cache chmod 000 with identical inode/size/mtime 1,375,000 12.6250
Readability restored; saved report poisoned and semantic revision downgraded 1,375,000 12.6250
Append one event 1,376,100 12.6351
Replace input pricing in the catalog 1,376,100 25.1451

The second process had no readable transcript/cache fallback; nonempty matching totals prove persisted-report reuse. Modes were restored in a finally block. The semantic mismatch was rewritten at the current revision. Assertions passed for all token and cost totals.

Local and committed-branch autoreview are clean through P2. The focused memo suite passed all 18 tests after correcting fixture discriminators and checking transcript stability across a rebuild. Final make check and full make test passed: 1,033 selections across 87 groups, with no failures, retries, or timeouts. Exact-head CI succeeded. Land-ready.

Keep the local-cost daily report and source inventory next to claude-v6.json so an unchanged transcript set skips the full JSON cache decode after process restart. Pricing and cache-artifact stamp changes still miss the memo and reprice from the persisted rows.

Investigated alongside steipete#3247.
@clawsweeper

clawsweeper Bot commented Aug 30, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 30, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-30T09:51:30.642107Z 778b379 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 778b3793c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

}

static let shared = CostUsageClaudeReportMemo()
static let persistedVersion = 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Invalidate persisted reports when the app version changes

When a user upgrades to a build that changes bundled Claude pricing, model aliases, or report aggregation while the transcripts, cache artifact, pricing artifact, and requested date range remain unchanged, this constant still accepts the report produced by the previous binary. loadClaudeDaily then returns that report before running the current CostUsagePricing/aggregation code, so costs can remain calculated with old application logic until another key input changes; include an application/report-logic version in the persisted key or invalidate this sidecar across relevant upgrades.

Useful? React with 👍 / 👎.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 30, 2026
@clawsweeper

clawsweeper Bot commented Aug 30, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed September 8, 2026, 12:18 AM ET / 04:18 UTC.

ClawSweeper review

What this changes

Persist Claude and Vertex AI daily cost reports beside their transcript caches so unchanged inputs can reuse calculated totals after relaunch.

Merge readiness

Ready for maintainer review

This remains a useful, merge-ready performance improvement: current main lacks persistent report reuse, both prior findings are resolved, and the maintainer verified the changed behavior on the reviewed head. No actionable correctness or security findings remain.

Priority: P2
Reviewed head: 1d690b69c7cd96e5f05d08cebb018fefb5e0fbc2

Review scores

Measure Result What it means
Overall readiness 🦞 diamond lobster (5/6) Focused implementation, resolved prior findings, compatibility coverage, and convincing production CLI evidence support merge readiness.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (terminal): The captured body and owner confirmation exercise the changed memo through the built cost CLI in five separate processes: persistent reuse succeeds with fallback files unreadable, incompatible semantics rebuild safely, and transcript/pricing changes produce updated totals.
Patch quality 🦞 diamond lobster (5/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The captured body and owner confirmation exercise the changed memo through the built cost CLI in five separate processes: persistent reuse succeeds with fallback files unreadable, incompatible semantics rebuild safely, and transcript/pricing changes produce updated totals.
Evidence reviewed 9 items Repository policy and scope: Read the complete root AGENTS.md. No additional AGENTS.md or maintainer-notes paths were found beneath Sources, Tests, docs, or .agents. Applied provider isolation, focused validation, and cache-safety guidance; no builds or tests were executed during this read-only review.
Current main still needs the optimization: Current main stores report entries only in its process-local dictionary; it has no persisted-envelope load or save. The broader source/docs search found the existing memo and scanner integration, not another persistent-report implementation.
Persistence and invalidation boundary: The new envelope checks format and report-semantics revisions, fails softly on missing or invalid JSON, and replaces the sidecar using a temporary file and rename. The existing scanner additionally compares provider/filter, roots, date window, timezone, source inventory, and cache/pricing stamps before returning a memo.
Findings None None.
Security None None.

How this fits together

CodexBar’s local cost scanner combines transcript usage with model pricing to supply CLI reports and spend displays. The saved report memo avoids rebuilding those totals when the source inventory, pricing, requested dates, and report semantics still match.

flowchart TD
  A[Local transcripts] --> C[Check inventory and report key]
  B[Pricing and requested dates] --> C
  D[Saved report memo] --> C
  C -->|Compatible| F[Daily cost report]
  C -->|Missing or changed| E[Scan cache and calculate totals]
  E --> D
  E --> F
  F --> G[CLI and spend displays]
Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test delta Production +152/-17; tests +89/-7; docs +1 Production growth implements persisted envelopes and complete report encoding, supported by compatibility and invalidation coverage.
Production-path proof 5 CLI processes; 1,250 events across 25 files The recorded runs distinguish persistent reuse from fallback and verify invalidation after semantic, transcript, and pricing changes.

Technical review

Best possible solution:

Retain the versioned, disposable report sidecar with existing invalidation and fallback behavior, keeping future report-semantics revisions synchronized with pricing and aggregation changes.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a discrete bug reproduction: this optimizes established report generation. Current-main source proves the memo is process-local, and recorded CLI runs demonstrate the proposed persistent reuse.

Is this the best way to solve the issue?

Yes. Extending the existing memo with a versioned, disposable sidecar preserves the scanner’s established validation path without replacing the transcript cache or changing user settings.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 2a71b479a5d4.

Labels

Label changes:

  • add rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦞 diamond lobster, so this older rating label is no longer current.

Label justifications:

  • P2: This is a bounded improvement to repeated local-cost work after relaunch, with no demonstrated urgent runtime regression.
  • rating: 🦞 diamond lobster: Overall readiness is 🦞 diamond lobster; proof is 🦞 diamond lobster and patch quality is 🦞 diamond lobster.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The captured body and owner confirmation exercise the changed memo through the built cost CLI in five separate processes: persistent reuse succeeds with fallback files unreadable, incompatible semantics rebuild safely, and transcript/pricing changes produce updated totals.
  • proof: sufficient: Contributor real behavior proof is sufficient. The captured body and owner confirmation exercise the changed memo through the built cost CLI in five separate processes: persistent reuse succeeds with fallback files unreadable, incompatible semantics rebuild safely, and transcript/pricing changes produce updated totals.

Evidence

What I checked:

  • Repository policy and scope: Read the complete root AGENTS.md. No additional AGENTS.md or maintainer-notes paths were found beneath Sources, Tests, docs, or .agents. Applied provider isolation, focused validation, and cache-safety guidance; no builds or tests were executed during this read-only review. (AGENTS.md:1, 1d690b69c7cd)
  • Current main still needs the optimization: Current main stores report entries only in its process-local dictionary; it has no persisted-envelope load or save. The broader source/docs search found the existing memo and scanner integration, not another persistent-report implementation. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageClaudeCache.swift:60, 2a71b479a5d4)
  • Persistence and invalidation boundary: The new envelope checks format and report-semantics revisions, fails softly on missing or invalid JSON, and replaces the sidecar using a temporary file and rename. The existing scanner additionally compares provider/filter, roots, date window, timezone, source inventory, and cache/pricing stamps before returning a memo. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageClaudeCache.swift:159, 1d690b69c7cd)
  • Scanner retains compatibility checks: The report-return path validates the complete memo key and honors forceRescan; report storage occurs only while the cache and pricing artifacts still match the scan. Claude and Vertex use provider-specific cache filenames. (Sources/CodexBarCore/Vendored/CostUsage/CostUsageScanner+Claude.swift:617, 1d690b69c7cd)
  • Prior findings resolved and upgrade fallback covered: Tests now include valid report discriminators and rejection of missing, old, and future semantic revisions. Additional coverage verifies missing/corrupt sidecar fallback, cold repricing, cancellation preserving sidecar bytes, and complete report-field round trips. (Tests/CodexBarTests/CostUsageScannerClaudeMemoTests.swift:154, 1d690b69c7cd)
  • Real production CLI proof: The supplied complete PR body and owner verification at perf: persist Claude cost report memo across launches #3284 (comment) record five separate built-CLI processes using 1,250 synthetic events in 25 files. With transcripts and the primary JSON cache unreadable, the second process retained 1,375,000 tokens and USD 12.625. Further runs rejected a poisoned old-semantics report, updated totals after an append, and repriced after catalog replacement. The owner also reports passing make check, 18 focused tests, the full suite, and CI on this head. Captured context sourceRevision: 546d3e3fdabb466822ed4a0d1d752371c1c34ef980115131eaee3d9030af0f83. (1d690b69c7cd)

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (4 earlier review cycles)
  • reviewed 2026-08-30T10:00:45.003Z sha 778b379 :: needs real behavior proof before merge. :: [P2] Version the persisted report against report semantics
  • reviewed 2026-09-08T03:25:46.424Z sha 6ac39a2 :: needs changes before merge. :: [P2] Include the report discriminator in both new JSON fixtures
  • reviewed 2026-09-08T03:37:38.945Z sha 1d690b6 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-08T03:43:22.415Z sha 1d690b6 :: needs maintainer review before merge. :: none

Integrate current main's transcript identity protections and reject saved
reports with missing or incompatible report semantics. Preserve pricing,
source-inventory, cancellation, and report-field round-trip behavior.

Co-authored-by: sunke <sunke@keep.com>
@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Sep 8, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Sep 8, 2026
@steipete

steipete commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Maintainer verification complete on 1d690b69c7cd96e5f05d08cebb018fefb5e0fbc2.

  • make check passed; local and final branch autoreview are clean through P2.
  • Final full make test: 1,033 selections across 87 groups, with no failures, retries, or timeouts. The focused memo suite also passes all 18 tests, including valid poisoned reports with missing/old/future semantic revisions, corrupt sidecars, cancellation, cold repricing, and complete field round trips.
  • Five separate processes ran the real built CodexBarCLI cost --provider claude --days 1 --format json against 1,250 synthetic events in 25 files. The second returned the same 1,375,000 tokens / USD 12.625 while both the transcripts and JSON cache were unreadable, proving persisted reuse. A valid poisoned old-semantics memo was rejected and rebuilt. Appending an event produced 1,376,100 tokens / USD 12.6351; changing catalog pricing produced USD 25.1451. Assertions passed; owned file modes were restored afterward.
  • CI run 34183942607 succeeded on this exact head.

Changelog is consolidated in #3491. Land-ready; no merge performed here. Contributor credit remains preserved for @eggyrooch-blip / sunke.

@clawsweeper clawsweeper Bot added rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Sep 8, 2026
@steipete
steipete merged commit d9f019d into steipete:main Sep 8, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦞 diamond lobster Very strong PR readiness with only minor maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants