Skip to content

Add Muse provider (Meta Muse) - #3340

Open
sanjay3290 wants to merge 5 commits into
steipete:mainfrom
sanjay3290:feat/muse-provider
Open

Add Muse provider (Meta Muse)#3340
sanjay3290 wants to merge 5 commits into
steipete:mainfrom
sanjay3290:feat/muse-provider

Conversation

@sanjay3290

@sanjay3290 sanjay3290 commented Sep 1, 2026

Copy link
Copy Markdown

Adds Muse (Meta Muse Code) as provider #70.

Muse reports local token usage, read from the session logs the CLI writes.
It reports no quota — Meta publishes no endpoint for one, verified below.

Where the numbers come from

Muse Code records every model turn to
~/.local/share/muse/sessions/<Y>/<M>/<D>/<session>/session.jsonl, so CodexBar
derives the same local token history it already builds for Claude and Codex —
no network call, no credential, no Keychain access.

Token semantics were verified across 1,431 recorded events, without exception:

Relation Meaning
reasoning_tokensoutput_tokens reasoning is part of output
cached_tokensinput_tokens cached is part of input
cached_tokens == cache_read_tokens same value, always

A turn totals input + output. Summing the cache or reasoning counters would
double-count — one sampled turn reported 41,201 cached tokens against a
41,231-token input. automated_review_completed carries its own total_tokens,
which equalled input + output in every observed event.

Two record kinds also carry a usage object and are excluded:
resource_usage_sampled (CPU/RSS gauges, not tokens — 4,393 of them against
1,235 real turns) and workflow_child_lifecycle (a child's rollup, already
recorded on its own). An unrecognized kind carrying token counts downgrades
coverage to partial rather than vanishing from the totals.

Why there is no quota

With a valid API key, every usage-shaped path returns 404 — identical to a
path that does not exist:

Path Status
/v1/usage, /v1/billing/usage, /v1/me, /v1/account, /v1/credits, /v1/limits, /v1/quota, … 404
/v1/zzz_nonexistent 404
/v1/models 200

The documented x-ratelimit-* headers are real, but ride only on billed
inference responses:

GET /v1/models  → 200, no rate-limit headers
GET /v1/status  → 200, no rate-limit headers
POST /v1/chat/completions (max_tokens=1) → 200
  x-ratelimit-limit-requests: 3000
  x-ratelimit-remaining-requests: 3000
  x-ratelimit-limit-tokens: 4000000
  x-ratelimit-remaining-tokens: 4000000

Reading them would mean issuing a billed completion on every refresh, which
would also consume the limit it reports; and being per-minute, they read at ~0%
except mid-burst. So the API key is used only to validate itself against the
free GET /v1/models (200 vs 401).

Scanning cost

An 883 MB tree of 4,388 logs: cold scan 16 s, warm scan 0.26 s, identical
totals. Day directories outside the history window are skipped unopened, lines
without an input_tokens field are rejected before JSON parsing, and each
file's size/mtime/per-day totals are cached. A budget-exhausted scan keeps the
files it finished and resumes next refresh.

Verification

  • Reader output cross-checked against an independent reference implementation on
    a frozen snapshot — identical to the token:
    1,235 requests · 112,776,365 in · 1,047,888 out · 113,824,253 total
  • swiftlint --strict → 0 violations / 2,090 files
  • swiftformat --lint → 0 files need formatting
  • Scripts/regenerate-provider-manifests.sh --check → 70 providers
  • check-documentation-links.mjs → 191 links · check-app-locales.mjs → OK
  • 35 Muse tests, 39 architecture-gatekeeper tests passing

Review feedback

Both Codex findings are fixed, and the code they referenced is gone:

  • P1 credential leak — the hardcoded api.meta.ai candidate is removed.
    MUSE_BASE_URL now goes through ProviderEndpointOverrideValidator and
    throws rather than falling back to Meta. Pinned by two tests.
  • P2 muse auth --help as auth checkmuse auth only offers auth set;
    there is no non-interactive status command. Login state now comes from
    ~/.config/muse/auth.json, which also supplies the real account email. Only
    the plaintext metadata is parsed, so no Keychain prompt is possible.

Also registered Muse in the architecture gatekeeper, the token-account
credential catalog and the dashboard cost contract — all three were missing and
would have failed CI.

- Add UsageProvider.muse with descriptor, settings, fetcher
- Support META_API_KEY / MUSE_API_KEY and CLI probe (muse --version)
- Probe api.meta.ai/v1/usage candidates with fallback to identity card
- Add BinaryLocator.resolveMuseBinary, icon, docs, manifests (70 providers)
- Update README and providers overview
@clawsweeper

clawsweeper Bot commented Sep 1, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-01T03:20:23.877740Z 7cc7f8b PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7cc7f8b2cc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

baseURL.appendingPathComponent("usage"),
baseURL.appendingPathComponent("billing/usage"),
baseURL.appendingPathComponent("me"),
URL(string: "https://api.meta.ai/v1/usage")!,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep overridden credentials on the configured host

When MUSE_BASE_URL points to a custom endpoint and its candidate routes return 404/501, unusable JSON, or a network error, this unconditional fallback sends the same bearer credential to api.meta.ai. A key intended only for a proxy, test server, or enterprise endpoint is therefore disclosed to an unrelated host; candidate URLs should remain relative to the configured base URL unless the base URL is the default Meta host.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — the unconditional api.meta.ai candidate is gone, along with the whole
candidate list. With a valid key those paths return 404 (identical to a
nonexistent path), so they were never real endpoints.

MUSE_BASE_URL now resolves through the existing ProviderEndpointOverrideValidator
seam and throws MuseUsageError.invalidEndpointOverride instead of falling back to
Meta, so a key scoped to a private gateway cannot reach api.meta.ai. Two tests
pin it: "rejected base URL override never falls back to the Meta host" and
"fetch keeps the credential on the configured host".

Comment on lines +162 to +163
let loginCheck = ShellCommand.run(binary, args: ["auth", "--help"], timeoutSeconds: 5)
let isAuthenticated = loginCheck.exitCode == 0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not treat help output as an authentication check

When the Muse binary is installed but the user has never logged in, muse auth --help can still exit successfully because it only displays command help; the accompanying docs/muse.md:17 likewise describes it as a reachability check. Setting isAuthenticated from that exit status bypasses the missing-credentials error and reports a successful “Muse CLI” identity for an unauthenticated installation, so this needs an actual noninteractive auth-status or credential check.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, and confirmed against the real CLI: muse auth only offers auth set
there is no non-interactive auth-status command, so the help-exit-code check
could never have been correct.

Login state now comes from ~/.config/muse/auth.json, which muse login writes.
It also yields the real account email and login mechanism, so the identity card
is populated rather than empty. Only the plaintext metadata is read; the
credential itself stays in the Keychain and is never touched, so refreshing Muse
cannot raise a Keychain prompt. docs/muse.md is corrected too — it previously
claimed Keychain storage, which was wrong.

The Meta Model API publishes no usage, billing, or account endpoint, so the
previous probe guessed at /usage, /billing/usage and /me and presented an
"API Key" card whenever they failed. Replace the guesswork with the
rate-limit headers Meta documents, read from one GET /v1/models so a refresh
never spends tokens.

- Derive tokens-per-minute and requests-per-minute windows from
  x-ratelimit-limit/remaining-tokens/requests.
- Keep the credential on the configured host: drop the hardcoded
  api.meta.ai candidate, and validate MUSE_BASE_URL like every other
  provider endpoint instead of silently falling back to Meta.
- Surface transport, server and endpoint failures instead of reporting a
  configured-and-healthy provider.
- Read account identity from ~/.config/muse/auth.json, which muse login
  writes, rather than inferring login state from `muse auth --help` (it
  exits 0 whether or not anyone is logged in). Only the plaintext metadata
  is parsed, so no Keychain prompt is possible.
- Drop the local process runner in favour of no CLI spawn at all, which
  also removes the Muse binaryLocator the architecture gatekeeper rejects.
- Accept the documented MODEL_API_KEY alongside META_API_KEY, drop the
  invented MUSE_API_KEY, and fix the dashboard, changelog and status links.
- Remove the unreachable museBaseURL setting and the empty token-load stub.
- Add MuseProviderTests and refresh docs/muse.md.
@clawsweeper clawsweeper Bot added merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Sep 1, 2026
@clawsweeper

clawsweeper Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed September 1, 2026, 10:24 AM ET / 14:24 UTC.

ClawSweeper review

What this changes

The PR adds a Muse provider that validates Meta API keys and reads local Muse session logs to show account identity and daily token usage without inventing a quota.

Merge readiness

⚠️ Ready for maintainer review - 3 items remain

The prior findings are resolved and the final head has no discrete correctness defect; this remains open for explicit owner approval because it adds a supported core provider and an ongoing third-party local-log contract.

Priority: P2
Reviewed head: 9d37b79264a72aee6917c10f8ce2060752525c97
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) Strong real-log and API evidence plus focused tests support a good implementation; provider membership remains a maintainer product decision rather than a patch-quality defect.
Proof confidence 🦞 diamond lobster (5/6) Sufficient (live_output): The final production reader in the Muse provider parses durable local session logs into a shared token snapshot, and the contributor supplied a redacted after-fix comparison against an independent reader on a frozen 883 MB real log tree plus live API endpoint observations; the final follow-up only corrected documentation.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (live_output): The final production reader in the Muse provider parses durable local session logs into a shared token snapshot, and the contributor supplied a redacted after-fix comparison against an independent reader on a frozen 883 MB real log tree plus live API endpoint observations; the final follow-up only corrected documentation.
Evidence reviewed 7 items Current main does not already include Muse: The fetched current main tree contains none of the Muse provider, app implementation, or Muse documentation paths; the feature is still unique to this branch.
Local usage path is bounded and registered: The introduced cost path routes Muse to its dedicated local snapshot reader, while the descriptor marks it as a token-snapshot provider with no fabricated quota windows.
Prior cache-documentation finding is resolved: The final documentation names the version-2 cache and describes its per-event rows, matching the implementation after the prior P3 finding.
Findings None None.
Security None None.

How this fits together

CodexBar collects provider credentials, account status, and local usage into shared menu-bar and dashboard views. Muse inputs local CLI metadata, session logs, and an optional API key, then emits identity and token snapshots through the common provider registry.

flowchart LR
  A[Muse login metadata] --> C[Muse provider]
  B[Muse session logs] --> D[Local token reader]
  E[Optional API key] --> C
  C --> F[Account identity]
  D --> G[Daily token snapshot]
  F --> H[CodexBar menu and dashboard]
  G --> H
Loading

Decision needed

Question Recommendation
Should CodexBar accept Muse’s local session-log schema as a supported core provider contract? Approve core Muse support: Accept this local-log-backed provider and its documented partial-coverage behavior as a supported CodexBar surface.

Why: The implementation is coherent and no replacement core path exists, but adding a provider creates an ongoing product and maintenance commitment that code review cannot make on behalf of the repository owner.

Before merge

  • Resolve merge risk (P1) - Merging creates a core support commitment for Muse’s externally controlled session-log schema; CI cannot establish that future Muse releases will preserve the observed record shapes, although the reader deliberately reports partial coverage on unknown token-bearing records.
  • Complete next step (P2) - A repository owner must decide whether this new provider’s ongoing support contract belongs in core; no mechanical repair remains.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Implementation and regression coverage production +1,239, tests +759, docs +117 The sizeable provider addition includes focused parser, credential, and architecture-gate coverage rather than only registry wiring.
Changed surface 24 files; 2,098 added, 30 removed The change spans core parsing, application settings, registration, documentation, and tests, so provider-support approval should consider the full maintenance surface.

Merge-risk options

Maintainer options:

  1. Accept the Muse log-contract commitment (recommended)
    Merge after confirming that maintaining compatibility with Muse’s evolving local session records is an acceptable core-provider obligation.
  2. Pause the provider addition
    Leave the PR unmerged if the repository does not want to support a third-party local-log schema without an authoritative stability commitment.

Technical review

Best possible solution:

Approve and maintain the native Muse provider only if core support for its local-log contract is desired; otherwise keep the existing provider architecture unchanged rather than landing an unsupported surface.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug reproduction: this PR adds a provider rather than repairing an established broken behavior. The contributor did supply live API observations and a frozen real-log parsing comparison for the new behavior.

Is this the best way to solve the issue?

Yes, conditionally: it follows CodexBar’s existing descriptor and local-token-reader patterns and fixes all retained prior findings, but only a maintainer can decide whether Muse belongs in the supported core provider set.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against eb290548a739.

Labels

Label justifications:

  • P2: This is a bounded provider addition with normal user impact and no evidence of an urgent production regression.
  • merge-risk: 🚨 other: The new core surface depends on Muse’s externally controlled local session-log schema, a maintenance risk that passing repository checks cannot settle.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🦞 diamond lobster and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The final production reader in the Muse provider parses durable local session logs into a shared token snapshot, and the contributor supplied a redacted after-fix comparison against an independent reader on a frozen 883 MB real log tree plus live API endpoint observations; the final follow-up only corrected documentation.
  • proof: sufficient: Contributor real behavior proof is sufficient. The final production reader in the Muse provider parses durable local session logs into a shared token snapshot, and the contributor supplied a redacted after-fix comparison against an independent reader on a frozen 883 MB real log tree plus live API endpoint observations; the final follow-up only corrected documentation.

Evidence

What I checked:

  • Current main does not already include Muse: The fetched current main tree contains none of the Muse provider, app implementation, or Muse documentation paths; the feature is still unique to this branch. (eb290548a739)
  • Local usage path is bounded and registered: The introduced cost path routes Muse to its dedicated local snapshot reader, while the descriptor marks it as a token-snapshot provider with no fabricated quota windows. (Sources/CodexBarCore/CostUsageFetcher.swift:455, 9d37b79264a7)
  • Prior cache-documentation finding is resolved: The final documentation names the version-2 cache and describes its per-event rows, matching the implementation after the prior P3 finding. (docs/muse.md:54, 9d37b79264a7)
  • Focused coverage addresses the earlier overlap defect: The local-reader tests cover partially overlapping logs, token accounting, unknown schema drift, windowing, and warm-cache reuse. (Tests/CodexBarTests/MuseLocalUsageReaderTests.swift:208, 9d37b79264a7)
  • Credential routing keeps the configured host: The endpoint resolver rejects invalid overrides and the API fetcher performs one GET against the resolved base URL; focused tests assert no fallback to api.meta.ai. (Sources/CodexBarCore/Providers/Muse/MuseSettingsReader.swift:22, 9d37b79264a7)
  • Runtime proof supplied by contributor: The PR discussion records post-change API observations and an independent comparison of the final local-reader totals on a frozen 883 MB Muse session tree; the final commit after that evidence changes only the corrected documentation. (9d37b79264a7)

Likely related people:

  • steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
  • Yuxin-Qiao: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Obtain explicit approval to support Muse’s local-log contract as a core provider.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (3 earlier review cycles)
  • reviewed 2026-09-01T04:02:36.700Z sha 7cc7f8b :: needs real behavior proof before merge. :: [P1] Keep custom-host credentials off api.meta.ai | [P2] Use a real CLI authentication signal
  • reviewed 2026-09-01T06:35:36.087Z sha 825e7c8 :: needs changes before merge. :: [P2] Deduplicate individual events, not an entire overlapping log | [P2] Correct the local-source remediation message
  • reviewed 2026-09-01T13:56:07.211Z sha 6a2d063 :: found issues before merge. :: [P3] Correct the documented Muse cache filename

Muse Code records every model turn to
~/.local/share/muse/sessions/<Y>/<M>/<D>/<session>/session.jsonl, so CodexBar
can build the same local token history it already derives for Claude and
Codex, with no network call, no credential and no Keychain access. This
replaces the placeholder card with real numbers.

Token semantics were checked against 1,431 recorded events: reasoning_tokens
is a subset of output_tokens, and cached_tokens/cache_read_tokens are subsets
of input_tokens and always equal each other. A turn therefore totals
input + output; summing the cache or reasoning counters would double-count, in
one sampled turn by 41,201 tokens against a 41,231-token input. The
automated_review_completed shape carries its own total_tokens, which matched
input + output in every observed event.

Two record kinds also carry a usage object and are excluded:
resource_usage_sampled holds CPU and RSS gauges, and workflow_child_lifecycle
repeats a child's turns. An unrecognized kind carrying token counts downgrades
coverage instead of vanishing from the totals.

Scanning stays cheap on large trees. Day directories outside the history
window are skipped unopened, lines without an input_tokens field are rejected
before JSON parsing, and each file's size, mtime and per-day totals are cached
so an unchanged log is never reread. On an 883 MB tree of 4,388 logs a cold
scan took 16s and a warm scan 0.26s, for totals identical to an independent
reference implementation. A scan that exhausts its budget keeps the files it
finished, so the next refresh resumes.

No quota is shown. Every usage, billing and account path returns 404 with a
valid key, and the documented x-ratelimit-* headers ride only on billed
inference responses, so reading them would spend tokens on every refresh and
consume the limit being reported. The API key is now used only to validate
itself against GET /v1/models.

Also register Muse in the architecture gatekeeper, the token-account
credential catalog and the dashboard cost contract, which the provider needed
and did not have.
@sanjay3290

Copy link
Copy Markdown
Author

@clawsweeper re-review

Both P1 and P2 are fixed, and the PR now carries the behavioural proof that was
missing. Summary of what changed since 7cc7f8b:

  • Credential boundary: the hardcoded api.meta.ai fallback is removed and
    MUSE_BASE_URL goes through ProviderEndpointOverrideValidator, throwing rather
    than retargeting Meta. Regression tests included.
  • Auth signal: muse auth --help is gone; login state is read from
    ~/.config/muse/auth.json (no Keychain access).
  • Test coverage: 631 production lines previously had 0 tests. Now 35 focused
    tests across MuseProviderTests and MuseLocalUsageReaderTests.

On the speculative-contract concern: rather than guessing, the provider was
rebuilt on local session logs. Live proof, redacted:

GET /v1/usage, /v1/billing/usage, /v1/me (valid key) → 404
GET /v1/zzz_nonexistent                              → 404
GET /v1/models                                       → 200, no ratelimit headers
POST /v1/chat/completions (max_tokens=1)             → 200
  x-ratelimit-limit-requests: 3000 / remaining: 3000
  x-ratelimit-limit-tokens: 4000000 / remaining: 4000000

So the rate-limit headers exist only on billed inference, and no usage endpoint
exists at all. The provider therefore reports local token usage and no quota.

Reader output vs an independent reference implementation on a frozen 883 MB
snapshot — identical:
1,235 requests · 112,776,365 in · 1,047,888 out · 113,824,253 total
Cold scan 16s, warm scan 0.26s via a size+mtime cache.

@clawsweeper

clawsweeper Bot commented Sep 1, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 Urgent regression or broken agent/channel workflow affecting real users now. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. labels Sep 1, 2026
The cache aggregated each log's turns into per-day totals before storing them,
so overlap could only be resolved whole-file: a log holding one already-counted
event alongside unique ones was dropped entirely, silently under-reporting.

Store one row per recorded turn instead. Deduplication now skips exactly the
repeated ids and keeps the rest, and a partial-overlap regression covers it.
Totals on a frozen 883 MB tree are unchanged (1,235 requests, 113,824,253
tokens); the warm scan stays at ~0.2s.

Also correct the local strategy's diagnostic. It still advised setting
META_API_KEY for usage, left over from the earlier rate-limit design; usage
comes from the local logs and an API key only validates identity.
@sanjay3290

Copy link
Copy Markdown
Author

Both findings fixed in 6a2d063.

Deduplicate individual events, not an entire overlapping log — correct, and
the cause was the cache shape: each log's turns were aggregated into per-day
totals before being stored, so overlap could only be resolved whole-file. The
cache now stores one row per recorded turn, so deduplication skips exactly the
repeated ids and keeps the rest. Regression added — "a partially overlapping log
keeps its unique turns": two logs sharing one event id, where the second also
carries a unique turn, yields 2 requests rather than 1.

Totals on the frozen 883 MB tree are unchanged after the rewrite
(1,235 requests · 112,776,365 in · 1,047,888 out · 113,824,253 total), and the
warm scan is still ~0.2s.

Correct the local-source remediation message — fixed. It now reads
"Token usage comes from local Muse session logs; an API key only validates
identity," which matches docs/muse.md.

swiftlint --strict 0 violations, swiftformat clean, manifests 70, doc links 191,
14 local-reader tests and 39 gatekeeper tests passing.

The cache moved to per-event rows in v2; docs still named v1 and per-day totals.
@sanjay3290

Copy link
Copy Markdown
Author

P3 fixed in 9d37b79docs/muse.md named muse-sessions-v1.json with per-day
totals, but the cache moved to v2 with per-event rows when the deduplication
finding was addressed. The doc now describes the shipped shape.

Full suite run

GitHub Actions is gated on maintainer approval for this fork PR
(github-actions check suite reports action_required), so the suite has not
run in CI. I ran the repo's own sharded harness locally instead:

Scripts/ci_swift_test_by_suite.py --shard-index N --shard-count 8

shard 0/8  clean          shard 4/8  clean
shard 1/8  clean          shard 5/8  clean
shard 2/8  clean          shard 6/8  clean
shard 3/8  1 failure      shard 7/8  clean

986 discovered selections. The single failure is
browser uninstall invalidates cookie source immediately
(BrowserDetectionTests), which reproduces identically on upstream
8a732e743 with none of this branch's code — it is environment-dependent on a
machine with Chrome installed, not a regression from this PR.

One note for anyone else running that harness on macOS: Scripts/test.sh calls
python3, and when that resolves to Xcode's bundled 3.9.6 the run aborts with
"Swift test process containment requires waitid with WNOWAIT", because
os.waitid is missing there. Any Python 3.13+ works.

Still unverified on my side and worth CI confirming: the build on CI's Swift
6.3.3 toolchain, and the suite on a machine without Chrome.

@clawsweeper clawsweeper Bot removed status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Sep 1, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Sep 1, 2026
@NatanSlvdr

Copy link
Copy Markdown

Meta added subscriptions for Muse Code with included usage, would be cool to integrate it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants