Add Helmcode usage provider - #3422
Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 99a71143a1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Codex review: needs real behavior proof before merge. Reviewed September 5, 2026, 3:07 PM ET / 19:07 UTC. ClawSweeper reviewWhat this changesAdds Helmcode Cloud and NaN Builders usage tracking through dashboard cookies, with deployment selection, cached sessions, quota displays, documentation, and tests. Merge readiness⛔ Blocked before merge - 6 items remain This remains a useful, distinct provider addition. Earlier functional concerns are addressed and the new live evidence demonstrates meaningful progress, but the saved-cookie security boundary is not fully established by the inspectable proof. Priority: P2 Review scores
Verification
How this fits togetherCodexBar providers turn account usage responses into menu-bar and CLI summaries. This provider selects a dashboard deployment, obtains its session cookies, and converts quota and optional billing responses into usage windows. flowchart TD
A[Deployment and cookie settings] --> C[Select dashboard deployment]
B[Chrome session or saved cookies] --> C
C --> D[Filter cookies for each endpoint]
D --> E[Dashboard requests with redirects refused]
E --> F[Validate quota and update session cache]
E --> G[Read optional billing]
F --> H[Menu bar and CLI usage]
G --> H
Decision needed
Why: Positive runtime evidence is inspectable, but negative cases are described as stub-tested and the remaining captured body could not be retrieved. Before merge
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Retain the provider-owned implementation and scoped cookie records, with final-I/O evidence establishing cookie isolation, redirect refusal, and rejection recovery. Do we have a high-confidence way to reproduce the issue? Not applicable: this adds a provider; supplied real runs demonstrate quota retrieval, and no concrete new functional failure was established. Is this the best way to solve the issue? Yes, the provider descriptor, existing cookie cache, and shared transport are appropriate integration points; the remaining concern is boundary evidence rather than a demonstrated need to redesign the patch. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against 3a676e143e23. LabelsLabel justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (8 earlier review cycles)
|
|
Live authenticated proof added — an accepted real dashboard session now refreshes quota end-to-end. Summary of the validation run against the community tenant ( Production importer + fetcher, real session
Full CLI pipeline ( The live run also caught a real bug now fixed in @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
Co-authored-by: Codex <codex@openai.com>
4509073 to
c3adc77
Compare
Summary
periodEnd), rolling-window tiers rendered with their window length, and premium tiers hidden unless the subscription is premium; the prepaid balance is shown on Helmcode Cloud only (NaN Builders membership has none)CookieHeaderCacheso background refreshes and the bundled CLI reuse it without rereading Chrome; a rejected session evicts the entry and requires a fresh user-initiated import/dashboard; the provider icon is the official Helmcode markValidation
swift test --filter "Helmcode|MenuCardHelmcode|ProviderIconResources|ProviderArchitectureGatekeeper"— 94 tests in 8 suites, all passed (re-run independently by a second reviewer session)make check— clean (manifests, locales, packaging, script tests, swiftformat, swiftlint); swiftformat and swiftlint also clean on every changed file underTestsLinuxmake test— 1,016 selections across 85 groups, 85/85 first pass, 0 retries (run on the round-2 tip before the rebase onto origin/main; after the rebase: build, the focused suites above, andmake checkgreen; CI re-runs the full suite on this head)Notes
Helmcode inference API keys do not expose quota or billing, so the provider reads the same authenticated endpoints the dashboards use:
GET /api/usage/quota(required),GET /api/billingandGET /api/billing/credits(best-effort, fail-soft). Both tenants share the quota API; billing differs. The prepaid balance exists on Helmcode Cloud only, and/api/billing/creditsanswers 404 on NaN Builders, which the provider treats as "no balance for this tenant".Live evidence (NaN Builders tenant, redacted) — head
a3eac99fcBuilt with
swift build -c release --product CodexBarCLIata3eac99fc. The session came from a Chrome "Copy as cURL" capture of the dashboard's own/api/usage/quotarequest, read from a 0600 temp file and never printed; every output file was grepped for the cookie value before posting (no match). Token counts below are real.--log-level verboseprints the credential-boundary trace: the tenant decision, then one line per request naming host, path, and credential type (and, for cached sessions, cookie names with expired or path-excluded cookies listed by name).A. cURL capture, no
HELMCODE_DEPLOYMENT— the tenant is detected from the capture's hostB. bare Cookie header, pinned with
HELMCODE_DEPLOYMENT=nanbuildersC. rejected session on NaN Builders (garbage cookie)
D. rejected session on Helmcode Cloud — each pinned tenant only ever talks to its own host
Packaged app (screenshots from the
c3adc7756build; the later commits change credential routing, caching and diagnostics, not the card or Settings layout) — Deployment left on Automatic; the card and Settings show the detected tenant, the source labelweb · NaN Builders, per-model resets, and no premium rows (the dashboard hides the same two tiers as "NOT IN YOUR PLAN"):App refresh trace (system log at verbose level, cookie names only) — two user-initiated refreshes on the packaged app at head
a3eac99fc: the first starts from an empty cache, imports from Chrome, and stores the cookie records only after the quota request succeeds; the second replays the records with the per-endpoint cookie lists and no browser read. Cookie names only, never values. The live NaN session has a single root-path, unexpired cookie and the server did not redirect or reject it, so the exclusion, redirect-refusal, and rejection-then-eviction branches cannot be forced against the real tenant; those are pinned byHelmcodeCachedSessionBoundaryTestsandHelmcodeDeploymentDetectionTestswith exact-line assertions on the same diagnostics (path-excluded cookie never reaches billing, expired cookie dropped, 302 surfaces as a rejected session with the redirect refused, rejected cached tenant evicted before the next candidate). This dev build is ad-hoc signed, so its cache lived in process memory rather than the Keychain; the code path is identical:Provider eligibility
Per the "Hosted relay eligibility" bar in
docs/provider.mdand the criteria applied when declining AnyRouter (#2218): identifiable operator, clearly authorized upstream access, operating track record.1. Identifiable legal operator and jurisdiction. Helmcode and NaN Builders are operated by Helmcode S.L. (NIF B44788511), Avenida Quinto Centenario 27, 38683 Santiago del Teide, Santa Cruz de Tenerife, Spain, under Spanish law. Sources: https://helmcode.com/terms (legal notice naming Helmcode S.L., NIF, and address) and https://nan.builders/terms ("NaN is operated by Helmcode S.L. (NIF B44788511)"), so the community tenant and the enterprise tenant are the same legal entity.
2. Clearly authorized upstream access. Helmcode serves open-weight models and publishes the licence for each one on https://helmcode.com/models: GLM-5.3 under Z.ai's licence ("the MIT text plus one clause, a Z.ai security review for any model-as-a-service business above $10bn of revenue"), DeepSeek V4 Flash (MIT), Qwen3.6 (Apache 2.0), Gemma4 (Gemma licence), plus embedding, rerank, and speech models. The page states models are "served only from the EU". No closed-model API resale is advertised and there is no key pooling: NaN's acceptable-use policy says "Do not share, resell or transfer your API key" (https://nan.builders/terms). The NaN Builders community tier and its published allowances: https://nan.builders/#membership. Helmcode publishes no statement about third-party clients consuming the dashboard endpoints, which is why the provider treats that contract as unversioned (see Risk containment).
3. Public operating track record. Public pricing and status pages (https://helmcode.com/pricing, https://status.helmcode.com), a public changelog with entries since April 2026 (https://helmcode.com/changelog, first entry 2026-04-22), and production customers named on the site (Ayuntamiento de Cartagena, FRS, Easygoband, among others).
Author disclosure. I am a paying NaN Builders subscriber and have no affiliation with Helmcode S.L. I will maintain this provider and respond to dashboard-contract drift.
Risk containment
The provider is default-off and deliberately narrow:
defaultEnabled: false), not widget-selectable; nothing runs unless the user enables it.providerVariantscope), so a NaN Builders session is never replayed tocloud-api.helmcode.comor vice versa; a rejected session evicts the cache and requires a fresh import.HELMCODE_COOKIEkeep precedence and never read or write the cache; a pasted capture is sent only to the tenant detected from its host, never to both.