-
Notifications
You must be signed in to change notification settings - Fork 0
12 · Move the SDK and binary pins together and register the missing tools (issue #14) #69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
arena-ai-coding-agent
wants to merge
60
commits into
arena/01a097c4-mauscode
Choose a base branch
from
arena/01a0cbec-mauscode
base: arena/01a097c4-mauscode
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
60 commits
Select commit
Hold shift + click to select a range
f4679f2
Move the SDK, Claude CLI and Codex binary pins together
Owie6789 7f99259
Classify the whole 0.3.270 stream dialect and map the two new events
Owie6789 20f92b9
Register the renamed sub-agent and background task tools
Owie6789 76a1412
Make the effort, adaptive thinking and prompt suggestion surfaces rea…
Owie6789 f660175
Add the three drag and drop packages the roadmap ratified
Owie6789 f5c1044
Declare the sharp the icon script already imports
Owie6789 b66b13c
Keep the SDK's per-platform native binary out of the packaged app
Owie6789 b4e5c7c
Record what the SDK bump changed and what it costs
Owie6789 eb4a2bf
Keep a prompt suggestion inside the turn that produced it
Owie6789 df4b974
Register every name the pinned CLI folds into TaskOutput and TaskStop
Owie6789 57e8cd3
Hoist the prompt-suggestion generator out of the transformer closure
Owie6789 3306846
Mark the effort sub-menu props read-only
Owie6789 685aebd
State the new turn-control default once instead of per backend
Owie6789 dc182be
Give the transport one handler per chunk side effect
Owie6789 fe09b5a
Take the chunk helpers that were extracted from this transport
Owie6789 2c8be15
Record what the review round changed and what it left standing
Owie6789 43ff64d
Collapse the provider feature flags onto one all-off default
Owie6789 e57175b
Share one Claude model picker hook between the two surfaces
Owie6789 dbb9457
Take the SDK's own sendMessages options type in both transports
Owie6789 b75064c
Record the duplication sweep in the step-12 decision file
Owie6789 19ffbe1
Record the round-2 gate result and the complexity finding it cleared
Owie6789 41688f4
Select the Claude model from the list the picker actually shows
Owie6789 e5b602a
Give the ten backend manifests one probe runner
Owie6789 5a98fb4
Pin the transcript renderer's output before restructuring it
Owie6789 1900d14
Take the transcript dispatcher out of its closure
Owie6789 cacd3db
Say what a probe's exit code means in one place
Owie6789 97f4327
Take the nested ternary out of the plan indicator
Owie6789 5ea0b64
Record what the third review round was worth
Owie6789 dfe7044
Satisfy the pinned SDK's MCP peer contract
Owie6789 ead68c1
Keep a malformed provider line from taking the process down
Owie6789 6804c04
Say what a subagent actually did, and keep its descendants with it
Owie6789 b4c9ebb
Only an actionable subtitle wears button semantics
Owie6789 34ea81b
Give each prompt suggestion a turn, an engine and a preference
Owie6789 5e24490
Accept a suggestion without discarding the draft under it
Owie6789 50c6b2c
Name the prompt-suggestions switch for assistive tech
Owie6789 29a11b1
Give each sub-chat its own Claude effort, with a shared default
Owie6789 79294c9
Send the chosen effort down the Native transport too
Owie6789 5bbd966
Correct the docs the review proved wrong, and ratify the pins
Owie6789 3018d7c
Record round 4: twenty-one threads, sixteen fixes, four declines
Owie6789 c84a571
Reap the Qwen child when a malformed line ends the turn
Owie6789 e429440
Withdraw the suggestion when a provider error chunk arrives
Owie6789 3c3500e
Hide a stored suggestion once the preference is turned off
Owie6789 faae395
Compare the nesting map by content, so task-row memo survives
Owie6789 8b0ee3d
Give a tooltip-only subtitle a tab stop without calling it a button
Owie6789 1fe7cf9
Clear every open Sonar finding on this pull request
Owie6789 8071737
Record round 5 and the Sonar leak-period triage
Owie6789 1e415ee
Drop the redundant undefined unions on the session-open input
Owie6789 5683525
Compare the nesting map by a pure per-render fingerprint
Owie6789 bcdf47b
Record rounds 6 and 7 of the review
Owie6789 172fbb2
Serialize the fingerprint tuple instead of joining it
Owie6789 3e81a3d
Record the Sonar follow-up on the fingerprint fix
Owie6789 6e90855
Let the outer message memo see in-place tool IO
Owie6789 fe08956
Record round 8 and the outer-memo blind spot
Owie6789 920a934
Keep tool-call header subtitles off the tab order for Sonar S6845
Owie6789 646b8e8
Bound the round-8 serializers to parts that can still change
Owie6789 d2ee05c
Record round 9: S6845, the settled serializers, reset five
Owie6789 b9861ae
Split the fingerprint segment out of nestingFingerprintOf for Sonar
Owie6789 0fc9fb6
Read the fingerprint settle check through optional chains for Sonar
Owie6789 baa8884
Advance the message snapshot when a comparison comes back equal
Owie6789 5f16a60
Track result, error, and errorText wherever rendered fields are compared
Owie6789 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| # Step 12 benchmark record: SDK 0.3.270, CLI 2.1.270, Codex 0.154.0 | ||
|
|
||
| Date: 2026-09-23. Track: app (`arena/01a0cbec-mauscode`, issue #14). Status: | ||
| install-time weight measured in this sandbox; bundle size per target and turn | ||
| latency NOT measured, so the benchmark gate stays CLOSED for those two and CI | ||
| owns them. | ||
|
|
||
| ## Environment | ||
|
|
||
| Sandbox: 2 CPUs, 3 GB RAM, no display, egress intercepted for the hosts that | ||
| serve the Claude and Codex binaries. `.dump/app/roadmap/12-sdk-and-pins.md` §11 | ||
| asks for `NODE_OPTIONS=--max-old-space-size=4096 bun run build` and | ||
| `bun run package:linux`. Neither can run here: the heap flag alone exceeds | ||
| available memory. Both download scripts were attempted and both failed on egress | ||
| rather than on the pins, `claude:download` losing the TLS socket to the object | ||
| store and `codex:download` failing leaf-certificate verification. Everything | ||
| below is either a measurement taken here with the command that produced it, or an | ||
| explicit NOT RUN. | ||
|
|
||
| ## NOT RUN | ||
|
|
||
| | Measurement | Why | Who runs it | | ||
| | --- | --- | --- | | ||
| | Bundle size per target, with and without the bump | needs `bun run build` at a 4 GB heap | CI | | ||
| | Bundle size per target, with and without `@dnd-kit` | same, and step 30 owns the renderer budget | CI, then step 30 | | ||
| | Packaged artifact size, with and without the `build.files` exclusion | needs `bun run package:linux` | CI | | ||
| | Turn latency for one scripted prompt, before and after | needs a packaged app and provider credentials | a dev machine | | ||
| | `bun run claude:download` and `codex:download` integrity runs | sandbox egress is intercepted | CI | | ||
|
|
||
| ## Measured here | ||
|
|
||
| Install weight, `du -sh` on this checkout after all four dependency commits: | ||
|
|
||
| | Path | Size | | ||
| | --- | --- | | ||
| | `node_modules` | 1.9 GB | | ||
| | `node_modules/@anthropic-ai` | 440 MB | | ||
| | `node_modules/@anthropic-ai/claude-agent-sdk-linux-x64` | 214 MB | | ||
| | `node_modules/@anthropic-ai/claude-agent-sdk-linux-x64-musl` | 208 MB | | ||
| | `node_modules/@anthropic-ai/sdk` | 14 MB | | ||
| | `node_modules/@anthropic-ai/claude-agent-sdk` | 5.0 MB | | ||
| | `node_modules/@img` (sharp's platform binaries) | 37 MB | | ||
| | `node_modules/@dnd-kit` (three packages) | 2.1 MB | | ||
| | `node_modules/sharp` | 1.3 MB | | ||
|
|
||
| The two SDK platform packages are the whole story: 422 MB of a 1.9 GB install, | ||
| and they did not exist at `0.2.45`, which spawned bundled JavaScript instead of a | ||
| native binary (SDK changelog `0.2.113`). | ||
|
|
||
| `manifest.json` at `0.3.270`, the size each packaged target would have carried | ||
| without the `build.files` exclusion: | ||
|
|
||
| | Platform | Bytes | MB | | ||
| | --- | --- | --- | | ||
| | darwin-arm64 | 207500480 | 197.9 | | ||
| | darwin-x64 | 216316928 | 206.3 | | ||
| | linux-arm64 | 223862184 | 213.5 | | ||
| | linux-x64 | 223981040 | 213.6 | | ||
| | linux-arm64-musl | 216545032 | 206.5 | | ||
| | linux-x64-musl | 217894976 | 207.8 | | ||
| | win32-x64 | 227051168 | 216.5 | | ||
| | win32-arm64 | 218256032 | 208.1 | | ||
|
|
||
| Lockfile rows: 2888 at `33475d8`, 2960 after all four dependency commits, with 16 | ||
| package rows added and 16 removed by the pin bump itself. | ||
|
|
||
| Integrity, the one check this sandbox can make: `sha256sum` of | ||
| `node_modules/@anthropic-ai/claude-agent-sdk-linux-x64/claude` is | ||
| `3a624a5a7cd79bbad4d32bd7db36f1197ecf458bc5bf1e2aed81834a01ad3ef0`, which equals | ||
| the `linux-x64` checksum in the SDK's `manifest.json`, size `223981040`. The SDK | ||
| pin and the CLI pin are therefore the same artifact, not two independent guesses. | ||
|
|
||
| Gate wall clocks on the same 2 CPU sandbox, as a regression proxy rather than a | ||
| product number. Measured at the head that added this file, before the render | ||
| test harness landed — the counts are that run's, kept because the timings | ||
| beside them are its own: `npm run test` over 103 files and 1885 tests took | ||
| 50.5 s; `npm run test:node` 59 tests took 39.0 s; `npm run test:contracts` 382 | ||
| tests took 10.5 s; `npx biome check .` over 977 files took 3 s; `tsc | ||
| --noEmit` over the whole repo took 42 s to 65 s per run and reports 0 errors, | ||
| which is the ratchet's baseline rather than new debt. The suite has grown since | ||
| (harness, review fixes): the PR's CI quality job on the final head is the live | ||
| count, and it — not this snapshot — is what a later comparison reads. | ||
|
|
||
| ## What the numbers mean for the next step | ||
|
|
||
| Step 30 records the renderer budget, and `@dnd-kit` is the dependency a | ||
| performance claim will be checked against: 2.1 MB installed, three packages, no | ||
| importer yet, so its bundle cost is still zero and becomes measurable the moment | ||
| step 17 mounts the shared `DndContext`. The `build.files` exclusion is worth | ||
| 213.6 MB on a linux-x64 artifact and 207.8 MB on linux-x64-musl, which is the | ||
| difference between a packaged app that grew by about a fifth and one that did not. | ||
| The `sharp` declaration costs 37 MB of platform binaries in a dev install and | ||
| nothing in a packaged app, because it is a devDependency and the icon script runs | ||
| before packaging. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moderate
Location:
.dump/app/benchmarks/2026-09-23-sdk-0-3-pins.md:26Diff cause: This added benchmark explicitly leaves packaged artifact size unmeasured, while the PR changes native binary pins and makes a packaged-size claim.
Evidence: The package logs show raw Claude resources growing by about 0.9 MB and Codex resources by about 10.6 MB, for at least 11.5 MB of combined raw resource growth before compression. The record has no final artifact measurement, no asar inspection, and no with-and-without
@dnd-kitresult.Impact: The claimed packaged-size result is unverified, and the native resource increase can change the release artifact even when renderer JavaScript remains stable. A later performance decision has no reproducible baseline.
Required change: Record base and head unpacked and compressed artifact sizes, inspect the packaged archive, measure the dnd dependency separately, and correct the PR claim if the increase is intentional.
Verification: Run the same package command on base and head, record artifact byte counts and compression, and attach the asar contents and delta to the benchmark.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Declined — this sandbox cannot produce the artifact, and the record says so rather than claiming a measurement it does not have.
The benchmark's packaged-size and bundle-size rows are explicitly marked needs
bun run package:linuxand owned by CI / step 30 — the rows exist to name the measurement that CI must fill in, not to assert a number this environment never took. Your raw-resource deltas (~0.9 MB Claude, ~10.6 MB Codex) are the right input to that measurement and are consistent with the binary-pin moves; they are not themselves an artifact size (compression, asar packing, and platform all sit between resources and the release file).Filling the row with an estimate would violate the record's own rule: no measurement it does not have. When CI or step 30 runs the packager, the row gets base and head numbers with the archive inspected.