Skip to content

feat(claude-code): type a model key refusal as model_key_budget or model_key_auth, not agent_error - #710

Draft
andrewpark-scaleai wants to merge 2 commits into
andrew-park/rel-agent-typed-errorsfrom
andrew-park/rel-claude-code-key-error-types
Draft

andrewpark-scaleai wants to merge 2 commits into
andrew-park/rel-agent-typed-errorsfrom
andrew-park/rel-claude-code-key-error-types

Conversation

@andrewpark-scaleai

Copy link
Copy Markdown
Contributor

Stacked on #695 (base is that branch, not main). Waits for the protocol release that contains #695. Do not mark ready before that release is on PyPI.

Why

AgentEnv should say why every run failed, and heal itself where code can. Today a claude-code run whose model key is out of budget or rejected is blamed on the agent. #695 adds the model_key_budget and model_key_auth error types to the protocol. This PR makes the claude-code agent report them.

When the model endpoint refuses the key, the Claude Code CLI exits 1 and its terminal result event names the cause as fields: api_error_status and api_error_type (for example 400 / budget_exceeded from LiteLLM, 401 / authentication_error from Anthropic). The agent's A2A server ignored those fields and called builder.failed("cli_error", ...) with no error_type, so the protocol's default agent_error reached the framework.

What

  • The bridge parses the CLI's terminal result event into a TaskErrorType when the CLI fails. api_error_type == "budget_exceeded" gives model_key_budget. A 401 or 403, or a type of auth_error, authentication_error or permission_error, gives model_key_auth. Anything else stays agent_error. The A2A server passes it to builder.failed.

Why this is a separate PR

The claude-code image installs agentenv-framework-protocol from PyPI at or above the floor in its requirements.txt, not from this tree. CONTRIBUTING.md and AGENTS.md say an agent's floor rises only once the protocol it names is published. This code imports TaskErrorType and sends values the published protocol (0.1.556 and earlier) rejects, so it cannot run in the image until a release carries #695. Shipping it with #695 broke the claude-code image build. I kept it out of #695 rather than adding a try-import or a string fallback that would exist only to straddle one release.

Before marking ready: raise the floor in src/agent_env/agents/agents/claude-code/requirements.txt to the first published release that contains #695, and update tst/unit/agents/claude_code/test_image_contract.py to match. Until then the floor stays at >=0.1.351, and this PR's claude-code integration tests are expected to fail on the missing TaskErrorType import.

Design decisions and tradeoffs

  • Parse once, at the CLI boundary. The bridge already owns the CLI's event parsing (_terminal_result, _status), so the mapping lives there and the server only forwards a typed value.
  • Unknown stays agent_error. Rate limits (429) and bad requests (prompt too long) keep the old type. This PR only reclassifies the two causes the endpoint states outright.
  • Only claude-code for now. The gemini and codex servers have the same failed("cli_error", ...) default. Their CLIs' output for a budget or auth refusal was not captured, so they are left for later rather than guessed.

How to verify

The test events are trimmed from captures of Claude Code CLI 2.1.325 run against a local fake endpoint that answers with each error body.

uv run --extra dev pytest tst/unit/agents/claude_code -q

Result: 286 passed. Without the bridge change, test_model_key_errors.py fails for the four key cases (litellm-budget, litellm-auth, anthropic-auth, anthropic-permission) with error_type == "agent_error".

Ubuntu and others added 2 commits November 16, 2026 17:58
…y_budget or model_key_auth

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…del_key_auth, not agent_error

The bridge reads the CLI's api_error_type and api_error_status from its terminal result
event, and the A2A server passes the type to builder.failed, so agent-env no longer
blames the agent for an out-of-budget or rejected key.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant