Summary
policyengine_api/services/ai_analysis_service.py pins AI_ANALYSIS_MODEL = "claude-sonnet-4-20250514". That model is retired: on 2026-08-22 the Anthropic Models API returns 404 not_found_error for both claude-sonnet-4-20250514 and the claude-sonnet-4-0 alias (probed with client.models.retrieve, org key; request ids req_011CeJGHqFJWpbXNFgQxBJkG, req_011CeJGHrJZJkUEm9sFePUmW). The 0.97.0 SDK's deprecation table already listed it with an end-of-life date of June 15, 2026.
Impact
Every AI-analysis request that misses the runtime cache (runtime_cache/ai_analyses.py, 7-day TTL) goes through claude_client.messages.stream(model=AI_ANALYSIS_MODEL, ...) and should fail with the same 404. That covers the tracer explanations (tracer_analysis_service.py) and simulation analyses (simulation_analysis_service.py). No monitor covers this path, and the unit suites mock the Claude client, so nothing has flagged it. Inferred from the code path plus the Models API response — please confirm the failure rate from Cloud Run logs ("recompute-failed" cache events / not_found_error in policyengine-api request logs) before treating the exact blast radius as established.
Fix needed
Choose a current model and update AI_ANALYSIS_MODEL (and the snapshot tests that embed the model name, if any). This is a product/cost decision for the maintainers; the anthropic SDK pin landing in #3799 is deliberately separate from it. When the SDK is later moved to 1.x, temperature=0.0 at ai_analysis_service.py:71 must also go (1.x removed the sampling keywords), which is the right moment to do both together.
Summary
policyengine_api/services/ai_analysis_service.pypinsAI_ANALYSIS_MODEL = "claude-sonnet-4-20250514". That model is retired: on 2026-08-22 the Anthropic Models API returns404 not_found_errorfor bothclaude-sonnet-4-20250514and theclaude-sonnet-4-0alias (probed withclient.models.retrieve, org key; request idsreq_011CeJGHqFJWpbXNFgQxBJkG,req_011CeJGHrJZJkUEm9sFePUmW). The 0.97.0 SDK's deprecation table already listed it with an end-of-life date of June 15, 2026.Impact
Every AI-analysis request that misses the runtime cache (
runtime_cache/ai_analyses.py, 7-day TTL) goes throughclaude_client.messages.stream(model=AI_ANALYSIS_MODEL, ...)and should fail with the same 404. That covers the tracer explanations (tracer_analysis_service.py) and simulation analyses (simulation_analysis_service.py). No monitor covers this path, and the unit suites mock the Claude client, so nothing has flagged it. Inferred from the code path plus the Models API response — please confirm the failure rate from Cloud Run logs ("recompute-failed"cache events /not_found_errorinpolicyengine-apirequest logs) before treating the exact blast radius as established.Fix needed
Choose a current model and update
AI_ANALYSIS_MODEL(and the snapshot tests that embed the model name, if any). This is a product/cost decision for the maintainers; the anthropic SDK pin landing in #3799 is deliberately separate from it. When the SDK is later moved to 1.x,temperature=0.0atai_analysis_service.py:71must also go (1.x removed the sampling keywords), which is the right moment to do both together.