Skip to content

[Code Quality] Distinguish pre-agent vs long-run 0-turn copilot CLI failures for smarter retry #46506

Description

@github-actions

Description

The Agentic Workflow Audit repeatedly observes two distinct classes of copilot Execute-CLI 0-turn failures that are currently treated identically:

  1. Pre-agent startup failures — agent dies before doing any work (0 tokens, negligible cost)
  2. Long-run then exit failures — agent completes substantial work (30+ minutes, 119–210 AIC) but the CLI wrapper exits non-zero, discarding all completed output

Class 2 wastes significant resources with no recovery path.

From the Agentic Workflow Audit — 2026-07-18 #46492:

"3 of today's fails lost 30–31 min of real agent work + 119–210 aic to a post-work CLI exit. Distinguishing pre-agent startup fails (0-tok) from longrun-then-exit fails, and retrying only the latter, would recover completed work."

Suggested Changes

  • In the copilot engine driver (likely pkg/workflow/engines/copilot/ or similar), add token-usage detection to classify failures:
    • If TokenUsage == 0 at exit: classify as startup failure (no retry needed)
    • If TokenUsage > threshold at non-zero exit: classify as long-run failure (safe-outputs may still be recoverable)
  • Log the distinction in the failure summary for audit visibility.
  • Optionally: on long-run exit, attempt to read any partial safe-output declarations before discarding.

Files Affected

  • pkg/workflow/engines/copilot/ — CLI driver exit handling
  • pkg/workflow/ — run lifecycle / failure classification

Success Criteria

  • Long-run failures (>10K tokens) are distinguishable from startup failures in run logs.
  • Audit reports can report accurate wasted-AIC by failure class.
  • Existing tests pass; no behavior change for pre-agent startup failures.

Source

Extracted from Agentic Workflow Audit — 2026-07-18 #46492

Priority

High — recovers 119–210 AIC per occurrence on long-run failures; directly impacts fleet cost efficiency.

🔍 Task mining by Discussion Task Miner - Code Quality Improvement Agent · 308.6 AIC · ⌖ 5.31 AIC · ⊞ 7K ·

  • expires on Jul 19, 2026, 5:31 PM UTC-08:00

Metadata

Metadata

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions