Skip to content

feat(orchestrator): cap parallel PR reviewers at 3; document Fable on Bedrock (v3.4.0)#62

Draft
sfreudenthaler wants to merge 4 commits into
mainfrom
worktree-ai-workflows+v4-fable-parallel-review
Draft

feat(orchestrator): cap parallel PR reviewers at 3; document Fable on Bedrock (v3.4.0)#62
sfreudenthaler wants to merge 4 commits into
mainfrom
worktree-ai-workflows+v4-fable-parallel-review

Conversation

@sfreudenthaler

@sfreudenthaler sfreudenthaler commented Jul 2, 2026

Copy link
Copy Markdown
Member

Summary

  • Fable support: no routing code change needed. Bedrock inference-profile IDs for any Claude model follow <region>.anthropic.claude-<name>-<version> (e.g. global.anthropic.claude-sonnet-4-6), and the orchestrator already matches on the anthropic. family prefix, not a per-model allowlist. Verified via research: Fable 5 is GA on Bedrock (launched 2026-06-09) with exactly the ID this predicts — global.anthropic.claude-fable-5 — so it already routes to claude-executor (anthropic-bedrock) today, same as Sonnet/Opus, with no ANTHROPIC_API_KEY needed. Docs updated to use Fable as the documented example, plus a note that it's region-limited to us-east-1/eu-north-1 at launch.
    • (An earlier revision of this PR added a bare claude-* → direct-Anthropic-API routing branch on the wrong assumption that Fable's model_id wouldn't be a Bedrock inference profile. Reverted — it would have required a new ANTHROPIC_API_KEY secret per consumer, working against the existing Bedrock-first architecture.)
  • Parallel review cap: model_id already accepted a comma-separated list (each model reviews the PR independently via strategy.matrix, shipped in feat(orchestrator): accept comma-separated model_id; each model reviews the PR #59/v3.3.2). This adds a hard cap of 3 parallel reviewers — a 4th model now fails the route job with a clear ::error:: instead of silently fanning out further.
  • New example: examples/multi-model-review-workflow.yml is a complete, runnable consumer workflow demonstrating the 3-reviewer fan-out (Fable 5 + Sonnet 4.6 over Bedrock, GPT-5.5 over bedrock-mantle), linked from both example lists and the Quick Examples section in README.md.
  • Docs updated (CLAUDE.md, ARCHITECTURE.md, README.md, examples/consumer-repo-workflow.yml) with the 3-model cap and mixed-provider parallel-review examples:
    model_id: "global.anthropic.claude-fable-5, global.anthropic.claude-sonnet-4-6, openai.gpt-5.5"

Versioning: this is additive — no renamed/removed inputs, no changed defaults, no changed output shape. The new validation only rejects an edge case (>3 models) that isn't part of any documented contract and has no known consumers today. Per this repo's own precedent (MAJOR reserved for architecture shifts like the v2.0.0 claude-code-action GA migration and the v3.0.0 multi-executor routing overhaul; smaller additive features like the codex executor and the comma-separated model_id feature itself landed as 3.x), this should ship as v3.4.0, not v4.0.0.

Test plan

  • python3 -c "import yaml; yaml.safe_load(...)" on all changed workflow files
  • docker run rhysd/actionlint:1.7.7 — clean
  • Ran the routing regex/cap logic standalone: global.anthropic.claude-fable-5 routes to anthropic-bedrock (no code change needed); a 3-model mixed list (Fable + Sonnet + GPT-5.5) fans out to the correct 2 executors; a 4th model is rejected; empty model_id still defaults to anthropic-api
  • Live end-to-end run against a real PR (needs a consumer repo pointed at this branch/tag)

…rallel reviewers at 3

Adds Fable (and any future plain-named Claude model, e.g. claude-fable-5) as a
selectable review model on the direct Anthropic API path — previously only
Bedrock inference-profile IDs (containing "anthropic.") routed to Claude, so a
bare model name fell through to the Bedrock-generic/harness executor and
would have failed there. Also enforces the existing comma-separated
model_id list at a hard max of 3 parallel reviewers, matching the intended
one-prompt/multiple-models review fan-out.

Docs (CLAUDE.md, ARCHITECTURE.md, examples/consumer-repo-workflow.yml)
updated to describe the new routing case, the 3-model cap, and a
multi-provider parallel-review example.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VD5n4VXNdJQa4xxMdbfGTB
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

🤖 Bedrock Review — qwen.qwen3-next-80b-a3b

New Issues

  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:70 — Assumption: re.match(r"^claude-", m) may match unintended strings like claude- followed by invalid characters (e.g., claude-@#$), but no validation enforces model name format. What to verify: Are only valid Claude model aliases (e.g., claude-fable-5) allowed by the Anthropic API? If not, this could cause runtime failures in claude-executor.yml that are silently swallowed.

Existing

  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:68 — Prior finding still present: sys.exit(1) on model count exceedance terminates the job but does not emit a GitHub Actions error annotation via ::error:: before exit — the ::error:: line is printed, but sys.exit(1) may not guarantee it’s captured by GitHub Actions runner in all contexts. What to verify: Does GitHub Actions reliably capture stderr output from print(..., file=sys.stderr) when followed by sys.exit(1)? If not, the error may be invisible to users.

Resolved

  • .github/workflows/claude-orchestrator.yml:64 — Max model limit of 3 is now enforced with explicit validation and error, resolving prior ambiguity.

Run: #28623550024 · tokens: in: 4937 · out: 415 · total: 5352

Fable (and any new Claude model) already routes correctly through the
existing anthropic-bedrock regex, since Bedrock inference profiles follow
the same "<region>.anthropic.claude-<name>-<version>" form as Sonnet/Opus
(e.g. global.anthropic.claude-fable-5) — the match is on the "anthropic."
family prefix, not a per-model allowlist. The prior commit added a new
bare-"claude-*" -> direct-Anthropic-API branch based on a wrong assumption
about the model_id string, which would have required ANTHROPIC_API_KEY
per consumer — working against this org's Bedrock-only, no-API-key setup.
Reverts that branch; keeps the 3-model parallel-reviewer cap. Docs updated
to use the correct Bedrock-form example ID throughout.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VD5n4VXNdJQa4xxMdbfGTB
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

🤖 Bedrock Review — qwen.qwen3-next-80b-a3b

New Issues

  • 🟠 High: .github/workflows/claude-orchestrator.yml:201sys.exit(1) after ::error:: may not reliably surface error to GitHub Actions runner if stderr is not captured (same as prior finding, now duplicated in new code path)
  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:198print(f"::error::..." uses print instead of sys.stderr.write — may not be reliably flushed or captured by GitHub Actions runner

Existing

  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:70 — re.match(r"^claude-", m) may match invalid model names without validation against Anthropic's allowed aliases

Resolved

  • .github/workflows/claude-orchestrator.yml:68sys.exit(1) after ::error:: is now explicitly triggered in new code, but the underlying capture risk remains unchanged — no fix applied

Run: #28626560453 · tokens: in: 4613 · out: 398 · total: 5011

Verified via research: Fable 5 GA'd on Bedrock 2026-06-09 with exactly the
guessed inference-profile ID (global.anthropic.claude-fable-5), confirming
the existing anthropic.* regex needs no change. Only gap was the region
constraint, which isn't obvious from the model_id alone.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VD5n4VXNdJQa4xxMdbfGTB
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

🤖 Bedrock Review — qwen.qwen3-next-80b-a3b

New Issues

  • 🟠 High: .github/workflows/claude-orchestrator.yml:201sys.exit(1) after ::error:: may not reliably surface error to GitHub Actions runner if stderr is not captured
  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:198print(f"::error::...") uses print instead of sys.stderr.write — may not be reliably flushed or captured by GitHub Actions runner

Existing

  • 🟠 High: .github/workflows/claude-orchestrator.yml:201 — sys.exit(1) after ::error:: may not reliably surface error to GitHub Actions runner if stderr is not captured
  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:198 — print(f"::error::...") uses print instead of sys.stderr.write — may not be reliably flushed or captured by GitHub Actions runner
  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:70 — re.match(r"^claude-", m) may match invalid model names without validation against Anthropic's allowed aliases

Resolved

  • .github/workflows/claude-orchestrator.yml:70 — regex pattern changed from ^claude- to ^([a-z]+\.)?anthropic\. — no longer matches invalid model names by substring

Run: #28626796266 · tokens: in: 4714 · out: 496 · total: 5210

@sfreudenthaler sfreudenthaler changed the title feat: Fable model routing + 3-way parallel PR review (v4.0.0) feat(orchestrator): cap parallel PR reviewers at 3; document Fable on Bedrock (v3.4.0) Jul 2, 2026
New examples/multi-model-review-workflow.yml demonstrates the comma-separated
model_id fan-out end to end: one PR, one prompt, three reviewers routed to
different executors (Claude Fable 5 + Sonnet 4.6 over Bedrock, GPT-5.5 over
bedrock-mantle), each posting its own sticky comment. Linked from both
example lists and the Quick Examples section in README.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VD5n4VXNdJQa4xxMdbfGTB
@github-actions

github-actions Bot commented Jul 3, 2026

Copy link
Copy Markdown

🤖 Bedrock Review — qwen.qwen3-next-80b-a3b

New Issues

  • 🟠 High: .github/workflows/claude-orchestrator.yml:201sys.exit(1) after print(f"::error::...") may not reliably surface error to GitHub Actions runner if stderr is not captured
  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:198print(f"::error::...") uses print instead of sys.stderr.write — may not be reliably flushed or captured by GitHub Actions runner
  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:70re.match(r"^([a-z]+\.)?anthropic\.", m) may match invalid model names without validation against Anthropic's allowed aliases

Existing

  • 🟠 High: .github/workflows/claude-orchestrator.yml:201 — sys.exit(1) after ::error:: may not reliably surface error to GitHub Actions runner if stderr is not captured
  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:198 — print(f"::error::...") uses print instead of sys.stderr.write — may not be reliably flushed or captured by GitHub Actions runner
  • 🟡 Medium: .github/workflows/claude-orchestrator.yml:70 — re.match(r"^([a-z]+.)?anthropic.", m) may match invalid model names without validation against Anthropic's allowed aliases

Resolved

  • .github/workflows/claude-orchestrator.yml:201 — added explicit max model limit check (previously unbounded)
  • .github/workflows/claude-orchestrator.yml:198 — added error message for exceeding model limit (previously silent overflow)
  • .github/workflows/claude-orchestrator.yml:70 — clarified intent to match family prefix, not allowlist (documentation now matches behavior)

Run: #28629352397 · tokens: in: 5807 · out: 623 · total: 6430

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant