Skip to content

Support "auto" model for Pi and Codex via provider-prefix stripping - #8500

Merged
lpcox merged 7 commits into
mainfrom
copilot/github-8499-add-auto-support
Sep 12, 2026
Merged

lpcox merged 7 commits into
mainfrom
copilot/github-8499-add-auto-support

Conversation

Copilot AI commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Pi, Codex, and similar harnesses configure models using LiteLLM-style "provider/model" naming (e.g. copilot/auto) even when already talking to that exact provider. The api-proxy sidecar forwarded this literal string upstream unmodified, so Copilot rejected copilot/auto as an unrecognized model instead of resolving it via its existing auto selector.

Root cause

  • The existing auto pass-through only fired for the bare string "auto", and was only reachable when AWF_MODEL_ALIASES was configured (makeModelBodyTransform() returns null otherwise) — so it was effectively dead code for the common case of a user not configuring aliases.
  • No normalization existed anywhere for a redundant "<provider>/" self-reference prefix in the request body's model field.

Fix

  • model-utils.js: added stripRedundantProviderPrefix(model, provider) — strips a "<provider>/" prefix only when it case-insensitively matches the current provider, leaving cross-provider references (e.g. openai/gpt-4 while talking to Copilot) untouched.
  • model-resolver.js: resolveModel() now normalizes the requested model with this helper before alias/direct-match resolution, so copilot/auto hits the existing auto pass-through and copilot/gpt-4o resolves as a direct match.
  • model-body-rewriter.js / body-handler.js: added stripRedundantModelPrefixInBody(), wired in as an always-on step in the request pipeline — independent of AWF_MODEL_ALIASES — so the prefix is stripped even when no aliasing is configured at all.
// Before: forwarded as-is, rejected by Copilot
{ "model": "copilot/auto" }

// After: normalized before reaching upstream
{ "model": "auto" }

Docs

  • docs/api-proxy-sidecar.md: clarified that Copilot's auto (and copilot/auto) is supported through the Copilot provider path for Codex/Pi, distinct from the pre-existing OpenAI-native auto limitation.

Copilot AI linked an issue Sep 12, 2026 that may be closed by this pull request
Copilot AI and others added 4 commits September 12, 2026 21:04
…Codex

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
…gging

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for auto in Pi and Codex Support "auto" model for Pi and Codex via provider-prefix stripping Sep 12, 2026
Copilot AI requested a review from lpcox September 12, 2026 21:11
@lpcox
lpcox marked this pull request as ready for review September 12, 2026 22:09
Copilot AI balanced review requested due to automatic review settings September 12, 2026 22:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Prefix stripping currently affects non-Copilot adapters and can corrupt valid namespaced model IDs for custom upstreams.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds provider-prefix normalization so Pi and Codex can use copilot/auto through the API proxy.

Changes:

  • Adds shared prefix-stripping logic and request-body integration.
  • Updates model resolution and tests.
  • Documents Copilot auto support.
File summaries
File Description
docs/api-proxy-sidecar.md Documents Copilot auto behavior.
containers/api-proxy/model-utils.js Adds prefix normalization helper.
containers/api-proxy/model-utils.test.js Tests normalization behavior.
containers/api-proxy/model-resolver.js Normalizes models before resolution.
containers/api-proxy/model-resolver.test.js Tests prefixed model resolution.
containers/api-proxy/model-body-rewriter.js Adds body-level normalization.
containers/api-proxy/model-body-rewriter-prefix.test.js Tests request-body rewriting.
containers/api-proxy/body-handler.js Applies normalization to writable requests.
Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 2
  • Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread containers/api-proxy/body-handler.js Outdated
Comment thread containers/api-proxy/model-resolver.js Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

  1. Download the docs-preview-pr-8500 artifact from the workflow run
  2. Unzip and open index.html in your browser

Built from commit 51fab88

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Warning

Firewall blocked 7 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • msfeed25.pkgs.visualstudio.com
  • www.google.com
  • www.gstatic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "msfeed25.pkgs.visualstudio.com"
    - "www.google.com"
    - "www.gstatic.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Build Test Suite completed successfully!

Generated by Build Test Suite for #8500

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Smoke Claude passed

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by Smoke Claude for #8500

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

📰 DEVELOPING STORY: Smoke Docker Sbx reports failed. Our correspondents are investigating the incident...

📰 BREAKING: Report filed by Smoke Docker Sbx

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Security Guard completed successfully!

Security review of PR #8500 ("Support 'auto' model for Pi and Codex via provider-prefix stripping") is complete.

Finding: No security vulnerabilities detected. The PR implements safe, transparent model-name normalization:

  • Strips redundant provider prefixes only when they match the current provider (case-insensitive exact match)
  • Guards against null/empty inputs
  • Does not modify authorization, egress filtering, or container security controls
  • Maintains consistent normalization across two call sites with documented sync requirements

This is a feature-enablement PR with no security regression.

Generated by Security Guard for #8500

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Smoke Gemini reports failed. Facets need polishing...

💎 Faceted by Smoke Gemini

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Status
API ✅ PASS
gh CLI ✅ PASS
File ✅ PASS

Overall result: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by Smoke Claude for #8500 · claude · haiku45 · 55.2 AIC · ⊞ 4.5K ·
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot Engine@copilot @lpcox

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (api.github.com) reachable: allowed=200
✅ Blocked domain (example.com) denied: 403 CONNECT tunnel failed

Overall: PASS

cc @lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Sep 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

✅ GitHub MCP: PR list OK (2 merged PRs fetched)
✅ HTTP connectivity: 200 (github.com)
✅ File write/read: Created and verified
✅ BYOK inference: Working (direct mode via api-proxy)

Status: PASS — Direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com

Triggered by @lpcox

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 94.09% 94.10% ➡️ +0.01%
Statements 92.76% 92.77% ➡️ +0.01%
Functions 93.48% 93.48% ➡️ +0.00%
Branches 86.63% 86.64% 📈 +0.01%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/log-directory-setup.ts 96.2% → 100.0% (+3.78%) 96.3% → 100.0% (+3.71%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@lpcox
lpcox deployed to aoai-model September 12, 2026 22:25 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Services Connectivity

  • Redis PING: ✅ PONG
  • PostgreSQL pg_isready: ✅ accepting connections
  • PostgreSQL SELECT 1: ✅ 1

Overall: PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

📡 OTel Tracing Smoke Test Results

# Scenario Result
1 Module Loading otel.js loaded; isEnabled()true; exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled, _provider, _ProxyAwareOtlpExporter, _FileSpanExporter, _FanOutSpanExporter, _parseEndpoints, _parseOtlpHeaders, _buildResourceSpans, _createOtlpWorkloadIdentity
2 Test Suite ✅ 68/68 tests passed across 3 suites (otel.test.js, and split-module tests)
3 Env Var Forwarding env-passthrough.ts forwards GITHUB_AW_OTEL_TRACE_ID/GITHUB_AW_OTEL_PARENT_SPAN_ID to the agent; api-proxy-env-config.ts forwards GH_AW_OTLP_ENDPOINTS, OTEL_EXPORTER_OTLP_ENDPOINT, and trace context to api-proxy
4 Token Tracker Integration token-tracker-http.js implements the onUsage callback (OTEL hook point), invoked after usage normalization
5 OTEL Diagnostics ✅ Pipeline active: token-usage.jsonl has 9 real request records from this run (copilot/claude-sonnet-5). No local otel.jsonl file under the api-proxy logs dir — expected, since this run's OTLP endpoint is configured to export to Sentry directly rather than the file-based fallback exporter

Overall: All scenarios pass. No regression detected.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison Results

Runtime Host Version Chroot Version Match?
Python Python 3.12.14 Python 3.12.14 ✅ YES
Node.js v24.20.0 v2.98.0 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

⚠️ Not all tests passed — Node.js version mismatch between host and chroot environments. The smoke-chroot label was not added.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke test: FAIL

  • chore(workflows): upgrade gh-aw to v0.89.8
  • chore: update model-to-API mapping (2026-09-12)
  • PR reads: ✅
  • Playwright title check: ✅
  • File write/read: ✅
  • Discussion comment: ❌ permission unavailable
  • Build (npm ci && npm run build): ✅

Warning

Firewall blocked 7 domains

The following domains were blocked by the firewall during workflow execution:

  • accounts.google.com
  • android.clients.google.com
  • clients2.google.com
  • contentautofill.googleapis.com
  • msfeed25.pkgs.visualstudio.com
  • www.google.com
  • www.gstatic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "contentautofill.googleapis.com"
    - "msfeed25.pkgs.visualstudio.com"
    - "www.google.com"
    - "www.gstatic.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color pass ✅ PASS
Go env pass ✅ PASS
Go uuid pass ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx pass ✅ PASS
Node.js execa pass ✅ PASS
Node.js p-limit pass ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — PASS

All 18 test projects across Bun, C++, Deno, .NET, Go, Java, Node.js, and Rust built/installed successfully and all tests passed. No errors encountered.

Note: Java builds required -Dmaven.repo.local=/tmp/gh-aw/agent/m2repo since ~/.m2 was root-owned and not writable by the runner user in this sandbox; the Maven proxy settings for squid-proxy:3128 worked as configured.

Generated by Build Test Suite for #8500 · copilot · auto · 48.1 AIC · ⊞ 10.6K ·
Add label ready-for-aw to run again

@lpcox
lpcox merged commit 0603bd1 into main Sep 12, 2026
172 of 178 checks passed
@lpcox
lpcox deleted the copilot/github-8499-add-auto-support branch September 12, 2026 22:46
github-actions Bot added a commit that referenced this pull request Sep 13, 2026
…cation-architecture

Codex's OpenAI-native auto limitation was documented, but the
authentication-architecture guide did not note that Copilot's own auto
selector (including the copilot/auto LiteLLM-style alias used by Codex
and Pi) is unaffected and supported via api-proxy prefix stripping
(stripRedundantProviderPrefix, shipped in #8500). This was already
documented in docs/api-proxy-sidecar.md but missing here, causing a
cross-document inconsistency.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
lpcox pushed a commit that referenced this pull request Sep 14, 2026
…cation-architecture (#8505)

Codex's OpenAI-native auto limitation was documented, but the
authentication-architecture guide did not note that Copilot's own auto
selector (including the copilot/auto LiteLLM-style alias used by Codex
and Pi) is unaffected and supported via api-proxy prefix stripping
(stripRedundantProviderPrefix, shipped in #8500). This was already
documented in docs/api-proxy-sidecar.md but missing here, causing a
cross-document inconsistency.

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pi and codex should support "auto"

3 participants