Skip to content

feat: declare the Codex grammar through pi's constrainedSampling API - #43

Merged
code-yeongyu merged 2 commits into
code-yeongyu:mainfrom
zidou-kiyn:feat/constrained-sampling-grammar
Oct 3, 2026
Merged

code-yeongyu merged 2 commits into
code-yeongyu:mainfrom
zidou-kiyn:feat/constrained-sampling-grammar

Conversation

@zidou-kiyn

@zidou-kiyn zidou-kiyn commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Problem

createApplyPatchTool() attaches the Codex Lark grammar as a freeform property, but pi never reads that property. The provider request therefore always declares apply_patch as a plain JSON function tool, even for GPT models on providers that support OpenAI grammar tools.

Observed with pi 0.99.1, gpt-6.1-sol, and a Responses provider with compat.supportsOpenAIGrammarTools: true: the payload in before_provider_request contains { "type": "function", "name": "apply_patch" }.

Fix

Pi's public ToolDefinition.constrainedSampling hook accepts grammar variants (it is in the pinned 0.87.1 devDependency). This PR declares the existing grammar there:

constrainedSampling: { type: "grammar", variants: { openai_lark: APPLY_PATCH_LARK_GRAMMAR } },
  • When the model's provider sets compat.supportsOpenAIGrammarTools (pi's bundled openai and openai-codex catalogs set it for GPT models), pi sends apply_patch as a native custom tool with format: { type: "grammar", syntax: "lark" }. pi maps the raw custom-tool input back to the single input parameter, so execute is unchanged.
  • Everywhere else, pi falls back to the current function tool, so behavior there is unchanged.
  • The freeform property is kept for existing consumers. The grammar, description, and schema are byte-for-byte unchanged.

Verification

  • bun run check and bun run test pass (61 tests). The registration test now also asserts constrainedSampling.
  • End to end with pi -p -ne -e ./src/index.ts on pi 0.99.1 + gpt-6.1-sol: all three requests declared custom:apply_patch (grammar), both patches (add file, then update line) applied, and every response was 200.

Note, not changed here

pi 0.99 prints on every start: Host-provided extension packages must be declared in peerDependencies with a "*" range, not dependencies: typebox. #40 moved typebox into dependencies on purpose for hosts that install with peer resolution disabled, so this PR leaves it alone. Flagging it in case you want to revisit that trade-off.

Review in cubic

zidou-kiyn added a commit to zidou-kiyn/pi-preset that referenced this pull request Sep 30, 2026
…r status

Explain why the preset ships no mcp.json or defaultTools now that MCP,
codemode, and tool_search are built in, and why pi-apply-patch stays
required: pi has no built-in apply_patch, and its grammar only reaches
the model once code-yeongyu/pi-apply-patch#43 lands.
@code-yeongyu

Copy link
Copy Markdown
Owner

Thanks @zidou-kiyn, this is correct. I verified it against pi's own resolver at the pinned 0.87.1:

  • With grammar support: resolveGrammarConstrainedSampling(createApplyPatchTool(), true) returns { format: "lark", inputProperty: "input" } with exactly APPLY_PATCH_LARK_GRAMMAR. The tool's schema meets pi's grammar requirement (one required string property).
  • Without grammar support: it returns undefined, so providers without OpenAI grammar tools keep the plain function tool.
  • On main: the first case fails, because the grammar is never declared.

I pushed one maintainer commit (79af773):

  • It replaces the test assertion that pinned the constrainedSampling object with two behavioral tests through pi's public @earendil-works/pi-ai/api/constrained-sampling resolver, one per provider kind above.
  • It adds a CHANGELOG entry crediting you.

I'll merge once CI is green on the new head.

zidou-kiyn and others added 2 commits October 3, 2026 21:34
The freeform property on the tool definition is not read by pi, so
apply_patch always went out as a plain JSON function tool, even on
providers that support OpenAI grammar tools.

Pi's public ToolDefinition.constrainedSampling hook takes grammar
variants. Declaring the Codex Lark grammar there sends apply_patch as a
native OpenAI custom tool whenever the model's provider sets
compat.supportsOpenAIGrammarTools (pi's bundled OpenAI and OpenAI Codex
catalogs do), and falls back to the function tool everywhere else. The
freeform property is kept for existing consumers.
Replace the assertion that pinned the constrainedSampling object with tests
that run pi's public resolver (@earendil-works/pi-ai/api/constrained-sampling):
a provider with OpenAI grammar tools gets the Codex Lark grammar bound to the
'input' parameter, and any other provider keeps the plain function tool. Add
the CHANGELOG entry.
@code-yeongyu
code-yeongyu force-pushed the feat/constrained-sampling-grammar branch from 79af773 to 8f70b80 Compare October 3, 2026 12:35
@code-yeongyu
code-yeongyu merged commit f327003 into code-yeongyu:main Oct 3, 2026
7 checks passed
@code-yeongyu

Copy link
Copy Markdown
Owner

Merged as f327003, with you credited as co-author and in the CHANGELOG. Thanks @zidou-kiyn: plain pi now sends apply_patch as a real grammar tool on providers that support it.

@code-yeongyu code-yeongyu mentioned this pull request Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants