APM: add utilities to define and validate JSON schema. add the first schema - OpenAI chat completion schema

[penguingao]

Commit Message:
add utilities to define and validate JSON schema. add the first schema - OpenAI chat completion schema

This allows us to structurally define a JSON payload schema for AI requests. It is useful for validation, and
later transcoding.

This is part of #44681.

I used AI to generate this PR. I have read and fully understand this PR.

Additional Description: https://github.com/penguingao/thoughts/blob/main/20260727_AI_FILTER.md
Risk Level: Medium
Testing: unit tests
Docs Changes: updated the extension docs, but this is still an alpha filter under development
Release Notes: N/A - WIP alpha filter
Platform Specific Features: no

[repokitteh-read-only]

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #46645 was opened by penguingao.

see: more, trace.

[penguingao]

/cc @hillsp

[penguingao]

/cc @wbpcode

[penguingao]

/assign @botengyao @tyxia @wbpcode

[botengyao]

here is one pass, thanks!

/wait

[botengyao]

this looks to me the parser offloading and schema offloading are inconsistent

this can reject valid payloads—for example, a large image_url.url containing base64 data will be offloaded by the parser but rejected by the schema because that field is not marked offloadable?

conversely, large strings in unknown fields can be offloaded and accepted, but their paths are not included in requestOffloadableFieldPaths() / requestStreamableFieldOrder().

ideally, the parser offloading and schema streamability need a single source of truth; otherwise normalization/stream processing can lose track of externalized fields.

[penguingao]

Good point. Made image url offloadable.

As for offload source of truth. As things stand, the schema is the source of truth. If a field that's not offloadable according to the schema, and the parser offloads it, it's an intentional error condition - we don't want to permit requests with some fields like model name to be too big as they will for the most time be consumed by filters directly.

I have a todo to run validation on the fly when parser runs. In that world, if a field requires offload, but the schema doesn't like it, we'll reject the request early.