Skip to content

Production release inputs: metrics guard image and uncensored model name - #1

Merged
AnthonyRonning merged 2 commits into
mainfrom
claude/production-release
Nov 19, 2026
Merged

AnthonyRonning merged 2 commits into
mainfrom
claude/production-release

Conversation

@AnthonyRonning

Copy link
Copy Markdown
Contributor

What changes

First release meant to serve customers (v0.2.0). Engine settings stay as in v0.1.7: 120 vCPUs, three DSpark draft tokens.

  • Engine metrics behind the metrics key. The serving image is now built here: Tinfoil's exact Kimi image plus one file, maple_metrics_auth.py, loaded with --middleware. It requires the METRICS_API_KEY bearer on every path under /metrics and passes every other request through untouched. vLLM's own bearer check does not cover /metrics, and the shim's only gate for it is Tinfoil's online admin-key validator.
  • Served model name kimi-k3-1-pro-uncensored in place of kimi-k3-1-pro.
  • /v1/messages and /v1/messages/count_tokens closed again. They were opened in v0.1.8 for a benchmark run.
  • Release workflow builds and pushes the image, pins its digest in the configuration on a release-only commit, tags that commit and dispatches measurement. Same shape as inference-deepseek-uncensored.
  • EXPERIMENTS.md becomes CHANGES.md and lists every difference from stock.

Why the guard is safe to add

  • Pure ASGI: it hands vLLM the same scope, receive and send objects, so streaming and client-disconnect handling are unchanged. A test asserts object identity.
  • Standard library only. No new package in the image.
  • Fails closed: with no key in the environment every /metrics request is refused.
  • Covers every spelling vLLM would serve. vLLM routes ^/metrics(.*)$ to Prometheus, so the guard folds repeated slashes and case and matches the prefix.
  • Constant-time comparison of SHA-256 digests; the key is not kept in the clear.
  • The build fails if the base image's vLLM version, its --middleware hook, its /metrics mount or its guarded-prefix list differ from what the guard was written against.

Checks

  • python3 -m unittest discover -s tests (24 tests): the guard, and that the Dockerfile, configuration and guard agree.
  • python3 render_config.py --check: the template differs from stock v0.0.3 only in the listed entries; a configuration pinned by Tinfoil's update_config.py passes with --require-pinned-image, a tampered one fails.
  • CI builds the image without publishing, which runs the installer against the real base image.

Not verified yet

  • That METRICS_API_KEY reaches the vLLM process environment. It is declared as a container secret beside VLLM_API_KEY, which does. Qualification on the node will show it: no key and wrong key refused, metrics key accepted.
  • That the new ghcr package is publicly pullable after its first push.

🤖 Generated with Claude Code

AnthonyRonning and others added 2 commits November 18, 2026 17:59
Build the serving image here: Tinfoil's Kimi image plus one file that
requires the metrics key on every path under vLLM's /metrics. vLLM's own
bearer check does not cover that path, and the shim's only gate for it is
Tinfoil's online admin-key validator.

- maple_metrics_auth.py: pure ASGI middleware, standard library only,
  loaded with --middleware. Refuses when no key is configured.
- scripts/install_metrics_auth.py: build-time installer that fails if the
  base image's vLLM is not the build the guard was written against.
- Release workflow builds the image, pins its digest on a release-only
  commit and tags it; the renderer checks the pinned configuration.
- Serve the model as kimi-k3-1-pro-uncensored.
- Close /v1/messages and /v1/messages/count_tokens again.
- EXPERIMENTS.md becomes CHANGES.md and lists every difference from stock.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The first build copied the guard correctly but the check imported the
source copy beside the installer. Import it in a separate isolated
interpreter instead, the way the server will find it.

Co-Authored-By: Claude <noreply@anthropic.com>
@AnthonyRonning
AnthonyRonning marked this pull request as ready for review November 19, 2026 02:16
@AnthonyRonning
AnthonyRonning merged commit 0fa8938 into main Nov 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant