docs(examples): add Laguna-S-2.1 on Nebius H100 recipe - #374
Open
ytsarev wants to merge 1 commit into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new end-to-end documentation example for serving Poolside’s Laguna-S-2.1 FP8 on Nebius (single 8×H100 node) using Modelplane, including the full set of manifests and a corresponding recipe page.
Changes:
- Extend Vale vocabulary to accept “NVLink” and “Laguna*” terms used in the new docs.
- Add a complete “laguna” example manifest set (InferenceClass, InferenceCluster, ModelCache, ModelDeployment, ModelService).
- Add a new docs recipe page that embeds the new manifests in the Examples section.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| docs/utils/vale/styles/config/vocabularies/Modelplane/accept.txt | Adds new accepted terms used by the Laguna docs (NVLink, Laguna*). |
| docs/manifests/examples/laguna/model-service.yaml | New ModelService manifest for exposing the Laguna deployment via one OpenAI-compatible endpoint. |
| docs/manifests/examples/laguna/model-deployment.yaml | New ModelDeployment manifest configuring vLLM FP8 serving (TP=8) on a single 8×H100 node. |
| docs/manifests/examples/laguna/model-cache.yaml | New ModelCache manifest to stage Laguna weights from Hugging Face onto shared RWX storage. |
| docs/manifests/examples/laguna/inference-cluster-nebius.yaml | New Nebius-backed InferenceCluster manifest for a single 8×H100 node pool. |
| docs/manifests/examples/laguna/inference-class-nebius.yaml | New Nebius InferenceClass describing the 8×H100 SXM node shape and DRA device properties. |
| docs/content/examples/laguna.md | New recipe page wiring the manifests into the docs site. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ytsarev
marked this pull request as draft
July 30, 2026 15:42
Serve Poolside Laguna-S-2.1 (118B MoE) FP8 on a single Nebius 8x H100 node
(gpu-h100-sxm, eu-north) as one Standalone engine, TP=8. Two engine options:
vLLM (model-deployment.yaml, vllm/vllm-openai:v0.25.1) and SGLang
(model-deployment-sglang.yaml, lmsysorg/sglang:v0.5.12.post1-cu129), which has
native Laguna support and returns the reasoning block in a separate field. Adds
the InferenceClass, InferenceCluster, ModelCache, ModelService, both
ModelDeployments, and the recipe page; mirrors the kimi-k2 and glm-4.5-air
examples. poolside/Laguna-S-2.1-FP8 is a public OpenMDW-1.1 repo, so no Hugging
Face token is needed.
### Tested - vLLM (Modelplane v0.2.0, vLLM 0.25.1, TP=8)
# Serving /v1/chat/completions
-> content: "...</think>def reverse_string(s):\n return s[::-1]"
finish_reason: stop system_fingerprint: vllm-0.25.1-tp8
# Tool calling (tools + tool_choice:auto), then full tool-use loop
-> tool_calls: [{"function":{"name":"get_weather","arguments":"{\"city\": \"Paris\"}"}}] finish_reason: tool_calls
-> (loop) content: "The current weather in Paris is 18C with light rain." finish_reason: stop
### Tested - SGLang (SGLang v0.5.12.post1, build with Anthropic endpoint picker modelplaneai#360, TP=8)
# 1. Anthropic Messages API /v1/messages
curl "$ADDR/v1/messages" -H 'anthropic-version: 2023-06-01' \
-d '{"model":"laguna","max_tokens":512,"messages":[{"role":"user","content":"Hello!"}]}'
-> HTTP 200 {"type":"message","role":"assistant","content":[{"type":"text","text":"..."}],"stop_reason":"end_turn"}
# 2. Reasoning /v1/chat/completions with chat_template_kwargs.enable_thinking=true
-> content: "17 * 23 = 391" reasoning_content: "Okay, let me try to figure out..." (940 reasoning tokens)
# 3. Tool calling (tools + tool_choice:auto)
-> tool_calls: [{"function":{"name":"get_weather","arguments":"{\"city\": \"Paris\"}"}}] finish_reason: tool_calls
Signed-off-by: Yury Tsarev <yury@upbound.io>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (2)
docs/manifests/examples/laguna/model-service.yaml:5
- The sample curl command omits
chat_template_kwargs.enable_thinking. Per the PR’s own note, omitting it can cause a leading</think>to leak intocontent, so the copy/paste example will produce degraded output. Update the example to explicitly setenable_thinking:false(or document the difference).
# then call it, e.g.:
# curl "$ADDR/v1/chat/completions" -H 'Content-Type: application/json' \
# -d '{"model":"laguna","messages":[{"role":"user","content":"hello"}]}'
docs/content/examples/laguna.md:18
- The recipe page doesn’t mention the
enable_thinkinggotcha described in the PR (omitting it can leak a leading</think>intocontent). Adding a short note here will prevent users from thinking the deployment is broken when they do a first test call.
This recipe was run end to end on Nebius (`eu-north`): serving and tool calling
validated on a single 8x H100 node. `poolside/Laguna-S-2.1-FP8` is a public
repository, so no Hugging Face token or Secret is needed. Apply the platform
side first, then the ML side.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Adds a Modelplane example for Poolside's Laguna models: Laguna-S-2.1 served FP8
as a single
Standaloneengine on one Nebius 8x H100 node (gpu-h100-sxm,eu-north). Two engine options: vLLM (model-deployment.yaml) and SGLang(
model-deployment-sglang.yaml), which has native Laguna support and returns thereasoning block in a separate
reasoning_contentfield. Mirrors the kimi-k2 andglm-4.5-air recipes: InferenceClass, InferenceCluster, ModelCache, both
ModelDeployments, ModelService, plus the recipe page.
Validated end to end on Nebius (TP=8): serving, tool calling, and — on a build
with the Anthropic endpoint picker (#360) — the Anthropic Messages API. Evidence
below.
Reasoning and
enable_thinkingBehavior is identical across both engines (vLLM, SGLang) and both endpoints
(
/v1/chat/completionsand/v1/messages) — always sendenable_thinkingexplicitly:
enable_thinkingcontent</think>leaks intocontentfalsetruereasoning(vLLM) /reasoning_content(SGLang); Anthropic: a nativethinkingcontent blockTool calling is unaffected in all cases. The
</think>leak is a Lagunachat-template artifact — the template opens a think block on the assistant turn,
and the
poolside_v1reasoning parser is passthrough unless thinking is set — soit is not an engine bug and switching engines does not change it. The fix is
to send
enable_thinkingexplicitly (or set a server-side default on the engine).Notes:
poolside_v1reasoning parser failing to auto-initialize on v0.25.1 and falling back to a
passthrough parser. This deployment extracted reasoning fine with per-request
enable_thinking=true; flagged since it targets the pinned version./v1/messages): rejected on v0.2.0 (the endpointpicker rejects the request shape); works on builds that include feat(docs): Serve the Anthropic Messages API (Claude Code) end to end #360 (EPP v0.9.0
Anthropic parser) — not yet in a tagged release. Confirmed live on a feat(docs): Serve the Anthropic Messages API (Claude Code) end to end #360 build
(HTTP 200), see below.
E2E Tested
vLLM — Nebius 8xH100 (gpu-h100-sxm, eu-north), Modelplane v0.2.0, vLLM 0.25.1, TP=8.
SGLang — same node, SGLang v0.5.12.post1-cu129, on a build with the Anthropic endpoint picker (#360), TP=8.
I have:
nix flake check(or./nix.sh flake check) and made sure it passes.Added or updated tests covering any composition function changes.Docs and manifests only; no composition function changes.git commit -s.