Skip to content

docs(examples): add Laguna-S-2.1 on Nebius H100 recipe - #374

Open
ytsarev wants to merge 1 commit into
modelplaneai:mainfrom
ytsarev:laguna
Open

docs(examples): add Laguna-S-2.1 on Nebius H100 recipe#374
ytsarev wants to merge 1 commit into
modelplaneai:mainfrom
ytsarev:laguna

Conversation

@ytsarev

@ytsarev ytsarev commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Description of your changes

Adds a Modelplane example for Poolside's Laguna models: Laguna-S-2.1 served FP8
as a single Standalone engine on one Nebius 8x H100 node (gpu-h100-sxm,
eu-north). Two engine options: vLLM (model-deployment.yaml) and SGLang
(model-deployment-sglang.yaml), which has native Laguna support and returns the
reasoning block in a separate reasoning_content field. Mirrors the kimi-k2 and
glm-4.5-air recipes: InferenceClass, InferenceCluster, ModelCache, both
ModelDeployments, ModelService, plus the recipe page.

Validated end to end on Nebius (TP=8): serving, tool calling, and — on a build
with the Anthropic endpoint picker (#360) — the Anthropic Messages API. Evidence
below.

Reasoning and enable_thinking

Behavior is identical across both engines (vLLM, SGLang) and both endpoints
(/v1/chat/completions and /v1/messages) — always send enable_thinking
explicitly:

enable_thinking content reasoning
omitted a leading </think> leaks into content
false clean none
true clean extracted — OpenAI: reasoning (vLLM) / reasoning_content (SGLang); Anthropic: a native thinking content block

Tool calling is unaffected in all cases. The </think> leak is a Laguna
chat-template artifact — the template opens a think block on the assistant turn,
and the poolside_v1 reasoning parser is passthrough unless thinking is set — so
it is not an engine bug and switching engines does not change it. The fix is
to send enable_thinking explicitly (or set a server-side default on the engine).

Notes:

E2E Tested

vLLM — Nebius 8xH100 (gpu-h100-sxm, eu-north), Modelplane v0.2.0, vLLM 0.25.1, TP=8.

# 1. Serving - /v1/chat/completions
curl "$ADDR/v1/chat/completions" -H 'Content-Type: application/json' \
  -d '{"model":"laguna","messages":[{"role":"user","content":"Write a Python function that reverses a string. Output only the code."}]}'
-> content: "...</think>def reverse_string(s):\n    return s[::-1]"
   finish_reason: "stop"   system_fingerprint: "vllm-0.25.1-tp8"

# 2. Tool calling - tools + tool_choice:auto
curl "$ADDR/v1/chat/completions" -H 'Content-Type: application/json' \
  -d '{"model":"laguna","messages":[{"role":"user","content":"What is the weather in Paris? Use the tool."}],
       "tools":[{"type":"function","function":{"name":"get_weather","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],"tool_choice":"auto"}'
-> tool_calls: [{"function":{"name":"get_weather","arguments":"{\"city\": \"Paris\"}"}}]
   finish_reason: "tool_calls"

# 3. Full tool-use loop (feed the tool result back as role:tool)
-> content: "The current weather in Paris is 18C with light rain."   finish_reason: "stop"

# 4. Reasoning - /v1/chat/completions with chat_template_kwargs.enable_thinking=true
-> content: "To find the product of 17 and 23 ... 161 + 230 = 391"  (clean, no </think>)
   reasoning: "Okay, so I need to find the product of 17 and 23. Hmm, let me think..."

SGLang — same node, SGLang v0.5.12.post1-cu129, on a build with the Anthropic endpoint picker (#360), TP=8.

# 1. Anthropic Messages API - /v1/messages
curl "$ADDR/v1/messages" -H 'Content-Type: application/json' -H 'anthropic-version: 2023-06-01' \
  -d '{"model":"laguna","max_tokens":512,"messages":[{"role":"user","content":"Hello!"}]}'
-> HTTP 200  {"type":"message","role":"assistant","content":[{"type":"text","text":"..."}],"stop_reason":"end_turn"}

# 2. Reasoning - /v1/chat/completions with chat_template_kwargs.enable_thinking=true
curl "$ADDR/v1/chat/completions" -H 'Content-Type: application/json' \
  -d '{"model":"laguna","max_tokens":1024,"chat_template_kwargs":{"enable_thinking":true},"messages":[{"role":"user","content":"What is 17*23? Think step by step."}]}'
-> content: "17 * 23 = 391"   reasoning_content: "Okay, let me try to figure out..." (940 reasoning tokens)

# 3. Tool calling - tools + tool_choice:auto
-> tool_calls: [{"function":{"name":"get_weather","arguments":"{\"city\": \"Paris\"}"}}]
   finish_reason: "tool_calls"

I have:

  • Read and followed Modelplane's contribution process.
  • Run nix flake check (or ./nix.sh flake check) and made sure it passes.
  • Added or updated tests covering any composition function changes. Docs and manifests only; no composition function changes.
  • Signed off every commit with git commit -s.

@ytsarev
ytsarev marked this pull request as ready for review July 30, 2026 13:06
Copilot AI review requested due to automatic review settings July 30, 2026 13:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new end-to-end documentation example for serving Poolside’s Laguna-S-2.1 FP8 on Nebius (single 8×H100 node) using Modelplane, including the full set of manifests and a corresponding recipe page.

Changes:

  • Extend Vale vocabulary to accept “NVLink” and “Laguna*” terms used in the new docs.
  • Add a complete “laguna” example manifest set (InferenceClass, InferenceCluster, ModelCache, ModelDeployment, ModelService).
  • Add a new docs recipe page that embeds the new manifests in the Examples section.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
docs/utils/vale/styles/config/vocabularies/Modelplane/accept.txt Adds new accepted terms used by the Laguna docs (NVLink, Laguna*).
docs/manifests/examples/laguna/model-service.yaml New ModelService manifest for exposing the Laguna deployment via one OpenAI-compatible endpoint.
docs/manifests/examples/laguna/model-deployment.yaml New ModelDeployment manifest configuring vLLM FP8 serving (TP=8) on a single 8×H100 node.
docs/manifests/examples/laguna/model-cache.yaml New ModelCache manifest to stage Laguna weights from Hugging Face onto shared RWX storage.
docs/manifests/examples/laguna/inference-cluster-nebius.yaml New Nebius-backed InferenceCluster manifest for a single 8×H100 node pool.
docs/manifests/examples/laguna/inference-class-nebius.yaml New Nebius InferenceClass describing the 8×H100 SXM node shape and DRA device properties.
docs/content/examples/laguna.md New recipe page wiring the manifests into the docs site.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ytsarev
ytsarev marked this pull request as draft July 30, 2026 15:42
Serve Poolside Laguna-S-2.1 (118B MoE) FP8 on a single Nebius 8x H100 node
(gpu-h100-sxm, eu-north) as one Standalone engine, TP=8. Two engine options:
vLLM (model-deployment.yaml, vllm/vllm-openai:v0.25.1) and SGLang
(model-deployment-sglang.yaml, lmsysorg/sglang:v0.5.12.post1-cu129), which has
native Laguna support and returns the reasoning block in a separate field. Adds
the InferenceClass, InferenceCluster, ModelCache, ModelService, both
ModelDeployments, and the recipe page; mirrors the kimi-k2 and glm-4.5-air
examples. poolside/Laguna-S-2.1-FP8 is a public OpenMDW-1.1 repo, so no Hugging
Face token is needed.

### Tested - vLLM (Modelplane v0.2.0, vLLM 0.25.1, TP=8)

# Serving /v1/chat/completions
-> content: "...</think>def reverse_string(s):\n    return s[::-1]"
   finish_reason: stop   system_fingerprint: vllm-0.25.1-tp8
# Tool calling (tools + tool_choice:auto), then full tool-use loop
-> tool_calls: [{"function":{"name":"get_weather","arguments":"{\"city\": \"Paris\"}"}}]  finish_reason: tool_calls
-> (loop) content: "The current weather in Paris is 18C with light rain."  finish_reason: stop

### Tested - SGLang (SGLang v0.5.12.post1, build with Anthropic endpoint picker modelplaneai#360, TP=8)

# 1. Anthropic Messages API /v1/messages
curl "$ADDR/v1/messages" -H 'anthropic-version: 2023-06-01' \
  -d '{"model":"laguna","max_tokens":512,"messages":[{"role":"user","content":"Hello!"}]}'
-> HTTP 200  {"type":"message","role":"assistant","content":[{"type":"text","text":"..."}],"stop_reason":"end_turn"}
# 2. Reasoning /v1/chat/completions with chat_template_kwargs.enable_thinking=true
-> content: "17 * 23 = 391"   reasoning_content: "Okay, let me try to figure out..." (940 reasoning tokens)
# 3. Tool calling (tools + tool_choice:auto)
-> tool_calls: [{"function":{"name":"get_weather","arguments":"{\"city\": \"Paris\"}"}}]  finish_reason: tool_calls

Signed-off-by: Yury Tsarev <yury@upbound.io>
@ytsarev
ytsarev marked this pull request as ready for review July 30, 2026 23:52
Copilot AI review requested due to automatic review settings July 30, 2026 23:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

docs/manifests/examples/laguna/model-service.yaml:5

  • The sample curl command omits chat_template_kwargs.enable_thinking. Per the PR’s own note, omitting it can cause a leading </think> to leak into content, so the copy/paste example will produce degraded output. Update the example to explicitly set enable_thinking:false (or document the difference).
# then call it, e.g.:
#   curl "$ADDR/v1/chat/completions" -H 'Content-Type: application/json' \
#     -d '{"model":"laguna","messages":[{"role":"user","content":"hello"}]}'

docs/content/examples/laguna.md:18

  • The recipe page doesn’t mention the enable_thinking gotcha described in the PR (omitting it can leak a leading </think> into content). Adding a short note here will prevent users from thinking the deployment is broken when they do a first test call.
This recipe was run end to end on Nebius (`eu-north`): serving and tool calling
validated on a single 8x H100 node. `poolside/Laguna-S-2.1-FP8` is a public
repository, so no Hugging Face token or Secret is needed. Apply the platform
side first, then the ML side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants