docker: fix ATOM and SGLang release builds

[ThomasNing]

Summary

• install protobuf-compiler and libprotobuf-dev for the ATOM mesh Rust build
• bound Cargo to 32 jobs by default so high-core-count builders do not exhaust Docker's default file-descriptor limit
• install the missing SGLang ROCm runtime prerequisites: AMD SMI, source-built TileLang, and the pinned ROCm fast-hadamard-transform

Reproduction

The official release-image path was built and exercised end-to-end on an 8x MI350X (gfx950) host using the Dockerfile default ROCm 7.2.4 / PyTorch 2.10 base. Failures were found sequentially.

The mesh build initially lacked the compiler and then Ubuntu's well-known protobuf definitions:

Could not find protoc
google/protobuf/timestamp.proto: File not found.
google/protobuf/struct.proto: File not found.

On the 192-core builder, unbounded Cargo parallelism exhausted Docker's 1,024-file descriptor limit:

could not execute process ... rustc ...
Too many open files (os error 24)

After the unified ATOM+SGLang image built, GLM-5.2-MXFP4 SGLang TP4 exposed three prerequisites present in SGLang's own ROCm Dockerfile but absent from ATOM's SGLang layer:

ModuleNotFoundError: No module named 'tilelang'
ModuleNotFoundError: No module named 'fast_hadamard_transform'
Failed to initialize QuickAllReduce: name 'amdsmi_shut_down' is not defined

Installing the PyPI TileLang wheel was insufficient because ATOM subsequently pins another TVM FFI revision, producing an ABI failure during DSA execution:

libtvm_compiler.so: undefined symbol: tvm::ffi::ReprPrint(...)

This change therefore follows pinned SGLang v0.5.17 exactly: TileLang commit a55a8230, TVM FFI commit 37d0485b, and ROCm FHT commit 46efb7d7. TileLang is built in an isolated stage so repairing the base image's apt dependency state cannot replace the runtime's custom RCCL package. The QuickAllReduce message is resolved by installing /opt/rocm/share/amd_smi. The new build arguments remain overridable, and expensive earlier layers stay cacheable. No runtime source patch is applied.

Validation

• git diff --check
• verified protoc --version and both imported definitions in the pinned base image
• complete ATOM+SGLang unified image build and provenance/import/entrypoint verifier pass on MI350X
• ATOM GLM-5.2-MXFP4 TP4+MTP served a capped chat completion successfully
• SGLang GLM-5.2-MXFP4 TP4 loaded all 282 shards, captured all 40 graph batches, used TileLang for DSA prefill/decode plus AITER custom all-reduce, became healthy, and returned READY from /v1/chat/completions
• live runtime pins: TileLang 0.1.7.post3+cuda.gita55a8230, TVM FFI 0.1.8.post3.dev42+g37d0485b2

[github-actions]

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

• ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

• ✅ Run after the PR is approved and Pre Checkin passes
• ✅ Run immediately when an approval review is submitted
• ✅ Can be requested before approval with labels

Label	Tests
ci:full	Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom	Run native ATOM model accuracy tests
ci:vllm	Run ATOM vLLM OOT model accuracy tests
ci:sglang	Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 2013 --add-label <label>