Repository navigation
server: a learn step is one decode chunk, as serving's is, and its workspace is priced at one ubatch - #102
Open
joelteply wants to merge 4 commits into
Conversation
…rkspace is priced at one ubatch 211a1e0 moved server learning to llama_opt_learn_graph_step: graph_nodes=1 makes opt_learn_step decode the whole reply as one batch, and /learn/preflight priced that graph (llama_opt_measure_shadow_graph at reply_rows). The workspace then grows with reply x served window: on the M5 (Qwen3.6-35B-A3B, core 709fe470f) Aiko's workspace lease asked 11.14 GB with 2.3-4.4 GB free and waited 418 times in 15 minutes; nothing learned. - advance_learning steps with llama_opt_learn_step: one n_batch chunk per call, decoded by decode() in n_ubatch pieces like serving's, other slots decoding between steps. - the preflight measures the accumulation and commit graphs at one ubatch (min(n_ubatch, rows)); only the whole-reply graph phase still prices its rows. - /slots reports learn_gradient_steps (was learn_graph_steps). - learn-allocation-residency.py: a long reply learns inside a short plan's workspace; a reply several chunks long is revoked between chunks with the stable tail restored. Not changed here: the whole-reply graph API (llama_opt_learn_graph_step, llama_opt_measure_shadow_graph) has no server caller now; it is deleted once the chunked-vs-whole gradient validator (Cormac's spec) shows the K/V-projection bias is acceptable, or replaced by recomputing earlier chunks' K/V in the backward if it is not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
… -Werror=missing-field-initializers builds The ubuntu jobs on #102 failed on lines that predate it: /train's llama_opt_params left the four admission callbacks unnamed (server-train.cpp), and pending_learn was aggregate-initialized past its first fields (server-context-learn.inc). /train names the callbacks as null (it allocates under its own budget); pending_learn's fields carry defaults and are set by name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…r type, so gcc -Werror=ignored-attributes builds Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…-Werror=missing-field-initializers builds Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Joel: "You guys keep screwing up how training works in place" / "For training to be far more like inference or use fast disk as it needs".
211a1e0 switched server learning to
llama_opt_learn_graph_step:graph_nodes=1makesopt_learn_stepdecode the whole reply as one batch (src/llama-context-learn.cpp:199), and/learn/preflightpriced that graph (llama_opt_measure_shadow_graph(reply_rows)). The workspace grows with reply × served window.Measured (before)
M5, Qwen3.6-35B-A3B, core 709fe470f, 15 minutes after the deploy:
training.admission.waitingfired 418 times forlearning:75ece751…:speech-discipline:workspace. The workspace asked forfootprint_bytes=11137457312whileavailable_byteswas 2318860288–4437016576. Nothing learned.Change
advance_learningsteps withllama_opt_learn_step: onen_batchchunk per call, whichdecode()splits byn_ubatchas it does for serving, with other slots decoding between steps.min(n_ubatch, rows)). It no longer depends on reply length, so a workspace stays valid instead of being replaced as replies grow./slotsreportslearn_gradient_steps(renamed fromlearn_graph_steps).tests/learn-allocation-residency.py: a long reply learns inside a short plan's workspace, and a reply several chunks long is revoked between chunks.Open, with owners
llama_opt_set_walk_host_budget) with the host/NVMe tier. Owner: Fable.shared_compute_held, and probe the preflight rows. Owner: Fable. Cormac adds a sizing case asserting the priced workspace is bounded by one chunk.Not verified
learn-allocation-residency.pyneeds a run on a non-serving node.🤖 Generated with Claude Code
https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo