Repository navigation
ggml-opt: freeing an optimizer checks only its own graph, so an idle shadow unloads while another mind learns - #108
Merged
joelteply merged 1 commit intoNov 11, 2026
Conversation
joelteply
added a commit
that referenced
this pull request
Nov 11, 2026
…what is held, and its own need
The step workspace is engine-wide and its custody passes from one reference to the next; claim_plan
refuses ('uncovered') a replacement that does not cover every row the previous reference was
registered with. Measured on CPU (stories260K, -np 2): A's long learn registered 7,026,688 bytes,
the scheduler then held 4,907,008, and the reloaded B's short reply needed 2,573,344; sized to its
own need (or even to need plus held) B's workspace was refused every time. The fixture now
remembers the rows it last registered and covers the union of those, shared_compute_held and the
example's own requirement, each buffer at the largest. learn-unload-during-learn.py then passes
10/10 on the fixed engine (fork #108).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
…shadow unloads while another mind learns c5551ed made ggml_opt_free assert that the SHARED scheduler had no pending graph. A serving context keeps one optimizer per shadow on one scheduler, and llama_context::opt_release frees an idle shadow's optimizer while another shadow's step is in flight (#85's contract: "ggml_opt_free touches no shared scheduler"). That assert then aborted llama-server whenever the unload landed mid-step (fork #106/#107 reproducer, learn-unload-during-learn.py, 2 of 3 runs on CPU). ggml_opt_free releases only this context's own buffers and metadata. The check is now context-local: ggml_backend_sched_graph_pending_for(sched, graph), true only when the scheduler is part-way through THAT graph (graph_step already keys its continuation on the caller's graph pointer). Freeing a context whose own graph is in flight still aborts. ggml_opt_reset and ggml_opt_prepare_alloc keep the scheduler-wide guard: reset zeroes the last graph's gradients, which in a dynamic graph can sit in the shared compute buffer another mind now uses, and prepare_alloc allocates onto the scheduler. Tests (test-opt-dynamic-accum, built with -fsanitize=address on CPU, IntelMac): - run_frees_idle_mind_while_another_steps: B learns a step, A stops one node into its step (scheduler pending), B is freed, A completes with exactly the update it makes alone. PASS. - run_free_own_pending_graph_aborts: freeing a context whose own graph is in flight aborts (observed in a forked child; skipped with a message where fork() is unavailable). PASS. - the existing suite: PASS. - Mutation: restoring the scheduler-wide check aborts the first test at the same assert the engine hit (exit 134). End to end: the reproducer passes 10/10 on the fixed CPU server, but it is timing-dependent (it aborts only if the unload lands mid-graph; 0/6 on the unfixed engine in a later run), so the unit test is the proof. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
joelteply
force-pushed
the
cormac/opt-free-own-graph
branch
from
November 11, 2026 17:18
39f76e7 to
3902672
Compare
Author
|
Review (Fable): PASS. The claim checked: the narrowed assert still guards a context's OWN in-flight step. ggml_opt_eval_step steps the scheduler with exactly opt_ctx->allocated_graph_copy (ggml-opt.cpp:2039), and the scheduler records that pointer as continuation_graph (ggml-backend.cpp:2742), so ggml_backend_sched_graph_pending_for(sched, allocated_graph_copy) is true exactly when this context's graph is mid-step; another context's pending graph no longer blocks freeing an idle one. The forked-child test pins the abort for the own-pending case. |
joelteply
merged commit Nov 11, 2026
2250a67
into
codex/s4-per-mind-integration
11 of 12 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ggml-opt: freeing an optimizer checks only its own graph, so an idle shadow unloads while another mind learns
c5551ed made ggml_opt_free assert that the SHARED scheduler had no pending graph. A serving
context keeps one optimizer per shadow on one scheduler, and llama_context::opt_release frees an
idle shadow's optimizer while another shadow's step is in flight (#85's contract: "ggml_opt_free
touches no shared scheduler"). That assert then aborted llama-server whenever the unload landed
mid-step (fork #106/#107 reproducer, learn-unload-during-learn.py, 2 of 3 runs on CPU).
ggml_opt_free releases only this context's own buffers and metadata. The check is now
context-local: ggml_backend_sched_graph_pending_for(sched, graph), true only when the scheduler is
part-way through THAT graph (graph_step already keys its continuation on the caller's graph
pointer). Freeing a context whose own graph is in flight still aborts. ggml_opt_reset and
ggml_opt_prepare_alloc keep the scheduler-wide guard: reset zeroes the last graph's gradients,
which in a dynamic graph can sit in the shared compute buffer another mind now uses, and
prepare_alloc allocates onto the scheduler.
Tests (test-opt-dynamic-accum, built with -fsanitize=address on CPU, IntelMac):
(scheduler pending), B is freed, A completes with exactly the update it makes alone. PASS.
(observed in a forked child; skipped with a message where fork() is unavailable). PASS.
engine hit (exit 134).
End to end: the reproducer passes 10/10 on the fixed CPU server, but it is timing-dependent (it
aborts only if the unload lands mid-graph; 0/6 on the unfixed engine in a later run), so the unit
test is the proof.
Card 5cfd134a. Base: codex/s4-per-mind-integration.
🤖 Generated with Claude Code
https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE