Skip to content

ggml-opt: freeing an optimizer checks only its own graph, so an idle shadow unloads while another mind learns - #108

Merged
joelteply merged 1 commit into
codex/s4-per-mind-integrationfrom
cormac/opt-free-own-graph
Nov 11, 2026
Merged

joelteply merged 1 commit into
codex/s4-per-mind-integrationfrom
cormac/opt-free-own-graph

Conversation

@joelteply

Copy link
Copy Markdown

ggml-opt: freeing an optimizer checks only its own graph, so an idle shadow unloads while another mind learns

c5551ed made ggml_opt_free assert that the SHARED scheduler had no pending graph. A serving
context keeps one optimizer per shadow on one scheduler, and llama_context::opt_release frees an
idle shadow's optimizer while another shadow's step is in flight (#85's contract: "ggml_opt_free
touches no shared scheduler"). That assert then aborted llama-server whenever the unload landed
mid-step (fork #106/#107 reproducer, learn-unload-during-learn.py, 2 of 3 runs on CPU).

ggml_opt_free releases only this context's own buffers and metadata. The check is now
context-local: ggml_backend_sched_graph_pending_for(sched, graph), true only when the scheduler is
part-way through THAT graph (graph_step already keys its continuation on the caller's graph
pointer). Freeing a context whose own graph is in flight still aborts. ggml_opt_reset and
ggml_opt_prepare_alloc keep the scheduler-wide guard: reset zeroes the last graph's gradients,
which in a dynamic graph can sit in the shared compute buffer another mind now uses, and
prepare_alloc allocates onto the scheduler.

Tests (test-opt-dynamic-accum, built with -fsanitize=address on CPU, IntelMac):

  • run_frees_idle_mind_while_another_steps: B learns a step, A stops one node into its step
    (scheduler pending), B is freed, A completes with exactly the update it makes alone. PASS.
  • run_free_own_pending_graph_aborts: freeing a context whose own graph is in flight aborts
    (observed in a forked child; skipped with a message where fork() is unavailable). PASS.
  • the existing suite: PASS.
  • Mutation: restoring the scheduler-wide check aborts the first test at the same assert the
    engine hit (exit 134).
    End to end: the reproducer passes 10/10 on the fixed CPU server, but it is timing-dependent (it
    aborts only if the unload lands mid-graph; 0/6 on the unfixed engine in a later run), so the unit
    test is the proof.

Card 5cfd134a. Base: codex/s4-per-mind-integration.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE

joelteply added a commit that referenced this pull request Nov 11, 2026
…what is held, and its own need

The step workspace is engine-wide and its custody passes from one reference to the next; claim_plan
refuses ('uncovered') a replacement that does not cover every row the previous reference was
registered with. Measured on CPU (stories260K, -np 2): A's long learn registered 7,026,688 bytes,
the scheduler then held 4,907,008, and the reloaded B's short reply needed 2,573,344; sized to its
own need (or even to need plus held) B's workspace was refused every time. The fixture now
remembers the rows it last registered and covers the union of those, shared_compute_held and the
example's own requirement, each buffer at the largest. learn-unload-during-learn.py then passes
10/10 on the fixed engine (fork #108).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
…shadow unloads while another mind learns

c5551ed made ggml_opt_free assert that the SHARED scheduler had no pending graph. A serving
context keeps one optimizer per shadow on one scheduler, and llama_context::opt_release frees an
idle shadow's optimizer while another shadow's step is in flight (#85's contract: "ggml_opt_free
touches no shared scheduler"). That assert then aborted llama-server whenever the unload landed
mid-step (fork #106/#107 reproducer, learn-unload-during-learn.py, 2 of 3 runs on CPU).

ggml_opt_free releases only this context's own buffers and metadata. The check is now
context-local: ggml_backend_sched_graph_pending_for(sched, graph), true only when the scheduler is
part-way through THAT graph (graph_step already keys its continuation on the caller's graph
pointer). Freeing a context whose own graph is in flight still aborts. ggml_opt_reset and
ggml_opt_prepare_alloc keep the scheduler-wide guard: reset zeroes the last graph's gradients,
which in a dynamic graph can sit in the shared compute buffer another mind now uses, and
prepare_alloc allocates onto the scheduler.

Tests (test-opt-dynamic-accum, built with -fsanitize=address on CPU, IntelMac):
- run_frees_idle_mind_while_another_steps: B learns a step, A stops one node into its step
  (scheduler pending), B is freed, A completes with exactly the update it makes alone. PASS.
- run_free_own_pending_graph_aborts: freeing a context whose own graph is in flight aborts
  (observed in a forked child; skipped with a message where fork() is unavailable). PASS.
- the existing suite: PASS.
- Mutation: restoring the scheduler-wide check aborts the first test at the same assert the
  engine hit (exit 134).
End to end: the reproducer passes 10/10 on the fixed CPU server, but it is timing-dependent (it
aborts only if the unload lands mid-graph; 0/6 on the unfixed engine in a later run), so the unit
test is the proof.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
@joelteply
joelteply force-pushed the cormac/opt-free-own-graph branch from 39f76e7 to 3902672 Compare November 11, 2026 17:18
@joelteply

Copy link
Copy Markdown
Author

Review (Fable): PASS. The claim checked: the narrowed assert still guards a context's OWN in-flight step. ggml_opt_eval_step steps the scheduler with exactly opt_ctx->allocated_graph_copy (ggml-opt.cpp:2039), and the scheduler records that pointer as continuation_graph (ggml-backend.cpp:2742), so ggml_backend_sched_graph_pending_for(sched, allocated_graph_copy) is true exactly when this context's graph is mid-step; another context's pending graph no longer blocks freeing an idle one. The forked-child test pins the abort for the own-pending case.

@joelteply
joelteply merged commit 2250a67 into codex/s4-per-mind-integration Nov 11, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant