Repository navigation
server: GET /lora-adapters answers from the set the main loop publishes, never queued behind decoding slots - #98
Merged
joelteply merged 1 commit intoNov 7, 2026
Conversation
…es, never queued behind decoding slots IntelMac 2026-11-07 ~09:00Z: GET /lora-adapters gave no answer in 60 s on a loaded CPU engine (4 slots prefilling 10-17k-token prompts, one 2,048-token chunk ~45 s) while /health answered, because the route posted a normal-priority task and waited for the main loop to reach it. A core adopting a live engine therefore could not learn its adapter set, read it as empty and relaunched the engine cold (continuum card d9f2b63a: from=0 to=7, every resident's KV lost). The main loop now publishes the report (publish_lora_report, the one builder: every loaded adapter by the id requests address it by, with path, scale, task_name, prompt_prefix and any alora invocation) at load_model and at every change to the set: a scale change (SET_LORA), a runtime load (LOAD_LORA) and a retire. The route reads that snapshot under a mutex (lora_report_snapshot, public beside publish_busy_slots) and posts no task. SERVER_TASK_TYPE_GET_LORA, server_task_result_get_lora and its to_json had no other user and are deleted. The JSON is the same shape as before. Tests: tools/server/tests unit/test_lora.py 4 pass, 1 skip (slow), on the IntelMac build at this head. New test_the_adapter_report_follows_every_change_to_the_set: launch, scale change, runtime load (new ServerProcess.train_dir -> --train-dir) and retire (scale 0 first, as the engine requires). Mutation: without the publish after a runtime load, it fails. Not tested: the non-blocking property itself; the tiny CI model never makes a main-loop pass long enough to show the old wait. It holds by construction, because the route posts no task. Card d765a668. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
joelteply
merged commit Nov 7, 2026
d6e9b39
into
889580fc/seq-state-across-stream-counts
14 of 17 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
IntelMac 2026-11-07 ~09:00Z: GET /lora-adapters gave no answer in 60 s on a loaded CPU engine (4 slots
prefilling 10-17k-token prompts, one 2,048-token chunk ~45 s) while /health answered, because the
route posted a normal-priority task and waited for the main loop to reach it. A core adopting a live
engine therefore could not learn its adapter set, read it as empty and relaunched the engine cold
(continuum card d9f2b63a: from=0 to=7, every resident's KV lost).
The main loop now publishes the report (publish_lora_report, the one builder: every loaded adapter
by the id requests address it by, with path, scale, task_name, prompt_prefix and any alora
invocation) at load_model and at every change to the set: a scale change (SET_LORA), a runtime load
(LOAD_LORA) and a retire. The route reads that snapshot under a mutex (lora_report_snapshot, public
beside publish_busy_slots) and posts no task. SERVER_TASK_TYPE_GET_LORA, server_task_result_get_lora
and its to_json had no other user and are deleted. The JSON is the same shape as before.
Tests: tools/server/tests unit/test_lora.py 4 pass, 1 skip (slow), on the IntelMac build at this
head. New test_the_adapter_report_follows_every_change_to_the_set: launch, scale change, runtime load
(new ServerProcess.train_dir -> --train-dir) and retire (scale 0 first, as the engine requires).
Mutation: without the publish after a runtime load, it fails. Not tested: the non-blocking property
itself; the tiny CI model never makes a main-loop pass long enough to show the old wait. It holds by
construction, because the route posts no task.
Card d765a668.
Co-Authored-By: Claude Opus 5.5 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE