Skip to content

Commit c94de1d

Browse files
joelteplyclaude
andcommitted
tests: a learn's workspace custody covers the previous registration, what is held, and its own need
The step workspace is engine-wide and its custody passes from one reference to the next; claim_plan refuses ('uncovered') a replacement that does not cover every row the previous reference was registered with. Measured on CPU (stories260K, -np 2): A's long learn registered 7,026,688 bytes, the scheduler then held 4,907,008, and the reloaded B's short reply needed 2,573,344; sized to its own need (or even to need plus held) B's workspace was refused every time. The fixture now remembers the rows it last registered and covers the union of those, shared_compute_held and the example's own requirement, each buffer at the largest. learn-unload-during-learn.py then passes 10/10 on the fixed engine (fork #108). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
1 parent 97f2d87 commit c94de1d

1 file changed

Lines changed: 15 additions & 2 deletions

File tree

‎tools/server/tests/learn_fixture.py‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ def __init__(self, url, name, timeout=600):
3737
self.references, self.grants, self.shadows = [], [], []
3838
self.optimizers = {}
3939
self.minted = 0 # adapter names are immutable files: never reuse one
40+
self.workspace_rows = [] # the rows the current workspace reference was registered with
4041
# A failed assertion must not leave custody bound for the next script on this server.
4142
atexit.register(self.close)
4243
self.grant = self.idle_grant()
@@ -139,8 +140,20 @@ def custody(self, adapter, example):
139140
"-optimizer-%d" % adapter["id"],
140141
{"loaded": {"lora_id": adapter["id"], "path": adapter["path"]}},
141142
[row for row in rows if row["kind"] != "shared_compute_required"])
142-
workspace = self.register("-workspace-%d" % len(self.references), {"workspace": True},
143-
[row for row in rows if row["kind"] == "shared_compute_required"])
143+
# The step workspace is engine-wide and its custody passes from one reference to the
144+
# next: a replacement must cover every row the previous reference was registered with
145+
# (claim_plan), which can exceed what the scheduler holds now. So it covers the union of
146+
# the previous registration, what is held (shared_compute_held) and this example's need,
147+
# each buffer at the largest of them.
148+
workspace_rows = {}
149+
for row in self.workspace_rows + plan["requirements"]:
150+
if row["kind"] not in ("shared_compute_required", "shared_compute_held"):
151+
continue
152+
key = (row["device"], row["buffer_type"], row["host"])
153+
bytes_ = max(row["bytes"], workspace_rows[key]["bytes"]) if key in workspace_rows else row["bytes"]
154+
workspace_rows[key] = dict(row, kind="shared_compute_required", bytes=bytes_)
155+
self.workspace_rows = [row for row in workspace_rows.values() if row["bytes"] > 0]
156+
workspace = self.register("-workspace-%d" % len(self.references), {"workspace": True}, self.workspace_rows)
144157
return {"optimizer_residency": self.optimizers[adapter["id"]], "workspace_residency": workspace}
145158

146159
def learn(self, adapter, reply, grant=None, lr=0.001):

0 commit comments

Comments
 (0)