Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions docs/FACTORY_INTAKE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ comes out, with a person keeping the merge. The stations, and what SyberWork has
| One agent, one ticket, inside the plan | A claim on the ledger; a Build Thread whose mutable scope is the plan's files and whose objective names the request | [FACTORY_WORK.md](FACTORY_WORK.md) |
| Build | A Build Thread: a model proposes commits, the repair loop feeds it SyberWork's own check results | [BUILD_THREAD.md](BUILD_THREAD.md) |
| Review by someone who did not write it | The critic station: findings in the chain, a high finding refuses acceptance | [CRITIC.md](CRITIC.md) |
| A person decides | Accepting moves one branch; pushing, a pull request and a merge are separate acts | [BUILD_THREAD.md](BUILD_THREAD.md), `AGENTS.md` classes |
| A person decides | Accepting moves one branch; pushing, a pull request and a merge are separate acts. `factory work queue` orders what waits for them and says how to read each change | [BUILD_THREAD.md](BUILD_THREAD.md), `AGENTS.md` classes, [FACTORY_WORK.md](FACTORY_WORK.md) |

The station names follow Daniel Moka's "Build Your Own Software Factory" (craftbettersoftware.com). Two things
differ on purpose. The state of a request is an event on the hash-chained ledger, not a label; `label()` gives
Expand Down Expand Up @@ -267,5 +267,3 @@ repository. That naming is the owner's to confirm or change at admission.

- **`critic-weakened-tests`**: reference-critic rules for a newly skipped test, assertions removed from an
existing test, and a credential shape on an added line.
- **A queue for the person who merges**: ready work ordered by what it unblocks, with a reading order by
component.
65 changes: 57 additions & 8 deletions docs/FACTORY_WORK.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@
Build Thread: a plan is written before any code and checked by code, one agent claims the ticket, and the thread
that builds it may change the plan's files and no others.

Package: `syberlabs/factory/plan.py`, `work.py` and `work_cli.py`, on the ledger of [FACTORY.md](FACTORY.md).
Standard library only. Claim ceiling for this page: **implemented** and **tested** (`tests/test_factory_work.py`,
35 tests, and the `factory` simulation campaign). Nothing here is measured: no model has written a plan through
it, and no agent fleet has run on it.
Package: `syberlabs/factory/plan.py`, `work.py`, `review_queue.py` and `work_cli.py`, on the ledger of
[FACTORY.md](FACTORY.md). Standard library only. Claim ceiling for this page: **implemented** and **tested**
(`tests/test_factory_work.py`, 35 tests; `tests/test_factory_review_queue.py`, 15 tests; the `factory` simulation
campaign). Nothing here is measured: no model has written a plan through it, no agent fleet has run on it, and
the queue's order has not been compared with the order a person would choose.

## The four rules

Expand Down Expand Up @@ -157,6 +158,54 @@ with the thread, the candidate, the commit and the branch. That is the end of th
accepted change is on `syberlabs/<thread>`. Pushing it and opening a pull request are the thread's own publishing
actions, and merging is the act [BUILD_THREAD.md](BUILD_THREAD.md) leaves to a person on GitHub.

## The queue for the person who merges

A factory that builds faster than a person can read moves the wait to the merge. `work queue` does not shorten
the reading; it orders it. It lists every `ready` ticket, first to read first, and for each change the order to
read its files in. It is computed from the ledger, the Space graph and Git. No model is asked.

**The order of the queue** is an ordered list of keys; the first difference decides (`review_queue.sort_key`):

| Key | First | Why |
| --- | --- | --- |
| `blocks` | the change more tickets are waiting on | A ticket that is not ready yet waits on a change when its plan names a path the change touches. Merging that change first is what lets the others build on what will be there |
| `files` | fewer files | Small before large is shortest job first: the order in which the changes behind wait least in total |
| `diff_bytes` | the smaller diff | The same, when the file counts are equal |
| `waiting_since` | the one ready longest | So nothing waits forever behind newer work |

A change whose accepted commit is not in the repository the queue is read from (another clone, a deleted
branch) keeps its row, lists its plan's paths, and goes last: what cannot be read is not read first. Every row
carries the keys that placed it, so a person who orders differently sees what they overrule. Kind and size from
intake are shown and are not keys: nothing here knows which request matters more to the business.

**The reading order of one change** (`review_queue.reading_order`) groups the files the accepted commit changes
by component and puts a component after the ones it depends on (the Space graph's `depends_on`), so the code
that is called is read before the code that calls it. Each group is a layer; files with no component are the
last one. Components that depend on each other, and ties, go by name. The paths are the accepted commit's
against the thread's base, not the plan's: a plan may allow a file the change did not touch.

```text
1. github_issue:acme/app#12 rules crash on quantity zero
ticket 5c0f6f0be7b9d6b4 bug S class R owner mateo
placed by: blocks 1, 3 file(s), 612 bytes, ready since event 41
waiting on it: 9a41c2d07e6f1b35
branch syberlabs/7f3a9c at 1e2d3c4b5a; accepted, not pushed or merged here
read in 3 layer(s), each after what it depends on:
1. rules: fxapp/rules.py
2. main: fxapp/main.py
3. (no component): tests/test_rules_request.py
```

A row also names the other ready changes that touch the same files: whichever merges second is rebased.

The queue reads `work_ready` records, and those are written by `work settle`. `work settle` with no ticket looks
at every ticket that has a thread and records the ones that hold an accepted change, so run it before reading the
queue.

What it is not: it splits nothing. A change with several layers can be read layer by layer, and a person who
wants one pull request per layer makes them; the layers say where the cuts are. It pushes and merges nothing,
appends no event and moves no ref. It reads one ledger, so it lists the work of the agents that share it.

## From a terminal

```sh
Expand All @@ -168,6 +217,8 @@ python -m syberlabs.factory work claim OWNER/REPO#12 --by model:claude-1
python -m syberlabs.factory work start OWNER/REPO#12 --by model:claude-1 --check "tests=/abs/python -m unittest -q"
python -m syberlabs.factory work settle OWNER/REPO#12 # exit 0 once the thread holds an accepted change
python -m syberlabs.factory work release OWNER/REPO#12 --by model:claude-1 --reason "blocked on a decision"
python -m syberlabs.factory work settle # the same for every ticket that has a thread
python -m syberlabs.factory work queue # ready work, first to read first; --json for a tool
```

A ticket is named by its id, the first eight or more characters of it, or its request. `--home` is the factory
Expand All @@ -182,8 +233,8 @@ or put it in a script file.
- **Call a model.** The planner is a command; this packet ran only the stand-in
(`tests/fixtures/intake/standin_planner.py`). A planning agent behind it, Claude Code with read-only tools for
example, is an adapter nobody has written here.
- **Choose which ticket an agent takes.** `work list --status planned` shows what may be taken. Ordering it is
the queue packet.
- **Choose which ticket an agent takes.** `work list --status planned` shows what may be taken, in the order the
requests were decided. The queue orders what a person merges, not what an agent starts.
- **Enforce a class.** The floors route a plan and name the thread's class, so a plan a person approved above
`auto` does not become a pull request that says R. The class is still recorded, not enforced (`AGENTS.md`):
neither the kit nor a ruleset asks for the key it names. What a candidate cannot do is leave the plan's paths.
Expand Down Expand Up @@ -221,8 +272,6 @@ one proposed here. The `AGENTS.md` section it adds is canonical text and waits f

## Next packets

- **A queue for the person who merges**: `ready` work ordered by what it unblocks and how much it touches, with
a reading order by component from the Space graph.
- **A planning agent**: Claude Code with read-only tools behind the planner command, on its own credential lane
([CLAUDE_CODE.md](CLAUDE_CODE.md)).
- **Claims across machines**: the ledger anchored somewhere two operators can both read.
1 change: 1 addition & 0 deletions docs/ROADMAP_STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,4 +127,5 @@ A coordination record is not a case event. A displayed comment, diversity score,
| Replay viewer at scale: grid layout past eight components, io in the panel; 40 scrubs in 39 ms on a 3,275-span run | Implemented; one measured run | `syberlabs/static/factory.html` |
| Intake: a request from outside is screened by code (hidden text removed, order shapes and credential shapes found), six closed questions go to one decider command in the System One wire shape, an ordered rule list picks `agent_ready`, `needs_human`, `unclear` or `duplicate` with a 0.80 floor, every answer is on the ledger without the text, and `agent_ready` opens a ticket | Implemented and tested with a stand-in decider; no decision model run; pattern precision unmeasured; Class C, awaiting an owner's key and a confirmed first consumer | `syberlabs/factory/intake.py`, `factory intake ...`, `tests/test_factory_intake.py`, [docs/FACTORY_INTAKE.md](FACTORY_INTAKE.md) |
| Work from a ticket: a planner command names files, steps and risks; code checks the plan against HEAD, computes the class floor and the dependent components from the paths, and an ordered rule list routes it `planned` or `needs_human`; a claim on the ledger keeps one agent per ticket; the Build Thread it starts may change the plan's files and no others, records the class those paths floor at, and names the request in its objective | Implemented and tested with a stand-in planner; no model has planned through it and no pull request was opened from it; Class C, awaiting an owner's key and a confirmed first consumer | `syberlabs/factory/plan.py`, `work.py`, `factory work ...`, `tests/test_factory_work.py`, [docs/FACTORY_WORK.md](FACTORY_WORK.md) |
| A queue for the person who merges: every ticket whose thread holds an accepted change, ordered by how many tickets wait on it, then fewer files, the smaller diff and the longer wait, each with a reading order by component from the Space graph; read-only | Implemented and tested on the fixture app; the order has not been compared with a person's; awaiting the same key as the work line it reads | `syberlabs/factory/review_queue.py`, `factory work queue`, `tests/test_factory_review_queue.py`, [docs/FACTORY_WORK.md](FACTORY_WORK.md) |
| Real model adapters behind the commands; a learned vectorizer; journal forking; change coupling | Proposed (v2) | [docs/FACTORY.md](FACTORY.md) |
145 changes: 145 additions & 0 deletions syberlabs/factory/review_queue.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
"""The queue for the person who merges: every ticket whose thread holds an accepted change, in an order that says
why, with a reading order for each change.

A factory that builds faster than a person can read moves the wait to the merge. This module does not shorten the
reading; it orders it. Two things are computed, from the ledger, the Space graph and Git, and nothing is asked of a
model:

- **The order of the queue.** An ordered list of keys, the first difference decides (``ORDER``): a change that other
tickets are waiting on comes first (their plans name a path this change touches, so merging it first is what lets
them build on what will be there); then the change with fewer files; then the smaller diff; then the one that has
waited longest. Small before large is the shortest-job-first rule: it is the order in which the changes behind
wait least in total. The keys are printed with every row, so a person who orders differently can see what they
are overruling.
- **The reading order of one change.** Its files grouped by component, a component after the ones it depends on
(the Space graph's ``depends_on``), so the code that is called is read before the code that calls it. Each group
is a layer. A change with several layers can be read, and if the person wants, split, layer by layer; this module
splits nothing.

Read-only: ``queue`` appends no event, moves no ref and runs no command but ``git diff`` and ``git rev-parse``.
Accepting was a person's act on the thread, and pushing, opening a pull request and merging stay theirs.
"""

from __future__ import annotations

from pathlib import Path

from syberlabs.errors import Rejected
from syberlabs.factory import intake, work
from syberlabs.factory import plan as planmod
from syberlabs.gitspace import GitError, Repo

ORDER = ("blocks", "files", "diff_bytes", "waiting_since") # most first, then fewest, smallest, oldest (``sort_key``)
WAITING = ("open", "plan_held", "plan_rejected", "planned", "claimed", "working") # not ready, and may hold a plan
NO_COMPONENT = None


def reading_order(space: dict | None, paths: list[str]) -> list[dict]:
"""``[{"layer", "component", "paths"}]``: ``paths`` grouped by the component each belongs to
(``plan.components_of``; the first by name when globs overlap), a component after every component it depends
on. Components that depend on each other in a cycle, and ties, go by name. Paths with no component are the last
layer."""
groups: dict = {}
for path in sorted(paths):
owners = planmod.components_of(space, path)
groups.setdefault(owners[0] if owners else NO_COMPONENT, []).append(path)
touched = sorted(name for name in groups if name is not NO_COMPONENT)
needs = {name: set() for name in touched}
for edge in (space or {}).get("depends_on", []):
if edge["from"] in needs and edge["to"] in needs and edge["from"] != edge["to"]:
needs[edge["from"]].add(edge["to"])
placed: list = []
while len(placed) < len(touched):
free = [name for name in touched if name not in placed and needs[name] <= set(placed)]
placed.append(free[0] if free else next(name for name in touched if name not in placed))
layers = [{"component": name, "paths": groups[name]} for name in placed]
if NO_COMPONENT in groups:
layers.append({"component": NO_COMPONENT, "paths": groups[NO_COMPONENT]})
return [{"layer": number, **layer} for number, layer in enumerate(layers, 1)]


def _changed(repo: Repo, base: str | None, commit: str) -> tuple[list[str], int] | None:
"""The paths the accepted commit changes against the thread's base and the size of that diff in bytes, or None
when either commit is not in this repository any more."""
if not base:
return None
try:
return repo.changed_paths(base, commit), repo.diff(base, commit, max_bytes=0)[1]
except GitError: # one of the two commits is not here
return None


def sort_key(row: dict) -> tuple:
"""Where a row goes: more waiting tickets first, then fewer files, then the smaller diff, then the longer
wait. A change whose commit is gone has no diff to measure and goes after every change that can be read."""
order = row["order"]
return (not row["found"], -order["blocks"], order["files"], order["diff_bytes"] or 0, order["waiting_since"])


def queue(ledger, *, repo_root: str | Path, now: int | None = None) -> list[dict]:
"""Every ``ready`` ticket, first to read first. Each row: ``position``, ``ticket_id``, ``source``, ``title``
(the request's visible title), ``kind``, ``size``, ``act_class``, ``human_owner``, ``thread``, ``target_ref``,
``commit``, ``paths`` and ``diff_bytes`` (of the accepted commit against the thread's base; the plan's paths
and None when the commit is gone, with ``found`` False), ``blocks`` (tickets not ready yet whose plan names
one of these paths), ``overlaps`` (other ready tickets that touch one), ``layers`` (``reading_order``) and
``order`` (the keys that placed it)."""
events = ledger.events()
space = ledger.latest_space()
repo = Repo(Path(repo_root).resolve())
states = work.tickets(ledger, now=now)
waiting = {s["ticket_id"]: set(s["plan"]["files"] + s["plan"]["new_files"]) for s in states
if s["status"] in WAITING and s["plan"] is not None and s["plan"]["plan_digest"] is not None}
rows = []
for current in states:
if current["status"] != "ready":
continue
ready, started = current["ready"], current["work"]
found = _changed(repo, (started or {}).get("base"), ready["commit"])
paths, size = found if found is not None else (sorted(current["plan"]["files"] + current["plan"]["new_files"]), None)
item = work.find(ledger, current["ticket_id"], events)
try:
title = intake.load_snapshot(ledger, item["snapshot_digest"])["visible"]["title"]
except Rejected: # a missing or edited snapshot costs the row its title, not the queue its row
title = ""
rows.append({"ticket_id": current["ticket_id"], "source": current["source"], "title": " ".join(title.split())[:120],
"kind": (item.get("work") or {}).get("kind"), "size": (item.get("work") or {}).get("size"),
"act_class": (started or {}).get("act_class"), "human_owner": current["human_owner"],
"thread": ready["thread"], "target_ref": ready["target_ref"], "commit": ready["commit"],
"found": found is not None, "paths": paths, "diff_bytes": size,
"blocks": sorted(tid for tid, planned in waiting.items() if planned & set(paths)),
"layers": reading_order(space, paths), "order": {"waiting_since": ready["seq"]}})
for row in rows:
row["overlaps"] = sorted(other["ticket_id"] for other in rows
if other is not row and set(other["paths"]) & set(row["paths"]))
row["order"] = {"blocks": len(row["blocks"]), "files": len(row["paths"]), "diff_bytes": row["diff_bytes"],
"waiting_since": row["order"]["waiting_since"]}
return [{"position": number, **row} for number, row in enumerate(sorted(rows, key=sort_key), 1)]


def text(rows: list[dict]) -> str:
"""The queue as a person reads it in a terminal: one block per change, the keys that placed it, where it is,
and its layers in reading order."""
if not rows:
return "nothing is ready: no ticket's thread holds an accepted change\n"
lines = []
for row in rows:
source = row["source"]
size = "size unknown" if row["diff_bytes"] is None else f"{row['diff_bytes']} bytes"
lines.append(f"{row['position']}. {source['kind']}:{source['ref']} {row['title'] or '(no title)'}")
lines.append(f" ticket {row['ticket_id']} {row['kind'] or 'kind unknown'} {row['size'] or ''}".rstrip()
+ f" class {row['act_class'] or 'not recorded'} owner {row['human_owner']}")
lines.append(f" placed by: blocks {len(row['blocks'])}, {len(row['paths'])} file(s), {size}, ready since event "
f"{row['order']['waiting_since']}")
if row["blocks"]:
lines.append(f" waiting on it: {', '.join(row['blocks'])}")
if row["overlaps"]:
lines.append(f" touches the same files as: {', '.join(row['overlaps'])} (whichever merges second is rebased)")
where = f"{row['target_ref']} at {str(row['commit'])[:10]}"
lines.append(f" branch {where}; " + ("accepted, not pushed or merged here" if row["found"]
else "that commit is not in this repository, so the plan's paths are listed"))
many = len(row["layers"]) > 1
lines.append(f" read in {len(row['layers'])} layer(s)" + (", each after what it depends on:" if many else ":"))
for layer in row["layers"]:
lines.append(f" {layer['layer']}. {layer['component'] or '(no component)'}: {', '.join(layer['paths'])}")
lines.append(f"{len(rows)} change(s) ready. Nothing here pushes, splits or merges one.")
return "\n".join(lines) + "\n"
Loading
Loading