Hotfix/make qpi driver py more extensible - #33
Closed
Tinitto wants to merge 19 commits into
Closed
Conversation
Adds RFC 0003, which makes the driver backend set open to third parties: an operation becomes a closed enum (it needs a QPI-UI handler), a device becomes an open, data-described thing, and the CLI collapses to one verb `start --operation <op> --device <dev>`. Devices are registered built-in, via a `qpi_driver.devices` entry point, or by import path. Also reverts the WIP extension points explored on this branch, which did not build. A `pydantic.ImportString` annotation on a Typer option raises `RuntimeError: Type not yet supported` while Typer constructs the command tree, taking down every command including `version` and bare `--help`; and a `**kwargs` parameter on a Typer command silently becomes a required positional argument, so `qpi-driver process -d qblox` exited 2 with `Missing argument 'kwargs'`. Both are superseded by RFC 0003. `test_every_command_builds` guards both failure modes: it asserts the whole command tree is constructible and that no operation command grew a positional argument. Verified to fail on the reverted commit. Kept from the WIP: the `recv_timeout_ms` move into the kwargs dict, the deleted commented-out logging setup, and the README import cleanup. Also registers RFCs 0002 and 0003 in the mkdocs nav; 0002 was missing.
Implements RFC 0003 §5-§7 for the Python SDK. The two hard-coded PROCESS_DRIVERS/MONITOR_DRIVERS dicts and the `registry` parameter threaded through the CLI are replaced by one registry of data-only specs, and device builders now return an unstarted driver that the caller starts. An operation (process, monitor) stays a closed enum, because QPI-UI must have a handler for each. A device is the open half: it describes itself with a DeviceSpec declared next to its own code, so a device and its description cannot drift apart, and adding one needs no CLI change. Returning rather than running is what makes a QPU assertable without a server — tests/test_qpu.py now checks every -o option lands on the driver with no NNG socket and no worker subprocess involved. BREAKING CHANGE: run_driver, qpu.run_process, bluefors_gen1.run_monitor, PROCESS_DRIVERS, MONITOR_DRIVERS, DriverRunner, custom_executors (both on QpuDriver and in resolve_executor), and the unread OPERATION class attributes are removed. CHANGELOG.md carries the removed -> replacement table. The driver-authoring surface — QpiDriver, handle_event, emit, every, Event, EventType, Executor — is untouched, as is the CLI grammar. Also fixes the README custom-executor example, which passed a `custom_executor=` keyword no function accepted.
An -o key was documented by hand and validated nowhere: a typo such as `-o data_dirr=/data` was silently ignored, so a driver ran with a default nobody chose, and finding out what a device accepted meant reading its builder. Each OptionSpec now carries a parser, so a device's schema is what checks and converts its values — in one place, rather than each builder hand-rolling int(), boolean-ish strings and path validation. That schema is enough to generate everything else: - `process --help` / `monitor --help` list every device and every -o key with its type, default, example and requiredness, built at import from the registry. A device whose install target is not fully installed is marked `unavailable: pip install "..."`, resolved from package metadata rather than by importing it — discovering what exists must never require having installed all of it. - `qpi-driver devices [--operation OP]` is the same catalog as text. - `qpi-driver catalog --json` is it as JSON, with a schema_version and a documented shape, for QPI-UI and the Go/TS SDKs to check against. Breaking: an unknown -o key exits 1 naming the valid keys, and a builder now receives options already parsed by DeviceSpec.parse_options rather than raw strings. Both are in the CHANGELOG. Also fixes the `use_sdk` process option in both READMEs, which has not existed for some time.
Adding a device meant editing the SDK. Two routes now do not, both landing in the same registry so nothing downstream can tell them apart: - A distribution advertises devices under the `qpi_driver.devices` entry-point group. `pip install mylab-devices` and its devices are in --help, in `catalog --json` and known to --device. An entry point that will not import, resolves to something other than a DeviceSpec, or reuses a registered name is logged and skipped — a third party's mistake must not stop the CLI from starting. - `--device mylab.devices:PrestoV2` imports the device instead, with no packaging at all. `module.attr` works too, per Pydantic's ImportString. For `process` it must be an Executor; for `monitor` a builder; either takes a DeviceSpec, which is how a custom device earns a declared option schema and a --help entry. Both routes run code the operator chose, as `python -m` does, so the obligations are about failing cleanly: a bad path exits 1 with one sentence, and the filesystem path Python volunteers in an ImportError is stripped out of it rather than going to the journal. The safe-path check on data_dir still applies on the import route, because the synthesized spec carries the QPU's declared options. resolve_device lives in builtins/discovery.py rather than in registry.resolve: adapting an imported object needs to know what an Executor is, and qpu imports registry, so the registry would have ended up depending on a device module. Also adds examples/custom_device/ (an Executor, a DeviceSpec, the entry-point stanza) and exports JobPayload/CircuitPayload from qpi_driver, which writing an executor needs.
`qpi-driver process …` and `qpi-driver monitor …` become `qpi-driver start --operation process|monitor …`. The operation was the subcommand, which put a QPI-UI-owned concept into the grammar: everything about launching a driver is identical whichever operation it is, and a third party can add a device but only QPI-UI can add an operation. So the operation is an argument, and the CLI has one verb for running a driver. --operation is required, reads QPI_OPERATION, and has no short form (-o is --option; -O beside it is a hazard in a line usually written once into a unit file). --device and --name now fall back to the operation's own defaults rather than the subcommand's. All three languages at once, because the dashboard's snippet generator emits the launch command per language — changing the grammar one language at a time would have meant branching that generator and unbranching it later. Go and TypeScript also stop claiming a `mock` process device they never had: `start --operation process` there now says the SDK ships no process devices and where to find one, instead of reporting an unknown device with an empty list of known ones. Moved with it: the snippet templates (Spec.subcommand() deleted in favour of the Operation/Device fields already in snippetData), all three install-systemd.sh installers, e2e/lib.sh and e2e/verify.py, and the READMEs. The installers' OPERATION environment variable is unchanged. The five snippet assertions in drivers_test.go had to be rewritten, not just kept: "start --operation process --device qblox" contains "process --device qblox", so they all still passed while no longer discriminating. Mutation-tested afterwards — restoring the old template fails exactly those five. make test-e2e-systemd needs a Docker daemon and was not run here.
The device table was an unexported map[string]deviceRunner inside package main, so nothing downstream could add a device, and qpi-driver/go/qpi-driver had no test file at all. Now there is an importable `devices` package — Operation, OperationSpec, OptionSpec, DeviceSpec, Options, Registry, Register/Devices/Resolve, Catalog — mirroring the Python SDK. Builders return an unstarted driver and qpidriver.Run runs it, so a device can be built and asserted on with no server; the bluefors monitor describes itself as a spec beside its own code, with its five options typed, replacing optOr/secondsOr. Go has no runtime import by name, so extension is compile-time, and the SDK now makes that pleasant instead of pretending otherwise: the cobra wiring moved out of package main into an importable `cli` package, so a binary of your own registers a device, calls cli.Execute, and gets the same start/devices/catalog commands, generated help and option validation. main.go is exactly those two steps. The README example was compiled as a real downstream module and run, which is how the two findings below turned up. `catalog --json` emits the shape Phase 2 froze: bluefors_gen1 is byte-identical to Python's, apart from `extra`, which names a Python install target and is empty where a device is compiled in. Two things fixed along the way: - Python declared `api_key` with `default=""` where Go declares none, so the two catalogs disagreed about a device they share. Python now declares no default and reads it with a fallback. - --help and `devices` advertised an operation's default device even in a build without it, and an omitted --device then failed over a device the operator never typed. Which devices a Go binary has is settled at compile time, so DefaultDevice is a fact about the operation, not the build. make test-e2e-systemd needs a Docker daemon and was not run here; make test-e2e-driver EXECUTOR=mock passes, Go driver included.
The TypeScript SDK now has what the other two do: a `qpi-driver/devices` entry point with Operation/DeviceSpec/OptionSpec/Options, registerDevice/devices/resolve, and a `qpi-driver/catalog` producing the frozen `catalog --json` shape. A device is described as data, so registering one earns it a --help line, a catalog entry, and -o values checked and converted by its own schema instead of by the builder. A device can also be named by import path: `--device ./dist/my.js#Export`. The separator is `#`, not Python's `:`, because `:` is a URL scheme separator in a JS module specifier — a name without `#` is still a registered name, so a typo still gets the known-devices error. Breaking, and the important one: **a CA fingerprint is now required and no code path connects without verifying the pin.** verifyFingerprint used to return quietly on an empty fingerprint, so an unpinned connection was reachable by leaving an argument out — which a copy-pasted command hits by accident. Three test call sites constructed drivers without one; that they stopped compiling is the change working. Also breaking: DeviceRunner is DeviceBuilder and takes parsed Options; an unknown -o key is an error naming the valid ones rather than being ignored; and --recv-timeout-ms is gone. That flag was parsed and thrown away, and it names a polling interval this SDK does not have — the transport is event-driven, so wiring it would have meant inventing a behaviour to justify a flag. --ca-file, the other dead flag, is honoured instead: the CA is written there after it verifies. src/builtins/cli.ts has a test file for the first time; 81 tests across the CLI, the registry and the catalog. One of them caught a bug in the path scrubber written for the §10 obligation — it was eating `file://` prefixes and relative paths, turning "Cannot find module '/a/b.cjs' imported from /src/devices.ts" into "'filb.cjs' from 'srcdevices.ts'". make test-e2e-systemd needs a Docker daemon and was not run here; make test-e2e-driver EXECUTOR=mock passes, TypeScript monitor included.
qpi-ui keeps its own copy of which driver backends exist, to render setup snippets. Nothing stopped it from disagreeing with the SDKs, and it already did: `processSpec` declared no options at all, while every process device reads five. The check follows the direction of truth. Devices and their options belong to the driver, so the server is tested against a checked-in testdata/catalog.json generated by `qpi-driver catalog --json`. Operations belong to the server — an operation is the set of event types it has handlers for — so the SDKs' operation enum is tested against this package's. Regenerating the fixture is then the deliberate act of recording a catalog change, and `make sync-driver-catalog` is named in every failure message. Five directions, each mutation-tested rather than assumed: removing `qblox` from catalog.go, adding a `rigetti` kind, misspelling `data_dir`, dropping `is_dummy`, and inventing a `telemetry` operation in the fixture each fail exactly the intended test. drivers.Option grows Help/Required/Default to mirror the SDKs' option schema, plus InSnippet to separate "the catalog knows this option" from "a copy-pasted command should set it". No process option is pre-filled — they all have working defaults, and setting data_dir in a snippet invites an operator to change something they have no reason to. bluefors_gen1 pre-fills channels and base_url and keeps the other three out. make test-go and make test-e2e-dashboard (110 specs) pass.
The root README still described qpi-driver as "a Python daemon that runs alongside quantum hardware" — a QPU driver with extras, which is the framing RFC 0003 §14 called out. It now describes the framework: an operation is what a driver does and a contract the server implements, a device is the backend implementing it and the open half, and a QPU is one device of one operation. - docs/driver/operations.md gains a runbook section on adding a third-party device on a production node, and on what the four -o validation errors look like — quoted from the CLI, not paraphrased, and checked byte-for-byte against it. - The -o tables in the Python README are generated from `catalog --json` by `make sync-driver-catalog`, grouped by schema so the five process devices do not repeat the same five options five times. - Each SDK README states what it actually ships, since only the Python SDK has a process device, plus its own extension story: entry points and import paths for Python, compile-time Register for Go, `#`-suffixed specifiers for TypeScript. - One consolidated migration table in the CHANGELOG, linked from all three SDK READMEs, covering the CLI grammar, both broken APIs, and the behaviour that changed without a rename. - RFC 0003 is Implemented; RFC 0001 §2 points at it rather than being rewritten. Two fixes found by doing rather than reading. The custom-executor snippet still did not run — Phase 1 fixed its bogus keyword, but it defines only execute() and Executor has a second abstract method, so the class could not be instantiated. Every Python snippet in a changed document is now executed against the real SDK. And the architecture diagrams claimed a "Result Sender Process"; results are pumped by a thread in the main process, and the whole worker arrangement belongs to `process` — a monitor has none of it. mkdocs build --strict is clean, which needed absolute GitHub URLs in the SDK READMEs (they are symlinked into docs/ and also published to PyPI and npm, so relative links resolve nowhere) and the removal of one pre-existing dead link to the gitignored .agents/ROADMAP.md.
Nothing in this repo measured coverage: the Python targets ran pytest with no --cov, Go used -cover with no floor, and jest.config.js set no threshold. All three now produce a number and enforce one, and each gate was checked by making coverage drop and watching it fail rather than by assuming it works. - Python: 96% over the framework modules (achieved 99%, branch coverage on). qpu.py went 40% → 100% and sdk.py 55% → 100%, which meant writing the tests the e2e suite cannot reach: an executor that will not resolve, a job that raises, a malformed payload, a socket that times out, a handler that raises, a CA fingerprint that does not match. - Go: 94% over ./devices/... and ./cli/... (achieved 97.8%). 94 and not 96 because cli.Execute calls os.Exit and cannot be covered in-process — a floor of 96 there would have meant either a fake test or an untrue number. - TypeScript: per-file thresholds on devices.ts, catalog.ts, cli.ts, ca.ts and index.ts (achieved 92–100%), with `global` set just under what the transport reaches, so it cannot regress either. What is excluded is excluded for a stated reason, recorded in the roadmap: the hardware executors need real instruments, the NNG transport needs a live server (and is covered by make test-e2e-driver, which runs all three languages' drivers against one), and process entry points cannot be called by an in-process test. Abstract method bodies became `...` so the coverage config can name them honestly instead of excluding a bare `pass`. Writing the tests found a dead branch: qpu.job_worker checked whether data_dir was already among the executor options, which it never could be — data_dir is a parameter of that same function, so **executor_options can never carry it. Verified: make test-py-base, test-py-cli, test-go-driver, test-js-driver, test-go, and test-e2e-driver EXECUTOR=mock.
The TypeScript SDK set no timeout on any network call it makes, where
Python passes requests(timeout=10) and Go uses http.Client{Timeout: 10 *
time.Second}. Node's fetch falls back to undici's defaults, which are
minutes rather than seconds, and tls.connect has none at all beyond the
OS TCP timeout, so a handshake that stalls after TCP connect waits
indefinitely. Behind a firewall that drops packets a driver therefore
hung inside run() instead of failing, and under systemd's
Restart=on-failure such a unit is never restarted, because it never
fails. Same class of problem as the CA-pinning gap closed in RFC 0003
Phase 6: a silent hang rather than an error.
All three sites now share the same hard-coded 10s the other two SDKs use.
No CLI flag: --recv-timeout-ms was just removed for naming a polling
interval this event-driven transport does not have, and a new flag would
re-litigate that. The only knobs are an internal DialOptions.timeoutMs
and the fetch helper's parameter, both defaulting to 10s, neither
reachable from index.ts or package.json exports — only the tests pass a
shorter one, so the suite does not wait out the real deadline.
- The two HTTP calls go through fetchWithDeadline, which reads the body
inside the deadline as well, so a server that sends headers and then
stalls is caught too. AbortSignal.timeout's "This operation was
aborted" says neither what timed out nor against what, so the message
is rebuilt: "the drivers/connect handshake timed out after 10000ms
against http://…/api/op/drivers/connect". Detection matches on `name`
down the cause chain rather than instanceof Error — the abort is
constructed inside Node and fails instanceof across jest's realm
boundary, which is a real failure this hit, not a hypothetical.
- dial() puts TCP connect, the TLS handshake and the SP header exchange
under one deadline, and unifies its three settle paths so the timer is
cleared once the dial ends. Nothing stays armed afterwards: idling is
normal here, and a live timer would destroy a working socket. The
message names the role, the address and the stage that stalled.
- connect() reads a string body now, so a non-JSON one (a proxy error
page) gets its own line instead of a bare SyntaxError.
Tests cover the case nothing reached before: a peer that accepts the
connection and goes quiet, in both its forms — a plain TCP listener that
never speaks TLS, and a TLS server that never sends its SP header. Plus a
successful dial that idles past the deadline and still receives, run()
against a server that never answers at the real 10s (the point of the fix
is that it rejects at all), a stalled response body, and a refused
connection passed through untouched rather than dressed up as a timeout.
driver.ts and nng.ts are now per-file gated at what the suite reaches, 82
and 95 statements against 78 and 91 before, and 70 and 84 branches
against 40 and 77. Gating them moves them out of `global`, which covers
only events.ts and the Bluefors monitor now, so its floors rise too.
Verified: make lint-js-driver, make test-js-driver, and make
test-e2e-driver EXECUTOR=mock, which runs the built TypeScript monitor
against a live server and so proves the deadlines do not fire on the
happy path.
The SDK READMEs used "the pinned root CA" and "the catalog" as if a reader already knew both. Each now says what it is where it first appears: pinning is refusing any root certificate whose SHA-256 is not the fingerprint the operator was handed out of band, and the catalog is the list of operations, devices and -o options a build knows about, sourced only from the DeviceSpecs. Along the way, the things that were plainly wrong: - The Go README's "Running a built-in as a systemd service" heading had nothing under it, and the JS README's manual procedure was inside an HTML comment while its installer example asked for --device qblox, a process device neither SDK ships. Both now document the monitor devices they have. - The "Upgrading?" note linked CHANGELOG.md#migration, an anchor that slides to whichever release most recently had a migration section. It now names the release boundary and links the change log itself. - The Python README introduced itself as "The Go SDK". - The root README said pip install ./qpi-driver[cli] — a directory with no pyproject.toml — and pointed -o quantify_device_config at a quantify.device.example.json that has never existed; the file is YAML. - docs/driver/operations.md closed with make test-e2e-driver-framework, a target that existed only in the Makefile's .PHONY list.
…ng their own runtime The Go and TypeScript install-systemd.sh were copy-pasted from the Python one and never re-read: both prompted with the Python device list and defaulted to OPERATION=process, DEVICE=mock. Pressing return through the prompts wrote and enabled a unit whose ExecStart can never succeed — neither SDK has a process device — and Restart=on-failure then crash-looped it. Both now prompt with and default to monitor/bluefors_gen1, the one device each actually ships. e2e/test_systemd_install.sh passes monitor/bluefors_gen1 explicitly, which is why nothing caught it. The JS installer also stopped halfway to tell the operator to install Node and run it again, which is not what a one-command installer is for. It now installs Node with nvm as the target user, and — because qpi-driver is a script with a '#!/usr/bin/env node' shebang and systemd's default PATH has no nvm install on it — writes a PATH into the unit that contains that node. Without it the unit started only where Node happened to be system-wide. While here: the Python installer asked for DRIVER_OPTIONS only for a monitor, answering its own FIXME the wrong way round. A process device reads -o keys too; only the data dir and the quantify paths are the installer's to fill in. The QPI_DATA_DIR the Go and JS installers create is for the downloaded root CA, not for a 'data_dir' option neither SDK has a device to read. Said so.
…ere they belong Neither file was where its purpose says it should be. tests/half_imported_device.py is not a test module. It exists to raise a real ImportError so discovery._without_paths can be checked against Python's own message rather than a hand-written one (RFC 0003 §10) — an input to a test, so it goes in tests/fixtures/ alongside the quantify configs. That directory held only data until now, hence the new __init__.py: a fixture that is a module has to be importable by name. qpi-driver/render_catalog_table.py is documentation tooling for this repository, run by make sync-driver-catalog to regenerate the <!-- catalog:begin --> table in the Python README. At the root of qpi-driver/ it read as part of a published SDK. It moves to a new top-level scripts/, and its docstring now says what it is for and how it is run instead of leaving that to be inferred.
Registering kind=mock with language=go succeeded and handed the operator setup commands for '--device mock' against a Go binary that has no process device at all. drivers.Snippets rendered any kind x language pairing, and nothing in internal/drivers knew that only the Python SDK ships a process device — so the pasted command exited 1 saying 'unknown process device', with nothing to say the registration itself was the mistake. Spec.Languages records which SDKs ship each device. handleDriverCreate rejects a pairing that does not exist, naming what that SDK does ship; Snippets falls back to the install command alone rather than rendering a broken one; and the dashboard disables the languages a kind is unavailable in instead of offering a choice the server will refuse. Custom stays registerable everywhere — it is code the operator writes against the SDK, so the SDK not having the device is the whole point. Spec.Languages is qpi-ui's copy of something only the SDKs know, which is how this drifted in the first place, so it is checked: make sync-driver-catalog now writes one fixture per SDK and TestSpecLanguagesMatchTheSdks compares the two directions. catalog.json becomes catalog.python.json. Generating the TypeScript fixture turned up that its CLI had no 'catalog --json' — JSON is the default there and only --text existed, while the Go and Python CLIs accept the flag and all three READMEs spell the command with it. Added, as the no-op the other two have. TestSnippetsOfficialGoUsesGoInstall asserted the bug (Snippets(Qblox, Go) rendering '--device qblox'), so it is now the test that the pairing renders nothing runnable. TestSnippetsBlueforsPerLanguage already covered what it was really for: that an official Go or TypeScript driver resolves per-language run snippets.
The token is a driver's whole identity: it is what handleDriverConnect looks the record up by and, transitively, what says which QPU the driver belongs to (RFC 0001 §8). 'name' is a cosmetic, non-unique display label an admin types into the dashboard — nothing looks a driver up by it, and no unique index exists. Yet the connect handler wrote it straight from the request body on every connect, so a --name in a unit file nobody re-reads silently overwrote what the admin chose, every restart. So the direction reverses. drivers/connect returns 'name'; the driver reads it after connecting (driver.name in Python and TypeScript, DriverName() in Go) and uses it where it always did — the outbound envelope's 'driver' field and its own logs. Removed, all three SDKs: --name/-n, QPI_DRIVER_NAME, the Name/name config field, and default_name/DefaultName/defaultName on the operation specs, an operation having no name to default. Name is gone from DriverConnectRequest; an older driver that still sends one is not rejected, the field is just not read. Two things fell out of it: - qpu._sanitize_name existed because QpuDriver overrode the executor's name with its own, and a display label is not an identifier — so a cryostat called 'lab-1' produced datasets whose 'backend' attribute said 'lab_1'. The executor keeps its own name now, which is what that attribute is for, and the sanitiser has nothing left to sanitise. The executor tests already asserted the new behaviour; only production disagreed with them. - The installers' QPU_NAME becomes SERVICE_NAME. It never named a QPU or a driver: it names the unit file, its SyslogIdentifier and its data directory. Host and Version stay accepted on DriverConnectRequest and are flagged as dead on the wire — no SDK has ever sent either. Also documented, in RFC 0001 §6, the asymmetry a reader of the envelope needs: the server parses an inbound 'driver' field and then ignores it, keying everything off the socket the event arrived on, and fills the same field with the driver's ID on the way out. A driver does not get to claim who it is there either.
The Python README offered three co-equal ways to run a backend the SDK does not
ship — pass an executor, name one by import path, register a device — and left
the reader to work out which was theirs. The Go and TypeScript READMEs each tell
one story ("Adding a device of your own"), and this one now does too: the device
first, its entry point second, the import path as the try-it-out route, and
QpuDriver(executor=…) as a note explaining when you would reach past all of it.
No mechanism changed.
Writing the worked example out properly turned up that it does not work.
qpi_driver.builtins called load_installed_devices() at its own import time, and
qpi_driver/__init__.py imports it on the way to binding Executor, JobPayload and
OptionSpec — so a device written the documented way, `from qpi_driver import
Executor`, was loaded against a half-initialised package and skipped:
WARNING skipping device entry point 'quantum_x': ImportError: cannot import
name 'Executor' from partially initialized module 'qpi_driver'
That is the SDK's whole story for shipping a device, and it worked for nobody.
Discovery moves to the end of qpi_driver/__init__.py, which is the first moment
the package a device imports is complete; importing any qpi_driver submodule
runs that file first, so no route into the registry skips it. The existing tests
missed it by mocking importlib.metadata.entry_points — installing the example
for real is what found it, and is what make test-docs will do.
The example itself: ThermometerExecutor becomes QuantumXExecutor and
probe_count becomes qubit_count, since it is a process device (a QPU) and a
thermometer is a monitor. Its execute() returned a dict where Executor declares
xr.Dataset — the contract process_result() and the driver's dataset writing both
depend on — so it returns a dataset. And its pyproject.toml asks for
qpi-driver[cli] rather than the bare SDK (without the extra there is no typer
and no qpi-driver script, so the README's next command could not run), resolved
from the sibling source tree so a test installs this checkout.
Nothing checked the documentation before a merge: docs.yml only ever ran on a v* tag. So every claim a document makes about this repository was checked by whoever last read it, and documentation drifts in one direction only — nobody re-runs the command they copied a block from. This release alone shipped a make target that existed only in .PHONY, a `pip install` path with no pyproject.toml at it, a config file named with the wrong extension, a TypeScript block that did not type-check, an executor example that could not be instantiated, and an entry-point device that no installed distribution could ever have registered. `make test-docs`, in a new CI job on every pull request, in five steps so a failure says which kind of claim broke: - **static** (scripts/check_docs.py) — every `make` target, repository path, markdown link and `qpi-driver` flag a document names. The flags are read from each SDK's own --help, and each document is checked against the SDK it belongs to, so a removed flag cannot linger in a README. - **snippets** — the Python blocks executed against the real SDK with `run()` stubbed; the Go and TypeScript blocks extracted and compiled against this checkout (scripts/check_doc_snippets.sh); and the four error transcripts in docs/driver/operations.md compared with what the CLI actually prints. - **example** — examples/custom_device installed against the local SDK and then asked for via `catalog --json`. The entry-point route had only ever been tested with importlib.metadata.entry_points mocked. - **catalog** — sync-driver-catalog re-run and diffed against a snapshot rather than against git, which cannot tell a stale generated table from any other uncommitted work. The snapshot is restored either way, so a failing run reports the drift instead of quietly fixing it. - **site** — mkdocs build --strict, which is what catches a dead nav entry or internal link. A block that is meant to fail opts out with `<!-- docs-check: skip -->`; a compiled block opts in with `<!-- docs-check: compile=<name> -->`, because a three-line struct literal is illustration, not a program. CHANGELOG.md is exempt altogether: it records commands that no longer work on purpose. What it found, beyond the entry-point bug already fixed: - The TypeScript registerDevice example did not type-check — `build`'s two parameters were implicitly `any` for want of a `: DeviceSpec` annotation, and the driver it constructed passed a property its options type did not have. - `qpi-driver --help` and `--version` exited 1 on the TypeScript SDK, where the Go and Python CLIs exit 0. exitOverride() turns help into a rejected promise and the bin reported it as a failure, so a `set -e` script that asks a CLI what it can do died on the answer. - The Go README's Config literals still carried a Name field, and the Bluefors and custom-device blocks would not have compiled with it. - The Python README's "Universal options" reference omitted --recv-timeout-ms. Also here, since it is the same "an installer should bring its own runtime" fix as the JS one and the FIXME asking for it is in the file: the Go install-systemd.sh downloads and unpacks the Go toolchain into /usr/local/go when the node has none, rather than exiting with a link. GO_VERSION overrides which; an architecture with no prebuilt tarball still exits with the link. `make lint-py` now covers scripts/ as well as the SDK — it holds the two Python tools this repository runs on itself.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The idea
RFC 0001 made "a driver" a first-class thing: an external process that exchanges
typed events with QPI-UI, built against an SDK. It delivered two kinds of driver —
a QPU that runs jobs (
process) and a cryostat monitor that reports upward(
monitor) — and a CLI that launches either.What it did not deliver is a way for anyone outside this repository to add one.
Today the set of runnable backends is two dictionaries compiled into the Python
SDK. A lab with a cryostat we have never heard of, or a QPU control stack we do not
ship an executor for, has exactly one route: fork the SDK, or write a bespoke
program and give up the CLI, the systemd installer, and the dashboard's setup
snippets along with it.
This RFC makes the backend set open, and does it without adding a second way to
do anything. Three ideas carry the whole design:
(
process,monitor) is a fixed enum, because QPI-UI must have a handler forthe events it involves. The device — which backend implements that contract — is
open-ended, and that is where extension happens.
A
DeviceSpecsays what a device is called, what-ooptions it takes, and howto build it. Nothing about a device is knowable only by running it — so
--helpcan list every option, and every device is unit-testable without aserver.
qpi-driver start --operation <op> --device <dev>. Adding anoperation stops meaning "add a subcommand", and adding a device stops meaning
"edit the SDK".
Changelog
Added
qpi-driver/py: Added the device catalog (RFC 0003 §5) —Operation,OperationSpec,DeviceSpec,OptionSpecandDeviceBuilderinqpi_driver.builtins.registry, withregister(),operations(),devices()andresolve(). A device now describes itself as data next to its own code, so--deviceaccepts it without any change to the CLI.qpi-driver/py: Addedqpi_driver.builtins.qpu.device_spec()for describing aprocessdevice that runs a given executor, including one the SDK does not ship.qpi-driver/py:process --helpandmonitor --helpnow list every device the operation can run and every-okey each one reads, with its type, default, example and whether it is required — generated from the device specs, so a new device appears with no CLI change. A device whose install target is not fully installed is listed asqblox — unavailable: pip install "qpi-driver[cli,qblox]", read from package metadata without importing anything, so--helpworks when a device does not.qpi-driver/py: Addedqpi-driver devices [--operation OP], the same catalog as readable text for every operation at once.qpi-driver/py: Addedqpi-driver catalog --json, the whole catalog as JSON for another program to read. The shape is a contract — QPI-UI checks its own catalog against it, and the Go and TypeScript SDKs mirror it — carries aschema_version, and is documented in full onqpi_driver.builtins.catalog.catalog_dict(RFC 0003 §9).qpi-driver/py: AddedDeviceSpec.parse_options(),OptionSpec.parse/OptionSpec.type_nameandqpi_driver.paths.as_safe_dir()— an option's value is checked and converted by its own schema, in one place, instead of each builder hand-rollingint(...), boolean-ish string parsing and path validation.qpi-driver/py: A distribution can now advertise devices through theqpi_driver.devicesentry-point group (RFC 0003 §6).pip install mylab-devicesand its devices appear in--help, incatalog --jsonand to--device, indistinguishable from a built-in, with nothing to change in the SDK. An entry point that will not import or does not resolve to aDeviceSpecis logged and skipped, never fatal.qpi-driver/py: A device can now be named by import path —--device mylab.devices:PrestoV2, ormylab.devices.PrestoV2, following Pydantic'sImportStringconvention. Forprocessit must import to anExecutorsubclass or instance, formonitorto a device builder; either accepts aDeviceSpec, which is how a custom device gets a declared option schema and a--helpentry. Having no declared schema, an import-path device's undeclared-ooptions are passed through as strings — its declared ones,data_dirincluded, are still validated, so the safe-path check applies on this route too. A path that will not import exits 1 with one line, with the filesystem path Python volunteers stripped out of it (RFC 0003 §10).qpi-driver/py: Addedqpu.device_spec(options=...)for a custom process device that wants to declare its executor's own-ooptions, andDeviceSpec.accepts_any_optionfor one that cannot.qpi-driver/py: Addedqpi-driver/py/examples/custom_device/— anExecutor, aDeviceSpec, and thepyproject.tomlentry-point stanza, with a README covering all three ways to run it.qpi-driver/py:JobPayloadandCircuitPayloadare now exported fromqpi_driveritself, which is what writing an executor needs.qpi-driver/go: Added the importabledevicespackage —Operation,OperationSpec,OptionSpec,DeviceSpec,DeviceBuilder,Options,Registry,Register,Devices,Resolve,Catalog— the Go counterpart of the Python SDK's device catalog (RFC 0003 §5, §8). The device table was an unexportedmap[string]deviceRunnerinpackage main, so nothing downstream could extend it.qpi-driver/go: Added the importableclipackage withNewRootCmdandExecute. Go has no runtime import by name, so extension is compile-time: a build of your own callsdevices.Registerand thencli.Execute, and gets the samestart/devices/catalogcommands, the same generated help and the same option validation as the shipped binary.qpi-driver/go/qpi-driver/main.gois now exactly that — register the built-ins, hand off. The README documents it with an example that is compiled as part of verifying it.qpi-driver/go: Addedqpi-driver devices [--operation OP]andqpi-driver catalog --json, emitting the same document as the Python SDK — byte-identical for the sharedbluefors_gen1device, apart fromextra, which names a Python install target and is empty where a device is compiled in.qpi-driver/go: Thebluefors_gen1monitor now describes itself as aDeviceSpecbeside its own code, with its five-ooptions typed.qpi-driver/go/cliandqpi-driver/go/deviceshave tests whereqpi-driver/go/qpi-driverhad none at all.qpi-driver/js: Added theqpi-driver/devicesandqpi-driver/catalogentry points —Operation,OperationSpec,OptionSpec,DeviceSpec,DeviceBuilder,Options,registerDevice,devices,resolve,catalog,renderCatalog— also re-exported from the package root (RFC 0003 §5, §8). Registering a device makes the CLI run it, with generated help, acatalog --jsonentry, and-ovalues checked and converted by the device's own schema.qpi-driver/js: A device can now be named by import path —--device ./dist/my-device.js#MyExport. The separator is#rather than Python's:because:is a URL scheme separator in a JavaScript module specifier; a name without#is still read as a registered name, so a typo gets the known-devices error rather than an import failure. The export may be aDeviceSpecor a builder. A path that will not import exits 1 with one line, with the absolute paths Node volunteers reduced to their last segment (RFC 0003 §10).qpi-driver/js: Addedqpi-driver devices [--operation OP]andqpi-driver catalog --json, in the same frozen shape as the Python and Go SDKs.qpi-driver/js:src/builtins/cli.tshas a test file, andsrc/devices.ts/src/catalog.tsare covered too — 81 tests where the CLI had none.qpi-driver/js:--ca-fileis now honoured: the downloaded root CA is written there after it has been verified, as the Python and Go SDKs do. It was parsed and ignored, so the file never appeared.qpi-driver,qpi-ui: Added coverage measurement and gates, where nothing measured coverage before.make test-py-clienforces 96% over the Python framework modules (achieved 99%),make test-go-driverenforces 94% over the Godevicesandclipackages (achieved 97.8%), andnpm testenforces per-file thresholds on the TypeScript catalog, CLI and CA pinning (achieved 92–100%). Each gate was checked by making coverage drop and watching it fail. The hardware executors and the NNG transport are reported but not gated: they need real instruments or a live server, and are covered bymake test-e2e-driver—.agents/ROADMAP-0003-driver-extensibility.mdrecords every exclusion with its reason.qpi-driver/py: The QPU worker, the result pump, the SDK's receive loop and shutdown, the pinned-CA download, and the job envelope's validation all have unit tests now — the failure paths the e2e suite cannot reach: an executor that will not resolve, a job that raises, a malformed payload, a socket that times out, a fingerprint that does not match.docs: Added an "Adding a device on a production node" runbook section (docs/driver/operations.md) — how an operator discovers what a node can run, the two ways a third-party device gets there, and what each of the four-ovalidation errors looks like, quoted from the CLI rather than paraphrased.docs: The-ooption tables inqpi-driver/py/README.mdare now generated fromcatalog --jsonbymake sync-driver-catalog, between<!-- catalog:begin -->markers, so they cannot fall behind the code.docs: Addedmake sync-driver-catalog, which regenerates both the qpi-ui drift fixture and the README option tables from the Python SDK's catalog.qpi-ui: Added a drift check between this server's driver catalog and the SDKs' (RFC 0003 §9).qpi-ui/internal/driversnow tests itself against a checked-intestdata/catalog.jsongenerated byqpi-driver catalog --json, and fails when the two disagree about which devices exist, which install target ships them, or which-okeys each reads — and the other way, when the SDKs know an operation this server has no handler for. Regenerate the fixture withmake sync-driver-catalog, which every failure message names; doing so is the deliberate act of recording a catalog change.qpi-ui:drivers.Optionnow carriesHelp,Required,DefaultandInSnippetalongsideExample, mirroring the SDKs' option schema. Theprocessdevices' five-okeys are filled in, whereprocessSpecpreviously declared none — they have working defaults, so none of them is pre-filled in a rendered snippet, but the catalog now says they exist.InSnippetis what decides that:bluefors_gen1pre-fillschannelsandbase_urland keepsapi_key,poll_intervalandtimeoutout of a copy-pasted command.Changed
qpi-driver: [BREAKING] Theprocessandmonitorsubcommands are replaced by onestartverb taking--operation, in all three SDKs at once (RFC 0003 §4). One verb because everything about launching a driver is the same whichever operation it is — and because a third party can add a device, but only QPI-UI can add an operation, so the operation is an argument rather than part of the grammar.qpi-driver process --device qblox …qpi-driver start --operation process --device qblox …qpi-driver monitor --device bluefors_gen1 …qpi-driver start --operation monitor --device bluefors_gen1 …--operationis required, has no short form (-ois--option, and-Obeside it would be a hazard), and readsQPI_OPERATION.--deviceand--name, when omitted, now come from the operation's own defaults. The dashboard's setup snippets, all threeinstall-systemd.shinstallers and the e2e harness render the new grammar; theOPERATIONenvironment variable the installers take is unchanged.qpi-driver/go,qpi-driver/js: [BREAKING]start --operation processnow says that the SDK ships no process devices and where to find one, instead ofunknown process device "mock"; known devices:with an empty list and a default device that never existed there (RFC 0003 §8).qpi-driver/js: [BREAKING] A CA fingerprint is now required, and there is no longer any code path that connects without verifying the pin. The SDK skipped verification entirely whencaFingerprintwas absent, so an unpinned connection was reachable by leaving an argument out — the kind of opt-out a copy-pasted command hits by accident (RFC 0003 §10).QpiDriverOptions.caFingerprintis now required, andverifyFingerprintthrows on an empty one rather than returning quietly.qpi-driver/js: [BREAKING]DeviceRunneris nowDeviceBuilder, and a builder receives(config, options)whereoptionsis anOptionscarrying values already checked and converted against the device's own schema, rather than aRecord<string, string>. An-okey the chosen device does not read is now an error naming the keys it does, where before it was silently ignored.qpi-driver/js: [BREAKING]--recv-timeout-msis gone. It was parsed and ignored, and it names a polling interval this SDK does not have — the TypeScript transport is event-driven, so there is nothing for it to time out. Accepting it was advertising a knob that did nothing.qpi-driver/py: [BREAKING] An-okey the chosen device does not read is now an error naming the keys it does, where before it was silently ignored. A typo such as-o data_dirr=/dataused to mean a driver running with a default nobody chose; it now exits 1. Anything that passed an unrecognised-okey deliberately, expecting it to reach the executor, must declare it in aDeviceSpec(seeqpu.device_spec()).qpi-driver/py: [BREAKING] A device builder is now handed options that have already been checked and coerced against its ownDeviceSpec—build_from_options(options=spec.parse_options(raw))— rather than raw strings. Calling a builder directly with{"job_timeout": "30"}no longer coerces it, and an omitted key is no longer defaulted by the builder.qpi-driver/py: [BREAKING] A device builder now returns an unstarted driver and the caller starts it, matching the TypeScript SDK (RFC 0003 §7). A driver can therefore be built and asserted on with no server running.run_driver(...)QpuDriver(...).run()qpu.run_process(device=..., ...)qpu.build_from_options(executor=..., ...).run()bluefors_gen1.run_monitor(...)bluefors_gen1.build_from_options(...).run()builtins.PROCESS_DRIVERS,builtins.MONITOR_DRIVERSbuiltins.devices(operation)/builtins.resolve(operation, device)builtins.DriverRunnerbuiltins.DeviceBuilder(returns a driver rather than blocking)resolve_executor(executor, custom_executors, ...)resolve_executor(executor, ...)— pass the class or instance itselfQpuDriver(custom_executors={"name": Cls})QpuDriver(executor=Cls)QpuDriver.OPERATION,BlueforsGen1Driver.OPERATIONDeviceSpec.operationqpu.execute_job()qpu.job_worker()qpi-driver/py: The driver-authoring surface is unchanged —QpiDriver,handle_event(),emit(),every(),Event,EventTypeandExecutorkeep their names and signatures. Only how a driver is registered and launched moved.Fixed
qpi-driver/js: The SDK set no timeout on any outbound network call, where the Python and Go SDKs both use 10 seconds. A server behind a firewall that drops packets left the driver hanging insiderun()instead of failing — Node'sfetchfalls back to undici's defaults, which are minutes rather than seconds, andtls.connecthas none at all beyond the OS TCP timeout, so a TLS handshake that stalls after TCP connect waited indefinitely. Under systemd'sRestart=on-failuresuch a unit is never restarted, because it never fails. Thedrivers/connecthandshake, the root CA download and both NNG dials now share the same hard-coded 10s deadline as the other two SDKs, and one that expires says what timed out and against which address rather than raising a bare abort. The deadline bounds the dial only: an idle connection afterwards is normal for this event-driven transport, which is why--recv-timeout-mswas removed rather than repurposed.qpi-driver/go:--helpanddevicesno longer advertise an operation's default device in a build that does not have it, and omitting--devicein such a build now asks for one, naming what is registered, instead of failing over a device the operator never typed. Which devices a Go binary has is decided when it is compiled.qpi-driver/py: Removed a dead branch inqpu.job_worker, which tested whetherdata_dirwas already among the executor options — it never could be, being a parameter of that same function.docs: The rootREADME.mddescribedqpi-driveras a Python QPU daemon; it now describes the driver framework it is, with the operation/device split that makes it extensible and a QPU as one device of one operation (RFC 0003 §14). Each SDK README states what it actually ships, since only the Python SDK has aprocessdevice.docs: Corrected the driver architecture in both the root and Python READMEs: results are pumped by a thread in the main process, not a third "Result Sender Process", and the whole worker arrangement belongs to theprocessoperation — amonitorhas none of it.docs: Fixed the custom-executor example inqpi-driver/py/README.mda second time: it defined onlyexecute(), soExecutor's other abstract method made it impossible to instantiate. Every Python snippet in a changed document is now executed against the real SDK, which is how this was found.docs: RFC 0003 isImplemented, and RFC 0001 §2 now points at it for the operation/device layer rather than being edited in place.qpi-driver/py: Fixed the custom-executor example inqpi-driver/py/README.md, which passed acustom_executor=keyword that no function accepted and would have failed withUnknown executor name 'custom'.