Repository navigation
refactor(schema): derive every word-list enum from a vocabulary table (S3) - #302
Merged
Merged
Conversation
… (S3) Every closed set of words in schema/ was written out alternative by alternative. DESIGN.md 10.1 S3 / D-PAT-3 requires the opposite shape: a vocabulary table keyed by word, its word list DERIVED from the table, and the enum an author meets as or() of that list -- so the table, the derived list and the accepted set cannot disagree, and a new word is ONE row. 135 tables now stand behind 136 sites: 137 of the umbrella's 187 spec union census rows are gone (measured: 187 -> 50, 0 added). The 50 left are the ones this pattern does not cover: 45 definition/shape/list-or-string/matcher unions, 4 word lists carrying an explicit empty-string alternative (an empty string is not a word, so it is not a table row), and one def-level union whose word set is a sub-expression of a larger word-or-struct union. The authored surface does not move. spec/cue_types_gen.go and spec/vocab_gen.go regenerate BYTE-IDENTICAL (both are pinned by TestGenReproducible), so the generated Go API -- and with it the sdk contract and every plugin that imports this module -- is unchanged. schema/vocab_s3_test.go is the gate: every table carries its @vocab owner, its derived list IS its rows, every table is cited and every citation resolves, and a row injected into every table must reach the derived list (proving the list derives rather than restates). It reports "no vocabulary tables found" on the pre-change tree, so it cannot pass over nothing.
Review — PASSHead SHA: Security screen: PASS — no secrets, no egress, no guardrail/spec/rulebook edits; the change is confined to Verdict: PASS ollama-cloud/deepseek-v4.1-flash — action-review. |
github-actions Bot
pushed a commit
to opencharly/opencharly
that referenced
this pull request
Nov 10, 2026
…the census (#971) The `spec` pin moves to the merged `refs/heads/main` tip 70528371bf06c226ba689eee2ee34e35af0fdc5f — opencharly/spec#302, squash c3ba5bcd1288f3d9ab7f58a3cdf51dec5e6a0b6f (byte-for-byte the head's tree), tag v0.2026314.1717, with the tag-on-merge CHANGELOG commit one above it, which is the ref the pin's `upstream:main` intent names. The advance is 137 word-list unions converted to §10.1 S3 vocabulary tables across 30 schema files, and the three ledgers that read the pinned CUE move with it in the same commit: * `design/dpat-baseline.tsv` — re-emitted by its own emitter, 137 rows removed (every one a `union` row in `spec/schema/*.cue`), 0 added; the spec `union` census reads 187 → 50. * `design/dulling.tsv` — nine citations the citation control caught as falsified by the advance, re-anchored from the drifting `<path>:<line>` form to the drift-proof `<path>:#<Definition>` form, each naming EVERY definition its old range covered. * `design/cite-baseline.tsv` — regenerated by `--emit-baseline`, exactly those nine rows lowered. `git ls-remote https://github.com/opencharly/spec refs/heads/main` and `git ls-tree HEAD spec` agree on the recorded pin. No `sdk` lockstep applies: `spec/spec/cue_types_gen.go` and `spec/spec/vocab_gen.go` carry the SAME blob hash at both pins, so no exported identifier and no vocabulary slice moved. Assisted-by: DeepSeek Harness ollama-cloud/deepseek-v4.1-flash (fully tested and validated) Co-authored-by: aitrawog <dsh-opencharly@opencharly.local>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
specis the producer: the schema every entity in the org is judged against. It carried187
unioncensus rows — disjunctions written out alternative by alternative. This PRconverts every pure word list among them into the §10.1 S3 vocabulary table:
137 rows are swept, 50 are left, and 0 are added (census emitted on a scratch umbrella
worktree whose
specgitlink is this head — below). 135 tables stand behind 136 sites. Thegenerated Go is byte-identical:
spec/cue_types_gen.goandspec/vocab_gen.gobothregenerate with no diff, so the module's API — and therefore the sdk contract and every
plugin that imports
github.com/opencharly/spec/schema— does not move at all.Refs opencharly/opencharly#945 (the census), #950 (the sequence), and the fourteen proven
legs: plugin-spice#15 and plugin-cdp#8 (the two read for this PR), plus plugin-jetkvm#15,
plugin-cua#10, plugin-punktfunk#20, plugin-agent-pi#11, plugin-gh#13, plugin-omarchy#11,
plugin-agent#15, plugin-task#65, plugin-desktop-kind#4, plugin-herdr#15, plugin-appium#12,
plugin-vm#94.
The one judgement this repo needs, and it is a measured correction of the brief. The
brief's working number was "98 of 187 are pure word lists". Measured at this pin with two
independent instruments (the calibrated line scanner and the CUE AST), the split is:
*"word"default (*"locked" | "shared" | "private")137 of the 142 word-list rows are swept; the other 5 are left with reasons (below). So the
headline is 137 of 187, not 98 — and the four rows the brief's own recipe would have
swept blind are exactly the ones that needed a decision.
What was swept, and what was left
Swept — 137 census rows, 28 schema files
vm.cue25 ·deploy.cue16 ·agent_control.cue16 ·kubevirt.cue10 ·firecracker.cue7 ·desktop.cue7 ·candy.cue7 ·box.cue6 ·engine.cue5 ·workflow.cue4 ·kubernetes.cue4 ·distro_vocab.cue4 ·vm_box_metadata.cue3 ·llm.cue3 ·00_common.cue3 ·seam.cue2 ·marketplace.cue2 ·instrument.cue2 ·cli_model.cue2 ·status.cue1 ·skill.cue1 ·pipeline.cue1 ·oci.cue1 ·node.cue1 ·kindcluster.cue1 ·init.cue1 ·distro.cue1 ·agent.cue1.Three shapes needed care, and each is now stated once in
schema/README.md("Vocabulary tables (S3)"):
sharing?: *"locked" | or(_sharingWord)with_sharing: {locked, shared, private}. Thedefault is part of the surface (omitting the key materializes it), and the accepted set is
unchanged. 26 rows are this shape.
interaction_modes?: [...or(_interactionModesWord)]— five rows.mode?: *"internal" | "external",autostart: *false | true): S3's one permitted inline enum. 34 such unions carry nocensus row and are untouched.
Left — 50 rows, and why each stays
#VmValue: (#Vm | #DeployValue)), a key-tagged struct union (#MatchOpMap: {equals: _} | …,#Step's five arms), a list-or-scalar (#PortScope,#MatcherList,llm.stop?: string | [...string]), a word-or-def (tool_choice?: "none" | "auto" | "required" | #LLMNamedToolChoice), or astring & !="" | intpair. These are S4 tag-table work (or genuinely not unions of words) and are the experimental-language-flag half the brief keeps out of scope; they stay censused."":#Op.extract,#Security.cgroupns,#Security.ipc_mode,#KubeVirtDomain.machine. An empty string is not a word, so it is not a vocabulary-table row — putting"": {}in a table would make the table's key set stop being a set of words, which is S3's definition. Sweeping them would need a second spelling (or(_xWord) | ""), which is the "invent a second convention" the brief forbids; and they cannot simply drop the alternative without moving the surface. Left, reported, not forced.#Tunnel: (("tailscale" | "cloudflare") | {…})— the word set is a sub-expression of a larger word-or-struct union, so converting the inner union leaves the line's own disjunction in place (the census row would survive with a new symbol). Theprovider:field inside the struct arm is swept.How tested
Every command below was run against this head
(
b293f5963647dec73b2f927ed89f125c75373893) and pasted from its capture file.1. The census delta — step 1, and the instrument
spec's census rows live in the umbrella'sdesign/dpat-baseline.tsv, which this PR doesnot touch (the lead lowers it when the pin advances). To measure the delta I ran the
umbrella's own gate against a scratch worktree of
opencharly/opencharlyatorigin/mainwhose
specgitlink I advanced in the index only (no file edited, no commit):Both directions of the gate were exercised: 137 rows report as a SHORTFALL (the census
overstates the debt — the intended direction, lowered by the lead) and zero rows report
as a new violation, which is the assertion that proves the sweep added no union anywhere,
or(...)included.The gate ran the tree at
b293f59from the pinned object store. Its first run failed LOUD onthree sibling pins (
plugin-desktop-kind,plugin-herdr,plugin-spice) whose commits wereabsent from this clone's stores;
git --git-dir .git/modules/<path> fetch originmade eachreadable, and the run above is the second, complete one. That is an environment condition,
not a finding about this diff, and it is stated because the gate refuses to judge fewer
modules than it claims.
The line scanner I used for row-by-row accounting was calibrated against the census before
I trusted it: run over the pre-change schema it reproduces exactly the census's 187
distinct
(file, symbol)union rows and its 197 occurrences — nothing in the census missingfrom the scan, nothing in the scan missing from the census. Only then were its 187 → 50
(137 removed, 0 added) figures used.
2. Regeneration, and a second run is byte-identical — step 2
spec's projection is generated in-process (internal/schemagen→schemaparams/the vocab emitter), so the instrument is the generator itself, not a CLI:
Both generated files are byte-identical to their pre-change content — no type, field
name, tag or doc comment moved. This is the strongest single fact in the PR:
vocab_gen.goholds the enum values in CUE source order (
ContextWords,OpVerbs,OpFields,AuthoringVerbs,EngineNames,DistroIDs, …), so an enum that reordered or narrowedduring the conversion could not have left it unchanged; and
cue_types_gen.goholds everyGo param struct, so the API did not move either.
3. The surface did not move — step 3, zero verdicts both directions
Two instruments, both able to fail (mutation controls below).
(a) Every swept site, exhaustively. An out-of-tree probe (module-cached CUE,
replacedat this worktree — evidence, not shipped code) resolves each of the 135 tables' sites in a
BEFORE tree (
git archive HEAD^) and the AFTER tree, then compares: the accepted word set,every word's acceptance, five near-misses per site (
"",zzz-not-a-word, uppercased,space-prefixed, suffixed), four wrong-type probes (
true,1,null, a one-element list),the default's resolved value, and — for the five list sites — the same battery through a list
value. Every comparison is symmetric, so a word accepted on one side and refused on the other
is a failure whichever direction it moves.
The three NOTEs are properties of the pre-existing schema, identical on both sides, and each
is why the claim rests on word acceptance rather than enumeration for those three:
#Vm.machinecarries anif backend == "krun" { machine?: !~"^(q35|virt|i440fx|pc)$" }conjunct that surfaces in its expression, and
#Vm.firmware/#Vm.backendsit underif-guards whose conjunction nests past the reader's depth. Their four/four/two words, andtheir defaults, are checked by the acceptance battery all the same.
(b) A real corpus — the org's own authored documents.
specis the schema every entityin the org is judged against, so the corpus is the org's own manifests: every YAML blob at
every pinned submodule's index commit, read from the pinned object stores (not from a
checkout). 2192 files across 407 repos: 714
charly.ymlmanifests (a repo root, a box, a candy or atool each), plus the QA scenario YAMLs and the GitHub-workflow files.
Each document is judged twice — whole-document against
#NodeDoc(the load-timevalidate-before-execute gate) and, for every entity body it carries
(
<name>: {candy: {…}}→#CandyValue,<name>: {vm: {…}}→#VmValue, …), against thekind def the host validates that body with — once with the BEFORE schema and once with the
AFTER schema, and the verdicts compared:
3619 judgements over 2192 real documents (3598 real probes plus 21 YAML files that do not
parse in either schema), 0 verdicts moved, and 0 cases where the verdict held but the error
message changed. Both directions are exercised by the corpus itself:
1655 accepts and 1964 rejects.
Mutation controls — the instruments can fail. A claim of zero needs a positive control or
it is indistinguishable from an instrument that reads nothing:
Both mutations were reverted before the commit;
git status --shortis empty at this head.A measured limit of the corpus half, stated rather than glossed. The corpus proves the
wire surface (
#NodeDocplus 27 kind defs over 1427 real bodies); it is not exhaustive overthe 135 swept enums, because most authoring sites in real documents carry the default
rather than the word (
category:is omitted and materializesdevelopment, so renamingcommandsmoved four verdicts while dropping the default word moved none). Enum coverage iswhat instrument (a) is for — every word of every table, on both sides, by construction.
(c) The repo's own authored documents.
go test ./...runs the schema's own CUE tests(
schema/*_test.go,spec/*_cue_test.go) which already pin accept/reject cases for#Box.init, the ref grammar, the deploy CPU shape, the libvirt GPU vocabulary, the pipelineand workflow envelopes — 29 packages pass on this tree.
4. The schema still loads — both composition paths — step 4
charly box validatedoes not compile a served schema (#301), so neitherproof may be that command.
internal/schemagen:-mode=paramsconcatenatesschema/*.cueminusnode.cueunderpackage spec+@go(spec)and composes itthrough
schemacompose(one CUE module instance);-mode=vocabconcatenates everyfile. Both ran above with
rc=0and byte-identical output — the two different units bothstill compile.
schemaconcat.ConcatSchema(schema.FS)compiled bycharly core's
sharedCueSchema, and — for a plugin —schemacompose.ServedSchema, one CUEmodule instance whose schema imports
github.com/opencharly/spec/schema. The first is whatevery test in this repo compiles (29 packages); the second is probed directly at this head:
That probe also pins the new risk this change introduces — 135 hidden
_xlabels in thecontract package. A hidden label is package-scoped, so a plugin's own
_contextmust notwiden
schema.#Contextand the contract's must not widen the plugin's; the two checks thatreject (
pluginOnlyon the contract side,buildon the plugin side) are exactly that, andthe fourteen merged plugin legs already prove the same separation for
_method/_action.5. Text-parsed disjunctions — the pre-sweep check — step 5
A derived enum has no text body, so anything that reads an enum's SOURCE or its compiled
alternatives by TEXT would see an empty set. Checked before the sweep, in this repo and in
the gates that read it:
internal/schemagen's readers go through the CUE API'sExpr(), not the text, andor()over a concrete list evaluates to the same disjunction — which is exactly what thebyte-identical
spec/vocab_gen.goproves, sinceenumValuesrequires every arm to be astring literal in source order. No file in
schema/orspec/parses a union's text; theone text-level reader of a union anywhere in the org is the umbrella's
design-dpat.sh, andits shortfall finding is the intended one above. Zero instances found in this repo — the
four precedents that found real ones did so in plugin code;
spechas none.6. The repo's gate, on the committed tree
go vetwas run before the push for the reason the brief gives — a helper name that collideswith a package symbol passes
gofmtand onlyvetcatches it; the new test file's helpers(
rowLabels,vocabOwner,harnessFor,decodeLists,compiledSchema) were checked againstschema's exported symbols for that reason. Lint ran undergo1.26.8becauseAGENTS.mdnames go1.27's export-data version as a false failure for this repo's pinned linter.
7. The new gate does not pass over nothing
schema/vocab_s3_test.goasserts the S3 shape over the shipped schema: every table carriesits
@vocabowner, every table's derived list is its rows in declaration order, everytable is cited by some
or(_x<Name>Word)and every citation resolves, no table has one row —and, because equality cannot distinguish a derived list from a restated one, it injects a
row into every table and asserts the derived list gains it, proving the mutation applied
first (
probe-injected-wordmust be absent from the unmutated list).It fails on the pre-change tree, which is what makes it a gate rather than a decoration:
Files changed
schema/00_common.cue…schema/workflow.cue(27 files)schema/README.md@vocabownership, the literal-default rule, the naming rule, and where tables liveschema/vocab_s3_test.go@vocab, one-to-one citations, row-injection mutation, ≥2 rows)spec/cue_types_gen.go,spec/vocab_gen.goRulebook compliance
R0 — skills first.
/charly-internals:go(the SDD/schema-change recipe),/charly-internals:plugin(the per-plugin CUE-schema contract),/charly-internals:git-workflow,/charly-internals:cutover-policyand/charly-internals:strict-policyloaded before thefirst read of the source; this repo's own
AGENTS.mdread fromorigin/mainbefore acting.R1 — RCA every anomaly, fix or file. Four anomalies, each root-caused before remediation:
(1) the umbrella gate refused three sibling pins whose commits were absent from this clone —
environment, remedied by fetching them, not a code finding; (2) the site instrument's first
version reported 18 false failures (union-arm paths, bottom-forbidden fields, list sites,
an unmutated re-read) — each was a defect in the INSTRUMENT, fixed in it; (3)
golangci-lintreported two findings in the new test (a case-only switch, an ignored
HasSuffix) — fixed;(4) the
cue fmtdivergence is a real, pre-existing, separable condition and is filedagainst
schema/*.cueis notcue fmt-clean:cue fmt --check --files schemalists every file on pristine main #274 rather than fixed here. No R1 finding was raised on this PR.R2 — no duplication. One table per vocabulary, placed beside its use, and the two
same-named field pairs with identical word lists (
#PackagingSystemdUnit/#CandyServicerestartandscope;#VmSource's tworootfsarms) are handled deliberately: the two#VmSourcearms share ONE table (same def, same word list), while the two#PackagingSystemdUnit/#CandyServicepairs keep TWO, because a field name is not avocabulary — merging two defs' sets is the trap plugin-spice#15 pinned, and here it would
silently couple two shapes that only coincide today. Stated once in
schema/README.md.R3 — no workarounds. No sleep, retry, suppression or fallback. The scratch umbrella
worktree advanced the pin in its INDEX only; no umbrella file was edited, and no census row
was hand-deleted.
R4 — product before prose.
schema/README.mddocuments the shape the CUE now carries;no prose routes around a defect. The one place the docs could have drifted silently — the
naming rule — is stated once and mechanically true (the test fails on a duplicate table).
R5 — delete legacy completely. The legacy form IS the inline union, and the claim-keyed
grep is the census:
unionrows forspecfall 187 → 50, with the remaining 50 each namedand reasoned above. No identifier is added or removed from the CUE (tables are
_-hidden,so nothing new is exported), and no field, def or Go name changes.
R6 — user work and git safety.
git statusbefore every action; one worktree ofspeccreated off fresh
origin/main(Part II feat(spec)!: host the loader-RESULT type family (#55 Phase B, cutover 1) #2), no submodule edited from the umbrella(Part II feat(spec): initial import — spec + proto wire/IR contract extracted from sdk (#55 Phase A1) #1), no force-push, no amend after the first push, no direct push to
main.R7 — prove behavior. The changed path executes live: both generators run, the module's
29 test packages run on the committed tree, the new gate runs and FAILS on the pre-change
tree, and both surface instruments are mutation-controlled (above). The artifact under test
is the emitted schema the runtime compiles, not a fixture.
R8 — live or skip, never fake a live service. N/A — this change touches no service boundary: it converts CUE schema text and adds an in-process CUE test, so there is no live service to run against and nothing is mocked. The two runtime-adjacent paths it does change (
internal/schemagen's concatenation andschemacompose's module instance) are executed, not simulated.R9 — binary equals source. No binary is involved:
specships no Go binary and itsprojection is in-process (
AGENTS.md: "spec ships no Go binary of its own"). The generatorswere run from this worktree's own module, not from a shared install.
R10 — fresh proof, on the gate that owns the change. This repo declares no per-repo
candy gate and no disposable bed (
AGENTS.md: "The merge gate is the org-widecharly/pr-validator… this repo carries no per-repo candy gate"), so the gate that ownsthis change is the module's own CI (build, test, gofmt, lint) plus
TestGenReproducibleplusthe new S3 gate — all run above on the committed tree with zero warnings. The CONSUMER-side
bed (a disposable charly bed composing the released
specpin) is the lead's leg of theproducer-first sequence: it would prove the pinned module, not this unmerged tree, so it is
named here and not claimed.
Cutover and landing. One phase, one commit: the whole S3 form lands at once, with no
shim, no dual spelling and no follow-up. The census rows live in the umbrella and are
lowered there by the lead when the pin advances — never from this PR.
Placement. The tables live in the CUE module that owns the words (
spec's schemapackage), which is where DESIGN.md §10.2 puts the envelope and the capability types
specowns;
@vocab(spec)is a legal owner (scripts/design-consistency.sh: "core, registry,spec or a §3.1 plugin") and is the one DESIGN.md §3.1 assigns to the capability table.
AI attribution (A3). The trailer carries this runtime's exact names. The harness's own
session descriptor names the profile and the session, and
dsh --versionprints the harnessversion — no config file is the evidence:
The tier
fully tested and validatedis the tier the gates above earned on this tree.Change classification
Schema-internal refactor of the contract module's CUE FORM. Nothing a consumer reads moves:
the generated Go params and the generated vocabulary lists are byte-identical, so every
plugin, the sdk contract and charly core see exactly the same types and the same word lists;
no definition, field, tag or authored word is added, renamed or removed; the accepted
authored surface is unchanged over 135 swept sites (3476 checks) and 2192 real documents
(3619 judgements). What DOES change is the CUE text itself — the derived form replaces the
written-out union — so anything that renders that text verbatim (a generated reference page)
will show the new form after its own regeneration, and a plugin whose served schema imports
github.com/opencharly/spec/schemawill compose 135 more hidden labels, which arepackage-scoped and provably inert (the compose probe above).
Breaking change + rollback: no consumer breaks and no
sdkbump is owed — thelockstep question the brief raises is answered by measurement, not by caution:
sdk'svmshared/spec_aliases.gore-exports spec types andsdk/kitimports it, so a spec bumpwould need an sdk bump only if a type or word it re-exports disappeared — and
spec/cue_types_gen.goandspec/vocab_gen.goregenerate byte-identical, so not oneexported identifier or vocabulary slice changed. Consumers (
plugin-tunnel,plugin-visionand every other importer of
github.com/opencharly/spec/schema) therefore compile and behaveagainst this head exactly as against
fa00807. Rollback is one inverse commit: revertb293f59, andcharly task cue-genregenerates to the same two files that are unchangedhere, so the rollback needs no projection step at all. The only thing that must move with a
revert is the umbrella's census, which must be re-emitted (the 137 rows return).
Filed issues
schema/*.cueis notcue fmt-clean:cue fmt --check --files schemalists every file on pristine main #274 —schema/*.cueis notcue fmt-clean: added this PR's evidence thatthe delta is 4570 lines on pristine
mainand that the new table lines add NO finding oftheir own (
cue fmtrewrites 0 lines carrying@vocabor a_xWordderivation), soadopting
cue fmtis a separate whole-schema reformat, not a cost of this change.Agent: charly
Assisted-by: DeepSeek Harness ollama-cloud/deepseek-v4.1-flash (fully tested and validated)