feat(schemaparams): generate params in-process over the offline compose

charly.yml

@@ -23,6 +23,9 @@ repo: github.com/opencharly/spec
 # -----------------------------------------------------------------------------
 # bootstrap-cue — ensure ./bin/cue is the pinned cue CLI (checksum-verified).
 # Idempotent: a present, correct bin/cue short-circuits without a download.
+# NOT part of cue-gen: generation is in-process. The CLI is the pinned tool for
+# AUTHORING the schema as a module (`cue eval ./schema`, `cue vet`, a scratch
+# `cue exp`), so it stays available without being on the generation leg.
 # -----------------------------------------------------------------------------
 bootstrap-cue:
   task:
@@ -33,8 +36,8 @@ bootstrap-cue:
         command: |
           set -euo pipefail
           # The version, URL, checksum and arch rules live in ONE place —
-          # internal/cuesetup -> cuetoolchain — shared with the charly-side generation
-          # verb, so this task and that verb can never run different toolchains.
+          # internal/cuesetup -> cuetoolchain — so this task and any other consumer
+          # of the pinned CLI can never run different toolchains.
           GOWORK=off go run ./internal/cuesetup bin
         context: [deploy]
 
@@ -112,25 +115,29 @@ proto-gen:
 # -----------------------------------------------------------------------------
 # cue-gen — regenerate every CUE-owned artifact.
 #
-#   spec/cue_types_gen.go  (cue exp gengotypes — the Go param structs)
+#   spec/cue_types_gen.go  (schemaparams — the Go param structs, generated in-process
+#                           from the CUE Go API; no `cue` CLI in the loop)
 #   spec/vocab_gen.go      (internal/schemagen — the vocab lists)
 #
 # Reproducible: two consecutive runs produce no diff (TestGenReproducible).
+# No bootstrap-cue dependency: the params generator is in-process, so the pinned
+# CUE CLI is not part of this task (bootstrap-cue remains for `cue eval ./schema`).
 # -----------------------------------------------------------------------------
 cue-gen:
   task:
     description: Regenerate all CUE-owned artifacts, including the protobuf wire contract
     dir: .
-    depends_on: [bootstrap-cue, wire-gen]
+    depends_on: [wire-gen]
     plan:
-      - run: gengotypes + schemagen the generated Go
+      - run: generate the Go params in-process + schemagen the vocab lists
         command: |
           set -euo pipefail
           # 1) The params file comes from the ONE pipeline (schemaparams): concatenate
-          #    schema/*.cue under `package spec` + `@go(spec)`, run the pinned cue
-          #    gengotypes (provisioned checksum-verified into ./bin), normalize the
-          #    struct tags. No step of it is re-implemented here or in schemagen, and
-          #    the charly-side verb calls the same function.
+          #    schema/*.cue under `package spec` + `@go(spec)`, compose them as a CUE
+          #    module instance, generate the Go types in-process with the CUE Go API,
+          #    normalize the struct tags. No step of it is re-implemented here or in
+          #    schemagen, the charly-side verb calls the same function, and no `cue`
+          #    CLI is provisioned: composition and generation share ONE resolution path.
           GOWORK=off go run ./internal/schemagen -mode=params -schema=schema -pkg=spec -out=spec/cue_types_gen.go
           # 2) Companion generator -> the vocab lists.
           GOWORK=off go run ./internal/schemagen -mode=vocab -schema=schema -out=spec/vocab_gen.go

cue.mod/module.cue

@@ -5,8 +5,9 @@
 // The language version is PINNED, not incidental: the schema's `close()` +
 // embedded-definition idiom (schema/00_common.cue's #Op arms) is legal through
 // v0.17 and rejected by v0.18.0's semantics ("field not allowed"), so v0.16.1 —
-// the version spec's own cuetoolchain pins for `cue exp gengotypes` — is the
-// version that keeps the package's VALUE identical to the concatenation's.
+// the CUE library version this module pins in go.mod, which both the in-process
+// params generator and every composing consumer evaluate under — is the version
+// that keeps the package's VALUE identical to the concatenation's.
 // Raising it is the §10.1 pattern rewrite (TODO 2.16), not a bump.
 module: "github.com/opencharly/spec"
 

cuetoolchain/cuetoolchain.go

@@ -1,19 +1,19 @@
 // Package cuetoolchain is THE single home for the pinned `cue` CLI (R3).
 //
-// The schema→Go pipeline runs `cue exp gengotypes`, and that generator lives in
-// `cuelang.org/go`'s INTERNAL tree (`internal/encoding/gotypes`), so no other
-// module can call it in-process. The CLI is therefore a toolchain the pipeline
-// provisions itself — checksum-verified, from a pinned release — so a reader who
-// is told to run a `charly` command never sees `cue`, a checkout path, or a `cd`.
+// The schema→Go params pipeline no longer runs this CLI: `schemaparams` composes and
+// generates in-process through the CUE Go API (opencharly/spec#271), so generation and
+// composition share one offline resolution path and no binary is provisioned. The CLI
+// remains the pinned tool for AUTHORING the schema as a module — `cue eval ./schema`,
+// `cue vet`, a scratch `cue exp` — provisioned checksum-verified from a pinned release
+// by the `bootstrap-cue` task and by the charly-side cache directory helper.
 //
-// The pin AND the provisioning live here, once: the `charly`-side generation verb
-// calls Ensure to obtain the toolchain, and spec's own `bootstrap-cue` task is
-// re-keyed onto it too (it used to carry the same version, URL and checksum inline
-// in shell — two declarations of one toolchain, which is exactly what this package
-// removes). One pin, one home, one fetcher.
+// The pin AND the provisioning live here, once: `cuetoolchain.Ensure` is what every
+// caller uses to obtain the toolchain (the `bootstrap-cue` task used to carry the same
+// version, URL and checksum inline in shell — two declarations of one toolchain, which
+// is exactly what this package removes). One pin, one home, one fetcher.
 //
 // STANDALONE: stdlib only, so both callers can import it without pulling in the
-// packages they regenerate.
+// packages they generate.
 package cuetoolchain
 
 import (
@@ -97,7 +97,7 @@ func Ensure(dir string) (string, error) {
 	}
 	sum, ok := SHA256(runtime.GOARCH)
 	if !ok {
-		return "", fmt.Errorf("the pinned cue %s is not checksum-pinned for %s/%s (only linux/amd64 is) — pin it in github.com/opencharly/spec/cuetoolchain, or run the pipeline in a container",
+		return "", fmt.Errorf("the pinned cue %s is not checksum-pinned for %s/%s (only linux/amd64 is) — pin it in github.com/opencharly/spec/cuetoolchain, or use a linux/amd64 environment",
 			Version, runtime.GOOS, runtime.GOARCH)
 	}
 	url := URL(runtime.GOARCH)

internal/cuesetup/main.go

@@ -1,9 +1,9 @@
 // Command cuesetup provisions the pinned `cue` CLI into a directory. It is the
-// task-facing entry point to cuetoolchain.Ensure, so `charly task bootstrap-cue`
-// and the `charly`-side generation verb (opencharly/charly#829) cannot disagree
-// about which toolchain the schema→Go pipeline runs, and so the pin, its URL and
-// its checksum exist in exactly one place (cuetoolchain) instead of once in Go and
-// once in shell.
+// task-facing entry point to cuetoolchain.Ensure, so `charly task bootstrap-cue` and
+// any other consumer of the pinned CLI cannot disagree about which toolchain they run,
+// and so the pin, its URL and its checksum exist in exactly one place (cuetoolchain)
+// instead of once in Go and once in shell. It is NOT on the generation leg: the
+// schema→Go params pipeline composes and generates in-process (no CLI).
 package main
 
 import (

internal/schemagen/main.go

@@ -4,7 +4,8 @@
 //
 //	-mode=params -out=<file>   Concatenate schema/*.cue into one file
 //	                           headed `package spec` + the file-level `@go(spec)`
-//	                           attribute, ready for `cue exp gengotypes`.
+//	                           attribute, and generate the Go types in-process
+//	                           through schemaparams (CUE Go API).
 //	-mode=vocab  -out=<file>   Compile the concatenation via the embedded
 //	                           cuelang.org/go API and emit spec/vocab_gen.go
 //	                           — the single-source vocabulary lists (kind keywords,
@@ -24,10 +25,10 @@
 // PARAM-GEN SCOPE (the two modes diverge by INPUT, not by mechanism): the
 // `vocab` mode reads the FULL schema (every schema/*.cue) — it needs #Node's
 // arms (node.cue) to derive KindWords, so its concatenation stays byte-identical
-// to the runtime `sharedCueSchema`. The `params` mode (which feeds `cue exp
-// gengotypes` → the Go param structs) reads the schema MINUS node.cue: it
+// to the runtime `sharedCueSchema`. The `params` mode (which feeds the in-process
+// Go generator → the Go param structs) reads the schema MINUS node.cue: it
 // defines the node-disjunction wrappers (#Node/#NodeDoc/#*Arm/#*Value), which
-// gengotypes degrades to `map[string]any` and which are NOT charly param structs.
+// the generator degrades to `map[string]any` and which are NOT charly param structs.
 // (The egress validation schemas moved to candy/plugin-egress in M16, so they no
 // longer live here to exclude.) The entity defs (#Box/#Deploy/#Op/#Vm/…) do not
 // reference node.cue, so the exclusion is compile-clean. Both modes share the ONE concatSchema helper
@@ -82,9 +83,9 @@ func fatal(format string, a ...any) {
 }
 
 // excludeParamGen reports whether a schema/*.cue file is EXCLUDED from the
-// param-gen concatenation (the gengotypes input). node.cue defines the
-// node-disjunction wrappers — not a charly param struct, and gengotypes degrades
-// it to `map[string]any`. See the package doc PARAM-GEN SCOPE note. The vocab
+// param-gen concatenation (the generator input). node.cue defines the
+// node-disjunction wrappers — not a charly param struct, and the generator
+// degrades it to `map[string]any`. See the package doc PARAM-GEN SCOPE note. The vocab
 // mode passes `nil` (no exclusion) so #Node's arms are present for KindWords.
 // (The egress validation schemas moved to candy/plugin-egress in M16, so there
 // are no egress_*.cue files here to exclude.)
@@ -106,8 +107,8 @@ func concatSchema(dir string, exclude func(name string) bool) (string, []string,
 }
 
 // specSource returns the concatenation headed with the `package <pkg>` clause and
-// the file-level `@go(<pkg>)` attribute — what `cue exp gengotypes` consumes to
-// emit that Go package. pkg is "spec" for the core schema and "params" for a
+// the file-level `@go(<pkg>)` attribute — what the generator reads to name the
+// Go package it emits. pkg is "spec" for the core schema and "params" for a
 // plugin's self-contained schema (same one concatenation contract — R3). The cue
 // API (vocab mode) compiles the same string. The exclude filter scopes the
 // param-gen input (see excludeParamGen).
@@ -120,18 +121,19 @@ func specSource(dir, pkg string, exclude func(name string) bool) (string, error)
 }
 
 // ----------------------------------------------------------------------------
-// retag mode — the Go-native yaml-tag doubling (replaces the former cue:gen sed)
+// params mode — the ONE schema→Go pipeline (concatenation, module-aware compose,
+// in-process generation, tag normalization)
 // ----------------------------------------------------------------------------
 
 // writeParams emits the GENERATED Go params file by calling the ONE pipeline
 // (schemaparams): concatenate the param-gen schema subset under the `package <pkg>` +
-// `@go(<pkg>)` header, run the pinned cue gengotypes (provisioned through
-// cuetoolchain.Ensure into ./bin), normalize the struct tags. None of those steps is
-// implemented here any more — that is what makes the pipeline ONE thing rather than a
-// copy per caller. `excludeParamGen` (node.cue) is this caller's only difference, and
-// it is a parameter.
+// `@go(<pkg>)` header, compose it as a CUE module instance, generate the Go types
+// in-process (CUE Go API, no `cue` CLI), normalize the struct tags. None of those steps
+// is implemented here any more — that is what makes the pipeline ONE thing rather than
+// a copy per caller. `excludeParamGen` (node.cue) is this caller's only difference,
+// and it is a parameter.
 func writeParams(dir, out, pkg string) error {
-	src, err := schemaparams.GenerateExcluding(dir, pkg, "bin", excludeParamGen)
+	src, err := schemaparams.GenerateExcluding(dir, pkg, excludeParamGen)
 	if err != nil {
 		return err
 	}

schema/README.md

@@ -28,8 +28,8 @@ are the single source of truth for these consumers:
 > package load, which is why the shared definitions live in `00_common.cue`: the
 > `00_` sorts first, keeping the concatenation's order — and so the generated
 > Go — byte-identical.) The `package spec` clause + the file-level `@go(spec)`
-> attribute that `cue exp gengotypes` needs are injected by the gen pipeline
-> (`internal/schemagen -mode=params`), NEVER written into the source.
+> attribute that the generator reads are injected by the gen pipeline
+> (`internal/schemagen -mode=params` → `schemaparams`), NEVER written into the source.
 
 ## The package form
 
@@ -47,11 +47,13 @@ the module that owns it is `specmodule.FS` (package `github.com/opencharly/spec`
 Composing with only the files would evaluate them under the consumer's toolchain default
 instead of the pinned version above.
 
-`cue.mod/module.cue` pins language version `v0.16.1` — the version spec's own
-`cuetoolchain` pins for `cue exp gengotypes`. It is not incidental: the schema's
-`close()` + embedded-definition idiom (`#Step`'s arms embedding the closed `#Op`)
-is legal through v0.17 and rejected by v0.18's semantics with `field not allowed`.
-Raising the language version is the §10.1 pattern rewrite (TODO 2.16), not a bump.
+`cue.mod/module.cue` pins language version `v0.16.1` — the version the embedded
+`cuelang.org/go` library is pinned to in `go.mod`, and the ONE declaration both the
+in-process generator and every composing consumer evaluate under. It is not
+incidental: the schema's `close()` + embedded-definition idiom (`#Step`'s arms
+embedding the closed `#Op`) is legal through v0.17 and rejected by v0.18's semantics
+with `field not allowed`. Raising the language version is the §10.1 pattern rewrite
+(TODO 2.16), not a bump.
 
 ## Regenerating `spec` — `charly task cue-gen`
 
@@ -62,31 +64,32 @@ charly task cue-gen
 ```
 
 The task is `cue-gen` in this repo's `charly.yml` (the `kind: task` surface). It
-depends on `bootstrap-cue` + `wire-gen` (which in turn depends on
-`bootstrap-protoc`):
-
-1. **Bootstraps the pinned cue CLI** into `./bin/cue` (gitignored) — `v0.16.1`,
-   the SAME version as charly's embedded `cuelang.org/go` library, so the CLI that
-   runs `gengotypes` matches the library that compiles the schema. It downloads
-   the checksum-pinned release tarball (a no-op once `./bin/cue` is at the
-   pinned version).
-2. **Concatenates** `schema/*.cue` via `internal/schemagen -mode=params`
+depends on `wire-gen` (which in turn depends on `bootstrap-protoc`):
+
+1. **Concatenates** `schema/*.cue` via `internal/schemagen -mode=params`
    (the SAME `schemaconcat.ConcatSchema` contract as the runtime
    `sharedCueSchema` — one concatenation contract; the per-file `package
    schema` clause is stripped), headed `package spec` + `@go(spec)`.
-3. **`cue exp gengotypes`** → `spec/cue_types_gen.go` (the Go param structs).
-4. **`internal/schemagen -mode=vocab`** → `spec/vocab_gen.go`
+2. **Composes and generates in-process** (`schemaparams`): the concatenation is
+   loaded as ONE CUE module instance through `schemacompose` (the same offline
+   resolution a plugin's served schema uses), and the Go param structs are
+   generated from the CUE Go API → `spec/cue_types_gen.go`. No `cue` CLI is
+   provisioned or executed.
+3. **`internal/schemagen -mode=vocab`** → `spec/vocab_gen.go`
    (`KindWords`, `DocDirectives`, `StepKeywords`, `ContextWords`, `OpFields`,
    `OpVerbs`, `AuthoringVerbs`, … — all read straight from the compiled schema
    via the cue API).
-5. **`gofmt`** the committed generated files.
+4. **`gofmt`** the committed generated files.
 
 `wire-gen` (chained first) additionally renders `protocol/schema/*.cue` into
 `proto/plugin.proto` and compiles the Go gRPC stubs.
 
-Each plugin's OWN `params/cue_types_gen.go` is generated from its self-contained
+Each plugin's OWN `params/cue_types_gen.go` is generated from its own
 `candy/plugin-*/schema/*.cue` in the plugin's OWN repository, via the SAME
-`internal/schemagen` + pinned cue pipeline (R3). The former superproject
+`schemaparams` pipeline (R3) — including a schema that imports
+`github.com/opencharly/spec/schema`, which resolves through the same offline
+compose and emits the type through the Go package that owns it
+(`github.com/opencharly/spec/spec`). The former superproject
 `taskfiles/Cue.yml` wrapper — which chained this repo's `cue:gen` and then looped
 over each plugin — was removed with the candy de-submodule work (charly#452,
 merged 2026-08-29); plugin param generation now lives with the plugin.
@@ -103,11 +106,13 @@ the `*_gen.go` files** — change the `*.cue` source and regenerate.
 
 ## The `@go(...)` annotations
 
-`cue exp gengotypes` is lossy: every CUE disjunction degrades to `any` /
+The params generator is lossy: every CUE disjunction degrades to `any` /
 `map[string]any` / an empty struct (Go cannot express a disjunction), and field
 names get a naive `leadingCap + underscores-preserved` mapping. Two annotation
 classes in the `*.cue` source fix this — both are **inert for runtime validation**
-(CUE attributes never participate in unification):
+(CUE attributes never participate in unification). Schema comments that name
+`gengotypes` describe this mapping: the generator is a faithful in-process port of
+`cue exp gengotypes` v0.16.1 (`schemaparams/gotypes.go`), so each statement still holds:
 
 - **Field renames** — `@go(<GoName>)` makes the generated Go field name match the
   hand-written charly param struct (`http` → `HTTP`, `cap_add` → `CapAdd`,