diff --git a/SKILL.md b/SKILL.md index 82d764c..62ad5c0 100644 --- a/SKILL.md +++ b/SKILL.md @@ -33,23 +33,22 @@ Each build system is made to emit a structured description of what it *actually builds*, normalized into one **canonical action model**, then compared: ``` -CMake File API codemodel ──extract_cmake.py──┐ - ├─► reconstruct.py ─► diff.py ─► worklist / converged? -Bazel aquery jsonproto ──extract_bazel.py──┘ ▲ - │ - any2bazel.json (migration decisions) +Reference build ──extract_.py──┐ + ├─► reconstruct.py ─► diff.py ─► worklist / converged? +Bazel aquery jsonproto ──extract_bazel.py───────┘ ▲ + │ + any2bazel.json (migration decisions) ``` -- **CMake side = File API codemodel-v2** (not `compile_commands.json`, which - lacks link info). **Bazel side = `bazel aquery`**. Both expose compile *and* - link actions. +- **Reference side** = the structured action description the reference build + exposes (see [Identify reference](#identify-reference)). **Bazel side = + `bazel aquery`**. Both expose compile *and* link actions. - **The model stores raw ACTIONS** (argv floor + annotations); all canonicalization/interpretation happens in the differ (`reconstruct.py` + `canonicalize.py` + `diff.py`), keyed on the model's `build_system` tag (for noise) and each action's mnemonic (for grouping). See `docs/DESIGN-action-based-ir.md`. -- **Parity has two stages**, both reported by one diff run ("steps" below = the - numbered procedure; these are the *stages* of what's checked): +- **Parity has two stages**, both reported by one diff run: - **Compile parity** — every TU compiled with equivalent flags/defines/ includes (project-wide TU-set), and the external link-dependency closure matches. @@ -58,8 +57,8 @@ Bazel aquery jsonproto ──extract_bazel.py──┘ ▲ the TUs. Done when both stages converge. The comparison is **asymmetric**: every - correctness-relevant CMake flag must be present on the Bazel side; extra Bazel - flags are tolerated. No artifact/symbol diff or test execution yet. + correctness-relevant reference flag must be present on the Bazel side; extra + Bazel flags are tolerated. No artifact/symbol diff or test execution yet. ### How targets are matched @@ -86,14 +85,13 @@ errors by design. ## The migration config: `any2bazel.json` Lives at the **migrated project's repo root**, committed alongside the BUILD -files as the durable record of migration decisions. The filename still carries -the old project name; it applies to the CMake→Bazel path (the only frontend that -reads it today): +files as the durable record of migration decisions. Each reference guide notes +whether its frontend reads it: ```json { "bazel_args": ["--config=macos", "--copt=-fno-exceptions"], - "target_map": { "some_cmake_exe": ":some_bazel_exe" }, + "target_map": { "some_reference_exe": ":some_bazel_exe" }, "dep_map": { "Catch2Main": "catch2_main" }, "exclude_targets": ["benchmark", "some_tool"], "include_tests": false, @@ -115,16 +113,16 @@ reads it today): Fields: - **`bazel_args`** — the extra args aquery must run with to mirror the real - build (see step 4). Recorded so the comparison is reproducible; read by the - operator, not the diff. -- **`target_map`** — `cmake_name → bazel_name` for intentionally renamed + build (see [Extract the Bazel action graph](#extract-bazel-graph)). Recorded so + the comparison is reproducible; read by the operator, not the diff. +- **`target_map`** — `reference_name → bazel_name` for intentionally renamed **executables**. Bazel targets are keyed by full label, so the value is usually `:name` (e.g. `"bssl": ":bssl"`). Libraries need no mapping. -- **`dep_map`** — `cmake_dep_name → bazel_dep_name` for an **external link - dep** spelled differently per build (CMake's archive basename `Catch2Main` - vs Bazel's `catch2_main`, or `OpenSSL::SSL` vs `ssl`). An explicit, recorded - rename — not a fuzzy match — so a residual `missing_dep` is a genuine gap. -- **`exclude_targets`** — CMake target names dropped entirely from the diff: +- **`dep_map`** — `reference_dep_name → bazel_dep_name` for an **external link + dep** spelled differently per build (e.g. `Catch2Main` vs `catch2_main`). An + explicit, recorded rename — not a fuzzy match — so a residual `missing_dep` is + a genuine gap. +- **`exclude_targets`** — reference target names dropped entirely from the diff: third-party/vendored code Bazel pulls as an external module, or tooling out of scope. The **only** lever for `missing_tu`/`missing_target` on whole subtrees. Excluded targets still appear under `excluded.config_excluded`. @@ -133,117 +131,116 @@ Fields: tests enabled and the **same** test scope (symmetric configure + aquery); turning it on against a tests-off extraction fabricates findings. When on, test sources are compared as their own project-wide TU-set union (like - libraries) and a coarse test-binary count check runs. See step 8. + libraries) and a coarse test-binary count check runs. See + [Diff tests](#diff-tests). - **`ignore.{defines,flags,flags_prefixes}`** — reviewer-approved compile flag/define differences. `flags`/`defines` match exact tokens; `flags_prefixes` by prefix. - **`ignore.{link_flags,link_flags_prefixes}`** — reviewer-approved LINK flag differences (per-executable link-flag diff, same asymmetric-subset policy as - compile flags). Common case: CMake repeats compile/codegen flags - (`-fvisibility=hidden`, `-fno-common`) on the link line where they're benign, - while Bazel doesn't. + compile flags). Common case: the reference repeats compile/codegen flags on + the link line where they're benign, while Bazel doesn't. - **`ignore.include_map` / `ignore.include_prefixes`** — for an include root - spelled differently per side (a dep in-tree under CMake, external under Bazel). - **Prefer `include_map`**: it rewrites both sides' spellings to a canonical - token and still verifies presence (several `from`s may map to one `to`; longest - wins). `include_prefixes` just deletes the path (a blind spot) — use only when - there's no counterpart to map to. Search **order** is not enforced (presence - only) — see `docs/FUTURE-include-order-collision-check.md`. + spelled differently per side (e.g. a dep in-tree in the reference, external + under Bazel). **Prefer `include_map`**: it rewrites both sides' spellings to a + canonical token and still verifies presence (several `from`s may map to one + `to`; longest wins). `include_prefixes` just deletes the path (a blind spot) — + use only when there's no counterpart to map to. Search **order** is not + enforced (presence only) — see `docs/FUTURE-include-order-collision-check.md`. The `ignore` and `target_map`/`exclude_targets` lists are applied at **diff time** to **both sides**, so you can tune them and re-diff without re-running -cmake/bazel. +the reference build or bazel. -## Frontends and maturity - -All frontends extract into one shared **action-based model**; a language/ -mnemonic-aware differ compares each against a Bazel `aquery` model. - -| Frontend | Reference source | Diffs | Status | -|----------|------------------|-------|--------| -| **CMake** | File API codemodel-v2 | C/C++ compile + link parity | **Mature** — the validated path, detailed below | -| **Maven** | forked `javac` argfiles | Java source-set parity | **Early** — argv-floor only | -| **VSCode / npm** | esbuild/tsc/`child_process` instrumentation | standalone TS emit check | **Experimental** — not wired into the main loop | - -**Trust and detail the CMake path.** The numbered procedure below is the -CMake→Bazel loop. The Maven and npm frontends share the model and differ but are -newer captures; see *Other frontends* at the end for how they differ. - -## Scope (MVP — check before running) - -Supported: static/shared/object libraries and executables. Compile-parity stage: -plain C/C++ sources, compile flags, defines, include presence; external deps as -abstract identities. Link-consistency stage: per-executable/shared-lib link -flags. Tests are opt-in (`include_tests`) and get the compile-parity stage only -(procedure step 8). - -**NOT yet supported — stop and tell the user if the project has these:** -- Custom commands / generated code (`configure_file`, protoc, `add_custom_command`) -- Per-test-binary identity alignment, and include search **order** (presence - only) — both have planned follow-ups -- Packaging / install rules -- Automatic external-dependency resolution (find_package → bzlmod) -- **Maven**: coordinate-identity deps (`group:artifact:version`, scope) and Java - flag canonicalization — argv-floor only. -- **VSCode / npm**: not integrated into this parity loop (`diff_ts.py` is a - standalone TS emit check only). - -## Migration loop steps +## Migration process > **Script paths vs. project paths.** The `scripts/…` paths below are relative > to **this skill's own directory** (where this `SKILL.md` lives) — NOT the > project being migrated. When running inside a target repo, invoke them by > absolute path, e.g. -> `python3 "$SKILL_DIR/scripts/extract_cmake.py" …` where `$SKILL_DIR` is this +> `python3 "$SKILL_DIR/scripts/extract_bazel.py" …` where `$SKILL_DIR` is this > skill's install location (e.g. `~/.claude/skills/any2bazel`). The artifacts > you *produce* — `model.*.json`, `aquery.json`, `diff.json`, the generated > `BUILD.bazel`/`MODULE.bazel`, and `any2bazel.json` — live in or beside the > target repo. > -> **Working directory:** run `cmake` and `bazel` from the **target repo root** -> (the Bazel workspace). Only the `$SKILL_DIR/scripts/*.py` helpers live -> elsewhere. +> **Working directory:** run the reference build and `bazel` from the **target +> repo root** (the Bazel workspace). Only the `$SKILL_DIR/scripts/*.py` helpers +> live elsewhere. > -> **`` placeholder:** the project's source/workspace root — normally -> the same path as ``. It MUST be **identical** in the `extract_cmake.py` -> and `extract_bazel.py` calls: both key translation units by their path -> relative to ``, so a mismatch makes every source look -> missing/extra and the diff becomes meaningless. - -### 1. Confirm scope -Inspect `CMakeLists.txt`. If you find custom commands, codegen, or -`find_package` of non-system libs, surface them and confirm before proceeding. - -### 2. Extract the CMake reference model -Single CMake pass emits both the File API reply and a `--trace` (the latter is -the only place `configure_file()` outputs are visible — they leave no node in -the build graph). The optional 4th arg feeds the trace to the extractor, which -records configure-time generated files in `configured_files`. +> **`` placeholder:** the project's source/workspace root. It MUST be +> **identical** in the reference extractor (`extract_.py`) and +> `extract_bazel.py` calls: both key translation units by their path relative +> to ``, so a mismatch makes every source look missing/extra and the +> diff becomes meaningless. + +### Prepare for migration + +#### Identify the reference build system {#identify-reference} + +All frontends extract into one shared **action-based model**; a language/ +mnemonic-aware differ compares each against a Bazel `aquery` model. Each +frontend has a reference guide with the context unique to that build system. +Match the marker files at the repo root against the *Detect via* column of + the table below. +- If more than one matches (e.g. a CMake project with a `package.json` for + tooling), ask the user which build is the reference. +- If none match, stop and offer to use the current project as a case study + to add support for a new build system frontend. + +| Frontend | Detect via | Reference source | Diffs | Status | Guide | +|----------|------------|------------------|-------|--------|-------| +| **CMake** | `CMakeLists.txt` | File API codemodel-v2 | C/C++ compile + link parity | **Mature** — the validated path | [REFERENCE-cmake.md](docs/REFERENCE-cmake.md) | +| **Maven** | `pom.xml` | forked `javac` argfiles | Java source-set parity | **Early** — argv-floor only | [REFERENCE-maven.md](docs/REFERENCE-maven.md) | +| **VSCode / npm** | `package.json` driving gulp/esbuild/tsc | esbuild/tsc/`child_process` instrumentation | standalone TS emit check | **Experimental** — not wired into the main loop | [REFERENCE-npm.md](docs/REFERENCE-npm.md) | + +**Read the matching reference guide in full before continuing**, and don't +read the others. Later steps defer to it for anything specific to the +reference build. + +#### Confirm scope {#confirm-scope} + +Inspect the reference build for anything unsupported. Surface them and confirm +with the user before proceeding. + +**NOT yet supported — warn the user if the project has these:** +- Custom commands / generated code +- Per-test-binary identity alignment, and include search **order** (presence + only) — both have planned follow-ups +- Packaging / install rules +- Automatic external-dependency resolution + +Each build system specific reference guide lists additional limits that you must +look for. + +#### Extract the reference model {#extract-reference-model} + +Run the reference build's extraction exactly as its reference guide describes. +Every extractor produces the same canonical model: ```bash -mkdir -p /.cmake/api/v1/query -touch /.cmake/api/v1/query/codemodel-v2 -cmake -S -B -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \ - --trace-expand --trace-format=json-v1 2> /trace.jsonl # + project flags -python3 scripts/extract_cmake.py model.cmake.json /trace.jsonl +python3 scripts/extract_.py <…> model..json ``` -> Configure-time generated compile inputs (e.g. CMake's `configure_file` output -> `zconf.h`) are recorded but **not yet diffed** — the Bazel-side extraction and -> the content differ are TODO (see `docs/TODO-configure-time-generation.md`). -> This is distinct from build-time codegen (genrules), which is also unmodeled. +Note the platform and options the reference build was configured with; the +Bazel side must mirror them. + +### Create initial BUILD.bazel files -### 3. Generate initial BUILD.bazel files *(LLM step)* -Read `model.cmake.json`. For each production target emit a `cc_library` / +Read `model..json`. For each production target emit a `cc_library` / `cc_binary` with `srcs`, `hdrs`, `copts`, `defines`, `includes`, `deps`. Library -grouping need not match CMake (TU-set comparison is grouping-agnostic), but keep -**executable** names aligned or add a `target_map` entry. Write `MODULE.bazel` -as needed — before picking rulesets or pinning versions, read +grouping need not match the reference (TU-set comparison is +grouping-agnostic), but keep **executable** names aligned or add a `target_map` +entry. Write `MODULE.bazel` as needed — before picking rulesets or pinning +versions, read [docs/BAZEL-RULES.md](docs/BAZEL-RULES.md) (which rulesets have been exercised here, why some were hand-written instead, and why a version must be resolved rather than recalled). Put `common --check_direct_dependencies=error` in the generated `.bazelrc` so a declared version that MVS overrides fails the build instead of being a warning nobody reads. -### 4. Extract the Bazel side +### Main migration loop {#main-migration-loop} + +#### Extract the Bazel action graph {#extract-bazel-graph} + > **Critical: aquery must be invoked the way the project is actually built.** > A bare `bazel aquery` omits config-gated and top-level flags and will > manufacture hundreds of false discrepancies. Mirror the real build: @@ -255,8 +252,9 @@ instead of being a warning nobody reads. > -fno-rtti` to be set at the top level, not in libraries). The tool cannot > infer these — get them from the project's build instructions and pass them > through, or record genuinely-irreducible differences in `any2bazel.json`. -> - Use the **same platform/options** as the CMake configure in step 2, or the -> two sides aren't comparable. +> - Use the **same platform/options** as the reference build in +> [Extract the reference model](#extract-reference-model), or the two sides +> aren't comparable. > - **Always** call aquery with `--features=-compiler_param_file` and > `--features=-linker_param_file` to disable use of param files. Param files > are never generated during `aquery` calls, so when they're in use they'll @@ -269,19 +267,25 @@ bazel aquery 'mnemonic("CppCompile|ObjcCompile|CppLink|CppArchive", //...)' \ --output=jsonproto > aquery.json python3 scripts/extract_bazel.py aquery.json model.bazel.json ``` +The mnemonics above cover C/C++. If the reference guide specifies a different +mnemonic set, use that instead. + If analysis fails, fix that first before trusting the diff. If a flag differs only because of a build-convention gap (e.g. `-std=gnu++17` vs `-std=c++17`, GNU-extensions on/off), that's a judgment call for `any2bazel.json`, not a BUILD-file bug. -### 5. Diff +#### Diff against reference build {#diff} + ```bash -python3 scripts/diff.py model.cmake.json model.bazel.json \ +python3 scripts/diff.py model..json model.bazel.json \ /any2bazel.json > diff.json # 3rd arg optional ``` `diff.json` has `converged` (⇔ zero `error` discrepancies), a `discrepancies` worklist (each with `kind`, `severity`, `target`, `tu`, `cmake_only`, `bazel_only`), and an `excluded` map of non-participating targets by role. +`cmake_only` is a historical name: for every frontend it means "present in the +reference, absent in Bazel". Synthetic target names ``, `` (and `` when `include_tests`) denote the unioned library-TU, external-dep, and test-TU comparisons. @@ -294,11 +298,12 @@ are easiest to reason about once the TUs underneath agree. `error`-severity kinds (above) block convergence and are what you fix. The diff also emits **WARN-only** kinds that don't block `converged` — `extra_tu` / -`extra_target` (compiled/built on the Bazel side but not CMake) and +`extra_target` (compiled/built on the Bazel side but not the reference) and `extra_test_tu` — note them but they need no action unless they point to something you didn't intend to add. -### 6. Triage +#### Triage findings {#triage} + ```bash python3 scripts/triage.py diff.json # grouped summary python3 scripts/triage.py diff.json --kind flags_diff # drill into one kind @@ -312,7 +317,15 @@ is systematic — one fix (a copt, an include, a `target_map`/`ignore`/`copts` entry) clears it in bulk; a value on one TU is local. Always triage before hand-reading the worklist. -### 7. Fix *(LLM step)*, then loop +#### Fix findings {#fix} + +> [!IMPORTANT] +> This is the step where you may modify and create build files. + +All per-iteration judgment goes into the generated `BUILD.bazel`, `MODULE.bazel`, `.bzl`, +and `any2bazel.json` (reviewed). `$SKILL_DIR/scripts/` are deterministic and must +**not** be edited per-run. + For each `error`, decide: real defect → fix the BUILD file; accepted difference → add to `any2bazel.json` `ignore` (only for warning/cosmetic flags, **never** correctness flags). @@ -322,87 +335,47 @@ correctness flags). | `missing_target` | add the missing `cc_binary`, or `target_map` a renamed exe, or `exclude_targets` if out of scope | | `missing_tu` | add the source to some library's `srcs`, or `exclude_targets` if it's a vendored/out-of-scope subtree | | `defines_diff` | add each `cmake_only` define to `defines`, or `ignore.defines` it | -| `includes_diff` | add the missing CMake include root to `includes`; for a dep whose root is spelled differently each side, `ignore.include_map` it (preferred) or `ignore.include_prefixes` it | +| `includes_diff` | add the missing reference include root to `includes`; for a dep whose root is spelled differently each side, `ignore.include_map` it (preferred) or `ignore.include_prefixes` it | | `flags_diff` | add each `cmake_only` flag to `copts`, or `ignore.flags` it | -| `link_flags_diff`| add each `cmake_only` flag to the target's `linkopts`, or `ignore.link_flags` it if benign (e.g. a compile flag CMake repeats at link) | +| `link_flags_diff`| add each `cmake_only` flag to the target's `linkopts`, or `ignore.link_flags` it if benign (e.g. a compile flag the reference repeats at link) | | `missing_dep` | add the missing external/system dep to the target's `deps`/`linkopts`; if it's just a name spelled differently per build (`Catch2Main` vs `catch2_main`, `OpenSSL::SSL` vs `ssl`), add a `dep_map` entry. External deps are captured from both `-l` flags and archive-file inputs (e.g. `external/catch2+/libcatch2_main.a`) | | `missing_test_tu`| (tests on) add the test source to a `cc_test`, or `exclude_targets` if out of scope | | `test_binary_count` | (tests on, warning) differing number of test executables — investigate which side has the extra/missing binary | -Then re-diff: if you edited `BUILD.bazel`/`MODULE.bazel`, re-run from step 4 -(re-extract the Bazel side); if you only edited `any2bazel.json`, re-run from -step 5. Repeat until `converged: true`. Report remaining `warn` items and the -`excluded` roles. +Kinds specific to a reference build system are listed in its reference guide. + +#### Repeat + +If you edited `BUILD.bazel`/`MODULE.bazel`/`.bzl`, re-run from +[Extract the Bazel action graph](#extract-bazel-graph) (re-extract the Bazel +side); if you only edited `any2bazel.json`, re-run from +[Diff against reference build](#diff). Repeat until `converged: true`. Report +remaining `warn` items and the `excluded` roles. + +### (Optional) Diff tests {#diff-tests} -### 8. (Optional) Diff tests Once production parity is reached, opt into test diffing: -- Re-extract **both** sides with tests enabled and the **same** scope: CMake - configured without `-D..._BUILD_TESTING=OFF`; aquery over `//...` (not a - single target). Asymmetric scope fabricates findings. +- Re-extract **both** sides with tests enabled and the **same** scope: the + reference build with tests enabled as its reference guide describes; aquery + over `//...` (not a single target). Asymmetric scope fabricates findings. - Set `"include_tests": true` in `any2bazel.json`, re-extract both models - (steps 2 and 4) with the test-inclusive configure/aquery, then re-run the - diff/triage/fix loop (steps 5–7). + ([reference](#extract-reference-model) and [Bazel](#extract-bazel-graph)) + with the test-inclusive configure/aquery, then re-run the + [main migration loop](#main-migration-loop). - Test sources are compared as a project-wide TU-set union (grouping/naming agnostic); a `test_binary_count` warning flags differing numbers of test executables. Per-binary identity alignment is a later layer — for now, `missing_test_tu` tells you a test source isn't compiled on the Bazel side (e.g. an un-ported test binary). +- Run the main migration loop until all tests converge. + +### Report results -### 9. Report Summarize: production targets reconciled, rounds taken, suppressions recorded in `any2bazel.json` (with rationale), excluded roles (dashboard/codegen) for human follow-up, and — if `include_tests` was on — test-source parity and any test-binary count gap. -## What you edit - -`$SKILL_DIR/scripts/` are deterministic and must **not** be edited per-run. All -per-iteration judgment goes into the generated `BUILD.bazel`/`MODULE.bazel` and -`any2bazel.json` (reviewed). - -## Other frontends - -The Maven and VSCode/npm frontends share the action model and differ but are -newer captures — treat their output as exploratory, and don't apply the CMake -`any2bazel.json` machinery to them (they don't read it). - -### Maven → Bazel (early) -Maven has no action graph; the reference is the **forked `javac` argument -file**. Java compiles a whole source set at once, so it's compared as a -project-wide union of `.java` sources (grouping/naming agnostic), analogous to -how libraries are compared. argv-floor only — coordinate deps aren't extracted. -```bash -mvn clean compile -Dmaven.compiler.fork=true # writes /target/*arguments -python3 scripts/extract_maven.py model.maven.json -bazel aquery 'mnemonic("Javac", //...)' --output=jsonproto > aquery.json -python3 scripts/extract_bazel.py aquery.json model.bazel.json -python3 scripts/diff.py model.maven.json model.bazel.json > diff.json -``` -Diff kinds: `missing_java_src` / `extra_java_src` (a `.java` compiled on only -one side). - -### VSCode / npm → Bazel (experimental) -An npm/gulp/esbuild build is instrumented in-process by a Node preload (hooks -esbuild, the TS language service, and `child_process`), emitting one NDJSON -record per action. `diff_ts.py` does a **standalone** TS source→emit check; it -is not part of the main diff loop. -```bash -NODE_OPTIONS="--import file://$SKILL_DIR/scripts/npm_instrument/preload.mjs" \ -VSCODE_EMIT_BUILD_IR=$PWD/actions.ndjson \ - npm run -python3 scripts/extract_npm.py actions.ndjson model.npm.json -python3 scripts/diff_ts.py model.npm.json model.bazel.json # standalone check -``` -The preload must be run from a copy **inside the target repo** -(`cp scripts/npm_instrument/*.mjs /.instr/`): `typescript-shim.mjs` -resolves `typescript` relative to its own location, so running it from this -repo fails with `ERR_MODULE_NOT_FOUND`. - -Worked example, with byte-parity results and the failure modes a bespoke JS -build brings (no action graph to extract, traversal-order-dependent output, -`node_modules` as both toolchain and foreign Bazel package): -[docs/CASE-vscode-migration.md](docs/CASE-vscode-migration.md). - ## Updating this skill To change this skill or its scripts, read diff --git a/docs/REFERENCE-cmake.md b/docs/REFERENCE-cmake.md new file mode 100644 index 0000000..49ed665 --- /dev/null +++ b/docs/REFERENCE-cmake.md @@ -0,0 +1,62 @@ +# Reference build: CMake + +Supporting context for migrating **from** CMake. The migration process itself +lives in [SKILL.md](../SKILL.md); this file only covers what is unique to CMake +as the reference build. + +**Status: mature** — the validated path. + +## Detecting it + +`CMakeLists.txt` files throughout the repo. + +## Scope traps + +When [confirming scope](../SKILL.md#confirm-scope), inspect `CMakeLists.txt` +for these and surface them before proceeding: +- Generated code: `configure_file`, `add_custom_command`, protoc or other + codegen steps. +- `find_package` of non-system libraries. There is no automatic + `find_package` → bzlmod resolution; each one must be mapped by hand. + +## Extracting the reference model + +The CMake side is the **File API codemodel-v2**, not `compile_commands.json` +(which lacks link info). It exposes both compile and link actions. + +A single CMake pass emits both the File API reply and a `--trace` (the latter +is the only place `configure_file()` outputs are visible — they leave no node in +the build graph). The optional 4th arg feeds the trace to the extractor, which +records configure-time generated files in `configured_files`. +```bash +mkdir -p /.cmake/api/v1/query +touch /.cmake/api/v1/query/codemodel-v2 +cmake -S -B -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \ + --trace-expand --trace-format=json-v1 2> /trace.jsonl # + project flags +python3 "$SKILL_DIR/scripts/extract_cmake.py" model.cmake.json /trace.jsonl +``` +> Configure-time generated compile inputs (e.g. CMake's `configure_file` output +> `zconf.h`) are recorded but **not yet diffed** — the Bazel-side extraction and +> the content differ are TODO (see +> [TODO-configure-time-generation.md](TODO-configure-time-generation.md)). This +> is distinct from build-time codegen (genrules), which is also unmodeled. + +The project flags passed to `cmake` define the platform/options the Bazel +aquery must mirror. + +## CMake-specific comparison notes + +- **Target names** in `target_map` and `exclude_targets` keys are CMake target + names (e.g. `"bssl": ":bssl"`). +- **`dep_map`**: CMake spells external link deps as an archive basename + (`Catch2Main`) or an imported target (`OpenSSL::SSL`), while Bazel spells them + `catch2_main` / `ssl`. +- **Link flags**: CMake repeats compile/codegen flags (`-fvisibility=hidden`, + `-fno-common`) on the link line, where they're benign, while Bazel doesn't. + This is the common `ignore.link_flags` case. + +## Tests + +For [test diffing](../SKILL.md#diff-tests), configure CMake **without** +`-D..._BUILD_TESTING=OFF` (or the project's equivalent option) so the +reference model includes test targets. diff --git a/docs/REFERENCE-maven.md b/docs/REFERENCE-maven.md new file mode 100644 index 0000000..c324fe2 --- /dev/null +++ b/docs/REFERENCE-maven.md @@ -0,0 +1,49 @@ +# Reference build: Maven + +Supporting context for migrating **from** Maven. The migration process itself +lives in [SKILL.md](../SKILL.md); this file only covers what is unique to Maven +as the reference build. + +**Status: early** — argv-floor only. Treat output as exploratory. + +## Detecting it + +A `pom.xml` at the repo root. + +## Scope traps + +Not yet supported: +- Coordinate-identity deps (`group:artifact:version`, scope) — not extracted. +- Java flag canonicalization. + +The C/C++ `any2bazel.json` levers and the fix table in SKILL.md don't apply to +Java source sets. + +## Extracting the reference model + +Maven has no action graph; the reference is the **forked `javac` argument +file**. +```bash +mvn clean compile -Dmaven.compiler.fork=true # writes /target/*arguments +python3 "$SKILL_DIR/scripts/extract_maven.py" model.maven.json +``` + +## Extracting the Bazel action graph + +Use the `Javac` mnemonic in place of the C/C++ mnemonic set: +```bash +bazel aquery 'mnemonic("Javac", //...)' \ + --features=-compiler_param_file --features=-linker_param_file \ + --output=jsonproto > aquery.json +``` + +## Comparison and diff kinds + +Java compiles a whole source set at once, so it's compared as a project-wide +union of `.java` sources (grouping/naming agnostic), analogous to how libraries +are compared. + +| kind | meaning | +|------|---------| +| `missing_java_src` | a `.java` compiled by Maven but not Bazel — add it to a `java_library`'s `srcs` | +| `extra_java_src` | a `.java` compiled by Bazel but not Maven | diff --git a/docs/REFERENCE-npm.md b/docs/REFERENCE-npm.md new file mode 100644 index 0000000..4df6ff2 --- /dev/null +++ b/docs/REFERENCE-npm.md @@ -0,0 +1,46 @@ +# Reference build: VSCode / npm + +Supporting context for migrating **from** an npm/gulp/esbuild build. The +migration process itself lives in [SKILL.md](../SKILL.md); this file only +covers what is unique to npm as the reference build. + +**Status: experimental** — not wired into the main parity loop. + +## Detecting it + +A `package.json` at the repo root whose build scripts drive gulp, esbuild, or +`tsc`. + +## Scope traps + +This frontend is **not integrated into the parity loop**: there is no +`diff.py`/`triage.py` pass. `diff_ts.py` is a standalone TS source→emit check +that replaces the diff and triage steps. + +## Extracting the reference model + +An npm/gulp/esbuild build is instrumented in-process by a Node preload (hooks +esbuild, the TS language service, and `child_process`), emitting one NDJSON +record per action. + +The preload must be run from a copy **inside the target repo** +(`cp "$SKILL_DIR"/scripts/npm_instrument/*.mjs /.instr/`): +`typescript-shim.mjs` resolves `typescript` relative to its own location, so +running it from the skill directory fails with `ERR_MODULE_NOT_FOUND`. +```bash +NODE_OPTIONS="--import file:///.instr/preload.mjs" \ +VSCODE_EMIT_BUILD_IR=$PWD/actions.ndjson \ + npm run +python3 "$SKILL_DIR/scripts/extract_npm.py" actions.ndjson model.npm.json +``` + +## Comparison + +```bash +python3 "$SKILL_DIR/scripts/diff_ts.py" model.npm.json model.bazel.json # standalone check +``` + +Worked example, with byte-parity results and the failure modes a bespoke JS +build brings (no action graph to extract, traversal-order-dependent output, +`node_modules` as both toolchain and foreign Bazel package): +[CASE-vscode-migration.md](CASE-vscode-migration.md).