Repository navigation
Name the three configure-step strategies and their costs - #49
Open
anakanemison wants to merge 1 commit into
Open
anakanemison wants to merge 1 commit into
anakanemison wants to merge 1 commit into
Conversation
SKILL.md lists configure_file among the things to warn the user about before proceeding, without saying what to do instead. Unattended agents do not stop, so they improvise, and the skill gave them nothing to improvise against: the choice of how to replace CMake's configure step was made silently and its cost was invisible until someone rebuilt from a clean checkout. Measured in any2bazel-lab, three strategies on larger projects: - consume CMake's build directory as Bazel sources (a MySQL Server migration, which cited the skill's scope boundary as the reason): the Bazel build does not rebuild from the repository at all; - implement configure_file as a Starlark rule (the unguided MySQL Server migration): rebuilds, and reproduced 15 of 15 comparable generated files byte-identically; - check in the probe results as static headers (GDAL, CMake): builds, but the configuration is no longer derived. On CMake itself two runs of the same prompt checked in all 28 configure-time headers; one matched the reference byte for byte, the other differed in three, including a curl_config.h that disables HSTS and a libarchive config.h that falls back to built-in hashes despite OpenSSL being present. Nothing in the diff can see this. The scope list, in SKILL.md and in docs/REFERENCE-cmake.md's scope traps, now covers build-time codegen only, and points configure_file to a new section of docs/REFERENCE-cmake.md beside the build type: prefer a Starlark configure_file rule; checking in a probe result is acceptable for one platform if the file says so and it is verified file by file against the configure output it came from; never point Bazel at CMake's build directory. Whether build-time codegen should also move from "stop" to guidance is a separate change, later in this series. In the lab's later runs with this change, all four agents that met a configure step derived it (a Starlark rule or captured substitutions), and on CMake's sources the three headers that had differed came out byte-identical in all three runs. Lab-Proposal: P20, P21, P5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 14 of 23 of a stacked series (tracking issue: #35). Based on
U13-build-type-to-flags, the previous part: review and merge that first.Kind: guidance. Lab proposal: P20, P21, P5.
What the migration runs showed
Two cmake migrations at the same commit checked in all 28 probed headers; in one of them three headers differed from the reference, one of them disabling HSTS in curl. With this guidance, the three headers came out byte-identical in every cmake run (runs 086, 087, 094).
The change
SKILL.md lists configure_file among the things to warn the user about before
proceeding, without saying what to do instead. Unattended agents do not
stop, so they improvise, and the skill gave
them nothing to improvise against: the choice of how to replace CMake's
configure step was made silently and its cost was invisible until someone
rebuilt from a clean checkout.
Measured in any2bazel-lab, three strategies on larger projects:
migration, which cited the skill's scope boundary as the reason): the
Bazel build does not rebuild from the repository at all;
migration): rebuilds, and reproduced 15 of 15 comparable generated files
byte-identically;
the configuration is no longer derived. On CMake itself two runs of the
same prompt checked in all 28 configure-time headers; one matched the
reference byte for byte, the other differed in three, including a
curl_config.h that disables HSTS and a libarchive config.h that falls
back to built-in hashes despite OpenSSL being present. Nothing in the
diff can see this.
The scope list, in SKILL.md and in docs/REFERENCE-cmake.md's scope traps,
now covers build-time codegen only, and points configure_file to a new
section of docs/REFERENCE-cmake.md beside the build type: prefer a Starlark
configure_file rule; checking in a probe result is acceptable for one platform if the file
says so and it is verified file by file against the configure output it
came from; never point Bazel at CMake's build directory. Whether build-time
codegen should also move from "stop" to guidance is a separate change, later
in this series.
In the lab's later runs with this change, all four agents that met a
configure step derived it (a Starlark rule or captured substitutions), and
on CMake's sources the three headers that had differed came out
byte-identical in all three runs.
Testing
pytest tests/at this commit: 196 passed (130 onmain).🤖 Generated with Claude Code