Skip to content

Name the three configure-step strategies and their costs - #49

Open
anakanemison wants to merge 1 commit into
mschaller/u13-build-type-to-flagsfrom
mschaller/u14-configure-step-strategies
Open

anakanemison wants to merge 1 commit into
mschaller/u13-build-type-to-flagsfrom
mschaller/u14-configure-step-strategies

Conversation

@anakanemison

Copy link
Copy Markdown

Part 14 of 23 of a stacked series (tracking issue: #35). Based on U13-build-type-to-flags, the previous part: review and merge that first.

Kind: guidance. Lab proposal: P20, P21, P5.

What the migration runs showed

Two cmake migrations at the same commit checked in all 28 probed headers; in one of them three headers differed from the reference, one of them disabling HSTS in curl. With this guidance, the three headers came out byte-identical in every cmake run (runs 086, 087, 094).

The change

SKILL.md lists configure_file among the things to warn the user about before
proceeding, without saying what to do instead. Unattended agents do not
stop, so they improvise, and the skill gave
them nothing to improvise against: the choice of how to replace CMake's
configure step was made silently and its cost was invisible until someone
rebuilt from a clean checkout.

Measured in any2bazel-lab, three strategies on larger projects:

  • consume CMake's build directory as Bazel sources (a MySQL Server
    migration, which cited the skill's scope boundary as the reason): the
    Bazel build does not rebuild from the repository at all;
  • implement configure_file as a Starlark rule (the unguided MySQL Server
    migration): rebuilds, and reproduced 15 of 15 comparable generated files
    byte-identically;
  • check in the probe results as static headers (GDAL, CMake): builds, but
    the configuration is no longer derived. On CMake itself two runs of the
    same prompt checked in all 28 configure-time headers; one matched the
    reference byte for byte, the other differed in three, including a
    curl_config.h that disables HSTS and a libarchive config.h that falls
    back to built-in hashes despite OpenSSL being present. Nothing in the
    diff can see this.

The scope list, in SKILL.md and in docs/REFERENCE-cmake.md's scope traps,
now covers build-time codegen only, and points configure_file to a new
section of docs/REFERENCE-cmake.md beside the build type: prefer a Starlark
configure_file rule; checking in a probe result is acceptable for one platform if the file
says so and it is verified file by file against the configure output it
came from; never point Bazel at CMake's build directory. Whether build-time
codegen should also move from "stop" to guidance is a separate change, later
in this series.

In the lab's later runs with this change, all four agents that met a
configure step derived it (a Starlark rule or captured substitutions), and
on CMake's sources the three headers that had differed came out
byte-identical in all three runs.

Testing

pytest tests/ at this commit: 196 passed (130 on main).

🤖 Generated with Claude Code

SKILL.md lists configure_file among the things to warn the user about before
proceeding, without saying what to do instead. Unattended agents do not
stop, so they improvise, and the skill gave
them nothing to improvise against: the choice of how to replace CMake's
configure step was made silently and its cost was invisible until someone
rebuilt from a clean checkout.

Measured in any2bazel-lab, three strategies on larger projects:
- consume CMake's build directory as Bazel sources (a MySQL Server
  migration, which cited the skill's scope boundary as the reason): the
  Bazel build does not rebuild from the repository at all;
- implement configure_file as a Starlark rule (the unguided MySQL Server
  migration): rebuilds, and reproduced 15 of 15 comparable generated files
  byte-identically;
- check in the probe results as static headers (GDAL, CMake): builds, but
  the configuration is no longer derived. On CMake itself two runs of the
  same prompt checked in all 28 configure-time headers; one matched the
  reference byte for byte, the other differed in three, including a
  curl_config.h that disables HSTS and a libarchive config.h that falls
  back to built-in hashes despite OpenSSL being present. Nothing in the
  diff can see this.

The scope list, in SKILL.md and in docs/REFERENCE-cmake.md's scope traps,
now covers build-time codegen only, and points configure_file to a new
section of docs/REFERENCE-cmake.md beside the build type: prefer a Starlark
configure_file rule; checking in a probe result is acceptable for one platform if the file
says so and it is verified file by file against the configure output it
came from; never point Bazel at CMake's build directory. Whether build-time
codegen should also move from "stop" to guidance is a separate change, later
in this series.

In the lab's later runs with this change, all four agents that met a
configure step derived it (a Starlark rule or captured substitutions), and
on CMake's sources the three headers that had differed came out
byte-identical in all three runs.

Lab-Proposal: P20, P21, P5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant