Preserve Python patches across uv lockfiles - #238
Merged
Mikola Lysenko (mikolalysenko) merged 3 commits intoSep 15, 2026
Merged
Conversation
Keep hosted and vendored Python patches installed across native uv, PEP 751, requirements, and script lockfiles. Preserve artifact hashes, paired metadata, and conditional versions; refuse ambiguous rewrites. Assisted-by: Codex:gpt-6-astra
Publish real CLI and uv installation evidence across fourteen releases, including frozen, locked, and ordinary installs in both patch modes. Record unavailable formats and legacy boundaries separately from passes. Assisted-by: Codex:gpt-6-astra
Tanmay Singla (Tanmay182003)
approved these changes
Sep 14, 2026
Mikola Lysenko (mikolalysenko)
enabled auto-merge (squash)
September 15, 2026 14:01
Resolve the conflict in patch/redirect/mod.rs: main (#237) patched the inline requirements.txt rewriter to strip trailing inline comments so the appended --hash is not swallowed into the comment; this branch had already replaced that body with redirect::requirements::rewrite, which splits the comment off outside quotes and re-appends it after the hash. Keep the delegation and update #237's test to expect the preserved comment (hash still active, user note kept). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko)
deleted the
codex/uv-patch-compatibility
branch
September 15, 2026 14:10
Mikola Lysenko (mikolalysenko)
added a commit
that referenced
this pull request
Sep 15, 2026
…for the unwired-revert guard The unwired-revert guard added after #238 protects a `repair`-reconstructed pypi ledger entry (no wiring records) from being deleted while a Python project file still resolves through the vendored wheel. Four gaps remained: - `keep_artifact` (`rollback/remove --preserve-state`) was still guarded, so a preserve-state revert of an unwired entry refused with `vendor_wiring_unknown_revert_blocked` although it would have deleted nothing. npm skips the guard there because the refusal exists only to protect the deletion; the pypi guard now does the same. - The lock enumeration failed OPEN: a `read_dir` error on the project root was read as "no Python locks here", `python_lock_paths` silently dropped any lock whose (followed) metadata failed, and the static probe list lacked uv.lock and pylock.toml. On a 0311 (execute-only) root, a listing-denied ACL or a transient EMFILE, `vendor --revert` deleted the referenced wheel and dropped the entry while uv.lock still resolved through it. The guard now lists the root itself (lstat only, so a symlinked lock with an unreadable target is still probed and fails closed on the read), and an unlistable root refuses with its own clause. - Once the guard cleared, the entry was still dispatched by flavor, which made a true orphan unreclaimable forever: flavor `uv` with uv.lock gone failed "cannot read uv.lock", and flavor `None` (what `repair` stamps for requirements/poetry/pdm/pipenv reconstructions) was unknown to the dispatch. An empty-wiring entry that passes the guard now skips the dispatch (it has nothing to replay) and proceeds to the uuid-dir removal. An UNKNOWN non-None flavor still fails closed: a newer backend's files may reference the artifact from a place this guard does not probe. - The requirements planner writes vendored pins into `-r` includes, but the guard probed only the root requirements.txt. `requirements_include_names` (new, exported from the core crate for the CLI's repair/orphan sweep) walks the same include tree as the planner — FIFO-safe, `Err` when a reached include exists but cannot be read — and the guard probes every in-root include, failing closed on an unreadable tree. Verified against real uv 0.9.30 and 0.5.31 through the built CLI: the refusal on a 0311 root keeps the wheel and `uv sync --frozen --offline` still installs from it; after `uv remove` the orphan is reclaimed and the ledger entry dropped. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko)
added a commit
that referenced
this pull request
Sep 16, 2026
…ily (#239) * Harden uv lockfile patch preservation after #238 Fail-closed revert for unwired Python entries: a ledger entry with no wiring (the shape `repair` re-synthesizes when state.json is lost) used to route into the uv / python-lock revert, iterate zero records, report success, and let the caller delete the vendored wheel and drop the entry while uv.lock, the pylock, the script, or requirements.txt still resolved through it. `revert_pypi_opts` now refuses while any Python project file references the uuid dir, mirroring the npm-family guard. Inventory: a PEP 723 script lock or PEP 751 lock supplements the project's poetry.lock / requirements.txt pins instead of hiding them; uv.lock keeps its exclusive precedence. Discovery: `python_lock_paths` follows symlinks and skips one unreadable entry instead of dropping every lock. Line endings: toml_edit re-emits every newline as LF, so CRLF pylock, script-lock and pyproject rewrites (and document-restore reverts) now restore the input's CRLF convention. Ledger recovery compares the purl name in PEP 503 form like `lookup`. Hosted pyproject edits render uv's own layout — header-less `[tool]` / `[tool.uv]` parents and a `[tool.uv.sources]` table after `[project]` — instead of dotted keys above `[project]`. The PEP 723 script block keeps fully dotted keys: only those vanish when empty after a parse round trip, which the out-of-order multi-package document revert relies on. `redirect_uv_missing_sha256` fires once per dep, not once per lock file; `repair` prefers uv.lock over an exported pylock when stamping flavor. Backtest harness: retry the cold-offline root-build failure with network for any uv version (not just 0.2.37), and pin first, latest, and every observed behaviour-boundary release of each 0.x family. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Follow each uv `[[distribution]]` lock shape when redirecting The experimental `[[distribution]]` grammar (uv 0.1.x–0.2.34) went through three shapes, and the rewriter keyed everything on the table name: - string sources + sub-table artifacts (`[distribution.sdist]`, `[[distribution.wheel]]`) through 0.2.5; - string sources + inline artifacts (`sdist = {…}`, `wheels = [...]`) from 0.2.6 through 0.2.17; - inline-table sources (`source = { registry = … }`) from 0.2.18. Emitting the string source into a 0.2.18–0.2.34 lock made uv reject it ("data did not match any variant of untagged enum SourceWire") and silently ignore the lock: an ordinary `uv sync` still installed the patch through the pyproject source, but `--frozen` / `--locked` failed. Emitting `[[distribution.wheel]]` into a 0.2.6–0.2.17 lock left the binary with a direct URL and no wheel, which it tried to build as a source archive ("Unsupported archive type: …whl"). Decide the source shape from the entry's own `source` syntax and the artifact shape from its own artifact keys (falling back to any sibling entry), independently. Vendored native wiring stays refused for the whole `[[distribution]]` era: every shape records absolute file paths for local artifacts. Reword the refusal to say so (it claimed "uv 0.1", but 0.2.34 refuses too). Add scripts/probe-uv-boundaries.py — the bisection tool behind every boundary pinned in scripts/backtest-uv.py (lock grammar, source and artifact shape, lock revision, `uv export`, `uv lock --script`, PEP 751 compilation, bare local wheel paths in requirements) — and pin the 0.2.5/0.2.6 and 0.2.17/0.2.18 pairs in the matrix. Document the exact boundaries. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Vendor into uv 0.2.35/0.2.36 locks that predate [package.metadata] The first two `[[package]]`-grammar releases wrote no root `[package.metadata]`; the metadata step refused them with `pypi_uv_lock_package_missing` although there was nothing to repoint. Skip the requires-dist rewrite when the root unit has no metadata table at all (a lock that HAS metadata but lacks the entry is stale and still refuses), so vendored native wiring covers every `[[package]]` release. Verified with real `uv sync --frozen` / `--locked` / ordinary installs on 0.2.35, 0.2.36 and 0.2.37. Pin 0.2.36/0.2.37 as the metadata boundary in the matrix and document the exact grammar shapes and coverage. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Record the 41-release uv compatibility matrix for source 310b904 Regenerate results.json, binaries.json, and the results section of docs/testing/uv-compatibility.md from a full run of scripts/backtest-uv.py against the rebuilt CLI: 41 pinned uv releases (first and latest of every 0.x family plus every observed behaviour boundary), 583 installed-byte comparisons and 240 lock-preservation checks, zero mismatches and zero lock changes. Every non-zero exit is a documented uv boundary or the cold-offline root-build case retried with network. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * utils/fs: export read_regular_to_string and is_symlink helpers Shared FIFO-safe reader (the shape every ecosystem module re-declared privately) and an lstat-based symlink probe, exported so the CLI crate's raw read_to_string sites and the Python lock writers can use them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * uv vendor: classify [tool.uv] dev-dependencies as direct; repoint every metadata entry classify_dependency scanned project.dependencies, optional-dependencies and [dependency-groups] but not the legacy `[tool.uv] dev-dependencies` array, which uv still honours (0.12.15 warns but records it exactly like `dependency-groups.dev`, in the root unit's [package.metadata.requires-dev]). Such a target classified Transitive: wire_uv took the override branch, [package.metadata.requires-dev] kept `specifier = "==…"`, and after a "successful" vendored scan every uv >= 0.2.37 `uv sync --locked` / `uv lock --check` exited non-zero while a plain `uv sync` rewrote the lock (ledger drift) and on 0.2.37/0.4.30 reinstalled the PRISTINE wheel. rewrite_root_metadata_entries also repointed only the FIRST matching element per array (a `break` in the requires-dist loop and per group in requires-dev) although its doc promised ALL: a package declared in both dependencies and an optional-dependencies extra has two requires-dist entries (the extra's with `marker = "extra == '…'"`) and uv repoints both, so the stale second entry kept --locked red. Fix: extend `declared` with the [tool.uv] dev-dependencies string members (pep508_name); repoint every matching requires-dist element as its own edit/wiring record (revert already replays N records per package), and rebuild each requires-dev group line once with every matching element rewritten so a group naming the package twice never yields overlapping spans. Both new fixtures are byte-exact uv 0.12.15 output. wire_uv now also returns a Vec<VendorWarning> of wiring-time advisories, threaded into the vendor outcome's warnings by pypi.rs (the channel the next commit's override-branch advisory uses). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * uv vendor: advise that override-dependencies wiring needs uv >= 0.5.6 The module doc's "claim 8" said [tool.uv.sources] applies to override-dependencies. Against real binaries that holds only from uv 0.5.6: on 0.2.35–0.5.3 a Transitive target wired through the override branch is silently reinstalled from the registry by a plain `uv sync` (the lock still names the vendored path, so `--frozen` installs it, which is why the matrix looked green). No lock-shape marker separates 0.5.3 from 0.5.6, so an offline refusal is impossible. Fix: the Transitive branch of wire_uv pushes an advisory VendorWarning with the stable code `pypi_uv_override_requires_uv_0_5_6` ("pin uv or make the package a direct dependency") through the outcome's warnings, and the module doc states the >= 0.5.6 boundary. The Direct branch stays silent. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * uv: repoint [manifest] constraints / build-constraints for the redirected package When the target is named in `[tool.uv] constraint-dependencies` (or build-constraint-dependencies), uv.lock carries `[manifest] constraints = [{ name = "urllib3", specifier = "==1.26.18" }]` (resp. build-constraints). Once the package has a source, uv >= 0.5.6 re-serializes that element as `{ name, path|url }` with the specifier dropped (uv 0.12.15 ground truth), so after vendoring or a hosted redirect `uv lock --check` / `uv sync --locked` exited 2 and a plain sync churned the lock. Neither backend touched those arrays: the vendored surgery only knew requires-dist / requires-dev / overrides, and the hosted rewrite_manifest returned early whenever the (script-lock-only) `requirements` key was absent. Vendored: rewrite_manifest_constraints repoints every matching element of both arrays, one whole-line `<key> = […]` fragment per key recorded as the new wiring kind `uv_lock_manifest_constraints` (unknown kinds are warned+skipped on revert, so the arm is added to the replace_fragment group). The records are pushed last so the reverse-order revert restores them before searching for a requires-dist element that may be byte-identical. Hosted: rewrite_manifest repoints matching `constraints` / `build-constraints` elements before the `requirements` early return, and still invents no `overrides` array for a direct dependency. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * uv vendor: keep a CRLF pyproject.toml / uv.lock pair CRLF through wire and revert wire_uv wrote `doc.to_string()` for pyproject.toml, and toml_edit re-emits every newline as LF: a pure-CRLF pyproject (git autocrlf on Windows) came back all-LF — whole-file churn — and revert_uv, which splices fragments rather than rewriting the file, could never restore it. The lock side had the same class in three places: rewrite_target_package_unit rebuilt the target unit with `join("\n")` and appended the reconstructed [package.metadata] block with "\n\n" (mixed endings inside a CRLF lock); add_manifest_override created `[manifest]\n…\n\n`, `overrides = […]\n` and the multi-line `,\n]` tail with LF; and revert_uv removed those fragments with hardcoded "{new}\n\n" / "{new}\n", so a CRLF lock's created overrides were left in place with a drift warning. Fix: render pyproject through preserve_line_endings, record the uv_override Rewritten old/new array fragments in the file's convention (a multi-line array renders LF and replace_fragment would miss), detect the lock's terminator once (newline_of) and build every spliced, appended and removed lock fragment with it; the target unit's span is trimmed of the trailing `\r` find_unit_span includes so the rebuilt unit splices back in front of the same `\r\n`. The regression test CRLF-converts the Direct, Transitive (created override + created [manifest]), Rewritten-override (multi-line arrays in both files) and requires-dev fixtures, asserts the wired pair has no bare LF and equals the LF wiring modulo terminators, and reverts byte-exactly. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * uv: say what a [[distribution]] lock actually cannot do instead of "records absolute paths" Both refusals (load_uv_project and the vendored ArtifactSource::Path arm of rewrite_python_lock) claimed uv < 0.2.35 "records absolute file paths". Against the real 0.1.45–0.2.34 binaries that is not the limitation: <= 0.2.6 cannot parse a relative path source at all; 0.2.17–0.2.34 install one under --frozen / plain sync, but `--locked` rejects any non-canonical spelling, plain sync (0.2.34) and every `uv lock` absolutize it, resolution is CWD-relative, and 0.2.17/0.2.18 never verify the wheel hash. A user reading the old text would look for an absolute path to fix rather than upgrade or switch to a requirements.txt install. Both messages now read: uv `[[distribution]]` lockfiles (uv < 0.2.35, experimental `uv lock`) cannot carry a portable local wheel: `--locked` rejects relative paths and `uv lock`/`uv sync` rewrite them to absolute ones; upgrade to uv >=0.2.35 for native vendoring, or use a requirements.txt installation. The vendored refusal gains a pinning test; the existing hosted assertion (`contains("0.2.35")`) still holds. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * uv vendor: gate the metadata no-op on the sub-tables too, not the bare header The 0.2.35/0.2.36 early return in rewrite_root_metadata_entries checked `unit_text.contains("[package.metadata]")`. The requires-dev sub-table header `[package.metadata.requires-dev]` does NOT contain that substring (`.` where the bare header has `]`), so a root unit carrying only the sub-table would have been treated as "no metadata": the requires-dev entry kept its specifier, `uv lock --check` / `uv sync --locked` went red, and nothing refused. Every uv release since 0.2.37 happens to write the empty bare header first, which is why this never fired. The no-op now requires the absence of BOTH the bare header and any `[package.metadata.` sub-table; the later scan already works from the sub-table header alone, so such a unit is repointed rather than skipped. Pinned by a test whose root unit has only [package.metadata.requires-dev]. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * uv vendor: refuse a symlinked pyproject.toml / uv.lock before any write atomic_write_bytes_preserving_mode stages the new content next to the path and renames over it. uv itself writes THROUGH a symlinked uv.lock / pyproject.toml, but our rename REPLACES the link with a regular file: the target keeps the pre-vendor bytes, git shows a typechange, and a revert (which also renames) can never put the link back. wire_uv now refuses fail-closed before either write, and revert_uv keeps the artifact and fails, both with `pypi_uv_symlink_unsupported` naming the offending file (lstat via crate::utils::fs::is_symlink — the link itself, not its target). The orchestrator sweeps the freshly built wheel on a wiring refusal, so no `.socket/vendor` residue is left. The module's private FIFO-safe reader is replaced by the shared crate::utils::fs::read_regular_to_string. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * vendor/pypi: fail-closed listing, include probing and orphan reclaim for the unwired-revert guard The unwired-revert guard added after #238 protects a `repair`-reconstructed pypi ledger entry (no wiring records) from being deleted while a Python project file still resolves through the vendored wheel. Four gaps remained: - `keep_artifact` (`rollback/remove --preserve-state`) was still guarded, so a preserve-state revert of an unwired entry refused with `vendor_wiring_unknown_revert_blocked` although it would have deleted nothing. npm skips the guard there because the refusal exists only to protect the deletion; the pypi guard now does the same. - The lock enumeration failed OPEN: a `read_dir` error on the project root was read as "no Python locks here", `python_lock_paths` silently dropped any lock whose (followed) metadata failed, and the static probe list lacked uv.lock and pylock.toml. On a 0311 (execute-only) root, a listing-denied ACL or a transient EMFILE, `vendor --revert` deleted the referenced wheel and dropped the entry while uv.lock still resolved through it. The guard now lists the root itself (lstat only, so a symlinked lock with an unreadable target is still probed and fails closed on the read), and an unlistable root refuses with its own clause. - Once the guard cleared, the entry was still dispatched by flavor, which made a true orphan unreclaimable forever: flavor `uv` with uv.lock gone failed "cannot read uv.lock", and flavor `None` (what `repair` stamps for requirements/poetry/pdm/pipenv reconstructions) was unknown to the dispatch. An empty-wiring entry that passes the guard now skips the dispatch (it has nothing to replay) and proceeds to the uuid-dir removal. An UNKNOWN non-None flavor still fails closed: a newer backend's files may reference the artifact from a place this guard does not probe. - The requirements planner writes vendored pins into `-r` includes, but the guard probed only the root requirements.txt. `requirements_include_names` (new, exported from the core crate for the CLI's repair/orphan sweep) walks the same include tree as the planner — FIFO-safe, `Err` when a reached include exists but cannot be read — and the guard probes every in-root include, failing closed on an unreadable tree. Verified against real uv 0.9.30 and 0.5.31 through the built CLI: the refusal on a 0311 root keeps the wheel and `uv sync --frozen --offline` still installs from it; after `uv remove` the orphan is reclaimed and the ledger entry dropped. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * repair/vendor: FIFO-safe reference scan that also sees `-r` include pins `repair`'s ledger reconstruction and `vendor --revert`'s orphan sweep share `scan_vendor_references`, which read every wiring-file candidate with a plain `tokio::fs::read_to_string`. A FIFO under one of those names — most easily the paired `<script>.py` derived from a `*.py.lock`, which the lock lister never stats — blocked forever in open(2) waiting for a writer, so `repair` never returned. The same raw read sat in `detect_reference_flavor` (the Python lock loop and the npm lock closure). All three now go through `read_regular_to_string`, keeping the existing skip-on-error tolerance. The scan also consulted the root requirements.txt alone, while the requirements planner writes a vendored pin where the original pin was — possibly inside a `-r`/`--requirement` include. After a state.json loss `repair` could not re-adopt such a wheel, and the orphan sweep deleted it as `vendor_orphan_removed` while `pip install -r` still resolved through it. The scan now appends the planner's in-root include tree via the new core `requirements_include_names` (an unreadable tree degrades to the root file, matching the per-file tolerance), so repair and the sweep see include-hosted pins. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * python_lock: fix uv version boundaries in comments; pin ≤0.2.5 sdist and sibling-shape rewrites The `[[distribution]]` shape comments disagreed with the PR's own probe matrix and docs: the inline-table-source era was labelled "0.2.20 through 0.2.34" (it begins at 0.2.18, the string→inline-table flip is 0.2.17→0.2.18) and the string-source/inline-artifact era "0.2.14–0.2.17" (it begins at 0.2.6, the sub-table→inline artifact flip is 0.2.5→0.2.6). The test doc for the latter also attributed bare `[[distribution.dependencies]]` edges to the whole era, but 0.2.6–0.2.15 still write source-qualified edges; only 0.2.16–0.2.17 write bare ones. Anyone deciding from the comments which real uv binary to reproduce a report with would pick the wrong one. Fix the four sites to the verified boundaries and pin two PR-introduced behaviours that had no test of their own: - `legacy_sdist_redirect_emits_sub_table`: a ≤0.2.5 lock (string source, `[distribution.sdist]` + `[[distribution.wheel]]` tables) redirected to a .tar.gz artifact gets a `[distribution.sdist]` sub-table with url/hash, loses the stale `[[distribution.wheel]]`, follows the source-qualified edge, and is byte-stable on rerun. - `artifactless_target_follows_sibling_shape`: an entry with no artifact keys takes its grammar from a sibling — `[[distribution.wheel]]` siblings yield a sub-table, inline `wheels = […]` siblings yield an inline array — and both are byte-stable on rerun. Both were GREEN on HEAD; they guard the shape selection against regression. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * lock_inventory: document that uv.lock exclusivity is keyed on parse success; pin poetry/script union and corrupt-uv.lock fall-through The comment above the pypi fallback chain said "uv.lock stays the EXCLUSIVE project inventory", which reads as presence-keyed. The code keys exclusivity on `python_lock_inventory` SUCCEEDING: an unparseable uv.lock contributes nothing and falls through to poetry.lock / requirements.txt exactly like a package-less poetry.lock does. That is the right behaviour — presence-keying would hide every requirements pin behind a corrupt lock and diverge from hosted, which skips the same file with `redirect_uv_lock_unsupported` and still reads the other pins — but a reader taking the comment at face value would "fix" it the wrong way. Reword the comment to say PARSEABLE and spell out why, and pin the two untested legs of the chain in `python_lock_union_tests`: - `poetry_lock_unions_with_script_lock_and_hides_requirements`: with no uv.lock, poetry.lock is the tool lock (hides requirements.txt) while a `tool.py.lock` still unions with it. - `unparseable_uv_lock_falls_through_to_requirements`: garbage-TOML uv.lock + requirements.txt yields the requirements pin. Both were GREEN on HEAD. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * python_script: scope the SourcesLayout revert claim to direct deps; pin where toml_edit places [tool.uv.sources] The SourcesLayout doc claimed an "out-of-order multi-package document revert converges" for the dotted script layout. That holds for DIRECT deps, each of which owns one `sources.<name>` key; TRANSITIVE pairs share a single `override-dependencies` array and `restore_value`'s array branch flags drift as soon as the entries stop lining up, so their revert is order-dependent whatever the layout. It also promised "a real `[tool.uv.sources]` table AFTER `[project]`", but toml_edit places a new table right after the last pre-existing table it visits in depth-first key order: after a trailing `[build-system]`, right after an existing `[tool.uv]`, and — with `[tool.ruff]` as the only tool table ahead of `[project]` — inside the tool block BEFORE `[project]`. Both statements would mislead whoever next touches revert or placement. Scope the doc to direct deps, describe the placement rule as observed, and pin the renderings with idempotency (a second rewrite returns None) after every assertion, including the two pre-existing rendering tests: - trailing `[build-system]` after `[project]`, LF and CRLF; - `[tool]` with `uv = { … }` as an inline table: the source lands as a dotted `sources.alpha` key inside the inline table, parses, settles; - `[tool.ruff]` before `[project]` (direct + transitive) and `[tool.uv]` followed by `[tool.ruff]`; - upgrade path from an older CLI's root-body dotted `tool.uv.sources.other = {…}`: the new source extends the dotted table instead of adding a conflicting header; - PEP 723 block with a user-authored `# [tool.uv.sources]` / `# [tool.uv]` header: our insertion joins the table, the extracted block parses, and every line is re-commented. All GREEN on HEAD; no rendering bug surfaced. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * pypi_lock: read symlinked locks like their targets, refuse to write through them Since Python lock discovery started following symlinks (#239), `read_file`'s lstat-based `symlink_metadata().is_file()` pre-check rejected every linked pylock.toml / *.py.lock with `pypi_lock_read_failed: … is not a regular file`. `contains_target` and `load_python_locks` bubble the FIRST unreadable sibling, so a stray non-referencing `pylock.dev.toml` link beside a REGULAR pylock.toml blocked vendoring of the whole project — on main the link was simply invisible. Reading through a link is safe (`open_regular_file`'s fstat already accepts a link to a regular file and still rejects FIFOs/directories), so `read_file` now delegates to the shared `read_regular_to_string`. Writing through one is not: every writer here stages next to the path and renames over it, which replaces the link with a detached regular copy — the target goes stale (uv itself writes THROUGH the link) and a later revert restores bytes but never the link (git: 120000→100644 typechange). The hosted replay side already refuses linked files fail-closed; `wire_python_locks` and `revert_python_locks` now check `is_symlink` on every file they are about to write (the lock AND the paired PEP 723 script) BEFORE the first write and refuse with `pypi_lock_symlink_unsupported` naming the file. A refused revert fails outright so the artifact and ledger entry stay for a retry. Tests (RED-first for the four symlink cases): stray sibling link no longer blocks the regular lock; dangling link is ignored by discovery; symlinked lock and symlinked script are refused before any write (link intact, target byte-identical, no .socket/); the revert twin fails and keeps the vendored bytes. Also pins `restore_document`'s CRLF re-application (LF ledger + CRLF live, the mirror, and the two-package out-of-order CRLF path that does not short-circuit) and documents the known order-dependence of reverting two TRANSITIVE script deps sharing one override-dependencies array. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * hosted: refuse symlinked rewrite targets before the ledger; FIFO-safe candidate reads The hosted write loop renames a staged copy over every `rewrite.files` key. Since discovery follows links, a symlinked uv.lock / pylock*.toml / *.py.lock / pyproject.toml (and, all along, a symlinked package-lock.json) was silently converted into a detached regular copy: the shared target stayed unpatched (uv writes THROUGH the link), and `--revert` could restore bytes but never the link. Reproduced against uv 0.12.15. The revert side (replay.rs) already refuses linked files; the write side now matches: before `save_redirect_state` and before any file write, every planned file is checked with the new `utils::fs::first_symlink`, and a hit fails the WHOLE rewrite (hosted rewrites are transactional) with exit 1, stable code `redirect_symlinked_file_unsupported` naming the file, `errorCode` in the JSON envelope, no ledger and no writes. Applies to every ecosystem and to dry runs, so a dry run predicts the refusal rather than a rewrite that will never happen. Separately, six candidate reads (REDIRECT_CANDIDATE_FILES, the Python locks and their paired scripts, the rush common/subspace locks, and the sync pnpm-workspace.yaml trust probe) used a plain `read_to_string`: a FIFO planted under any of those names wedged `scan --mode hosted` / `get --mode hosted` forever in open(2) waiting for a writer (reproduced). They now go through `read_regular_to_string` (non-blocking open + fstat), plus a new blocking twin `read_regular_to_string_sync` for the sync probe that keeps its NotFound/PermissionDenied/InvalidData classification. A non-regular file reads as unreadable and is skipped like a missing one. Tests (RED-first): `hosted_refuses_symlinked_lock_before_ledger_write` (real 0.12.15 uv.lock linked into a shared dir, wheel served by wiremock: exit 1, link intact, pyproject untouched, no .socket/), the package-lock.json twin with the human-output check, a regular-file control that redirects, and `hosted_scan_returns_with_fifo_candidate` (mkfifo pyproject.toml beside a real uv.lock, 90 s deadline with FIFO release on timeout — wedged before the fix, returns in ~1 s after). fs.rs unit tests cover the sync reader's error kinds and FIFO fail-fast and `first_symlink`'s ordering/dangling semantics. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * backtest-uv: variant lane, stdout export, and a doc-table renderer The real-uv matrix's only native-project fixture is `dependencies = ["urllib3==1.26.18"]`, which never reaches the `[tool.uv] dev-dependencies` / PEP 735 requires-dev paths, a duplicate requires-dist entry, a `[manifest]` constraints entry, or the transitive override branch — exactly the shapes where `uv sync --locked` was found to exit 2 after a "successful" scan. Add `variant_matrix`, run on every `[[package]]`-grammar binary: five pyproject shapes (tool-uv-dev, dependency-groups, extras-duplicate, constraints, transitive) locked fresh, scanned hosted and vendored, then installed into a fresh venv with --frozen, --locked (where listed) and a plain `uv sync`, recording exit code, lockUnchanged and the installed response.py hash. A plain sync that rewrites the lock is recorded, not raised: on uv < 0.5.6 the transitive fixture is expected to reinstall the registry wheel. Fixtures the binary cannot lock (no [dependency-groups] before 0.4.27, no [manifest] constraints on older uv) record formatSupported: false. The transitive cut-off lives in `[tool.uv] exclude-newer` rather than on the `uv lock` command line: a CLI flag is stored under `[options]` and its absence on sync makes uv re-resolve ("removal of global exclude newer"), failing --locked for a reason unrelated to the patch. Against the pre-fix CLI on 0.12.15 and 0.2.37 the lane is red where the review said it would be: vendored tool-uv-dev and extras-duplicate and both modes of constraints fail --locked on 0.12.15; on 0.2.37 tool-uv-dev and transitive fail --locked and plain sync reinstalls the pristine wheel. The export lane passed `uv export --output-file`, which only exists from 0.4.7, so the pinned export-boundary release 0.4.1 recorded four exit-2 rows and the doc showed "— / —" for a release whose export installs the patch. Read the export from stdout and write the file only on a successful exit (an empty file would pass for a lock in format_matrix); 0.4.1 now records Pass / Pass for the requirements export. write_summary carries the variant observations and gives each version row a `variants` verdict, plus `pythonVersion`. `--render-doc-table <results.json>` prints the generated section of docs/testing/uv-compatibility.md (run header, per-release table, project-variant table) so the doc is regenerated after a run instead of hand-edited; rendering the committed results.json reproduces the current 41 rows byte-for-byte. probe-uv-boundaries.py separates "the `uv lock` subcommand exists" (0.1.42) from "it writes a lock" (0.1.45; 0.1.42–0.1.44 panic "not yet implemented"), reports the panic line, and checks whether `uv export` accepts `--output-file` (0.4.7). Verified on real 0.1.41/0.1.44/0.1.45 and 0.4.6/0.4.7 binaries. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: correct the uv boundaries and describe the #239 hardening docs/testing/uv-compatibility.md carried three inaccuracies the real binaries contradict: `[[distribution]]` locks start at 0.1.45, not "0.1.x" (the `uv lock` subcommand appears in 0.1.42, panics through 0.1.44 and first writes a lock at 0.1.45); the root-build note blamed "uv 0.2.x and 0.3.0" when the cold --offline failure spans 0.2.35–0.3.5, 0.3.3 adds `--no-install-project` (which the harness passes), and 0.4.0 stops building a build-system-less root; and the vendored refusal for `[[distribution]]` locks claimed uv "records absolute file paths" when the real reasons are that a relative path source is unparseable through 0.2.6 and, on 0.2.17–0.2.34, rejected by --locked and absolutized by `uv lock` / plain `uv sync` with CWD-relative resolution and unverified hashes. Fix all three, add the `uv export --output-file` (0.4.7) boundary, define "lock-preservation check" precisely (every install attempt against a patched lock, failed installs included, with the --locked rows as the ones that measure preservation), and stop calling every non-zero exit a documented uv boundary: the 0.4.1 export rows were a harness gap. Document the hardening landing in #239: [tool.uv] dev-dependencies as a direct dependency with every duplicate requires-dist/requires-dev entry repointed, [manifest] constraints repointed, the pypi_uv_override_requires_uv_0_5_6 advisory (uv applies sources to overrides only from 0.5.6), symlinked project files refused before any write in both modes, vendored requirements' hash-enforcement boundaries (--require-hashes from 0.1.32, default from 0.5.x), and the CRLF-preserving vendored uv backend. Describe the new project-variant lane and `--render-doc-table`, and fence the results section with GENERATED markers so the merger regenerates it after the full rerun; the recorded numbers are unchanged and the rerun replaces the variant-table placeholder. docs/ecosystems.md and CHANGELOG.md: hosted uv coverage is requirements from 0.0.5 and native uv.lock from 0.1.45, not "every release since 0.1"; symlinked locks are discovered and refused before any write; the CRLF bullet covers the vendored backend; new bullets for the --locked fixes, the constraints repoint, the override advisory, the reworded [[distribution]] refusal, the unwired-revert guard fixes and the FIFO-safe CLI reads. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * uv: count [tool.uv] dev-dependencies as direct for hosted sources; bracket the uv lock boundary Hosted twin of the vendored classifier fix: rewrite_project_metadata decided "direct" from project.dependencies / optional-dependencies / dependency-groups only, so a package declared in the legacy [tool.uv] dev-dependencies array got an override-dependencies entry plus a trailing [manifest] overrides section it never needed — and uv < 0.5.6 does not apply [tool.uv.sources] to overrides at all. uv records that array under [package.metadata.requires-dev], which complete_python_lock_metadata already repoints, so the source alone carries the redirect. Pin uv 0.1.44 in the backtest matrix so the `uv lock` boundary (first lock written by 0.1.45; the subcommand panics through 0.1.44) is bracketed like every other one, and refresh the stale CRLF premise in the toml_surgery removal-helper test now that the uv wire renders in the file's convention. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * uv vendor: advise that the [manifest] constraints repoint needs uv >= 0.5.6 The 42-release matrix's new constraints variant showed the boundary the override branch already had: uv 0.2.37–0.5.3 serialize [manifest] constraints as { name, specifier } regardless of sources, so the repointed entry (the shape uv >= 0.5.6 writes and requires) makes `uv sync --locked` fail there, while --frozen and a plain sync still install the patch (the plain sync rewrites the entry back). No lock marker separates 0.5.3 from 0.5.6, so emit the advisory pypi_uv_constraints_require_uv_0_5_6 next to the repoint instead of refusing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Record the 42-release uv compatibility matrix for source 17d0dcb Regenerate results.json, binaries.json and the generated results section of docs/testing/uv-compatibility.md from a full scripts/backtest-uv.py run against the rebuilt CLI at 17d0dcb: 42 pinned uv releases (0.1.44 added to bracket the `uv lock` boundary), 1403 installed-byte comparisons and 1086 lock-preservation checks including the new project-variant lane ([tool.uv] dev-dependencies, [dependency-groups], extras duplicate, constraint-dependencies, transitive) on every [[package]] binary. Every mismatch and lock change is a documented uv boundary below 0.5.6: the 18 mismatches are the transitive variant's plain `uv sync` on 0.2.35–0.5.0 (sources not applied to override-dependencies), the 32 lock changes / non-zero --locked rows are those plus the constraints variant on 0.2.37–0.5.0 ({ name, specifier } serialization regardless of sources); the patch still installed in every constraints row. From 0.5.16 every shape passes in both modes with the lock byte-unchanged. The 0.4.1 export lane now records real installs (stdout export). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * utils/fs: accept Windows' PermissionDenied for a directory in the sync-reader test CreateFileW refuses to open a directory with ERROR_ACCESS_DENIED before the fstat regular-file check can classify it as InvalidInput; the callers skip either error kind, so the test now accepts both off Unix. Windows CI leg on 18d4046 was the only failure. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LLM Description written by Codex:gpt-6-astra
Keep hosted and vendored Python patches installed across uv native locks, PEP 751 locks, compiled requirements, and PEP 723 script locks. Hosted native rewrites now replace the registry source and preserve the matching artifact hash and wheel metadata; paired project/script metadata keeps patches installed during ordinary and
--lockedinstalls as well as frozen installs.Hashed requirements retain extras and environment markers while replacing complete hash continuations. Version-specific rewrites preserve other conditional versions. Paired script/project rewrites refuse competing versions or source conflicts before changing either file. Vendored scripts and standalone PEP 751 locks use local patched wheels, and inventory/repair recognizes their references.
Validation
cargo clippy --workspace --all-features -- -D warningspassed.e11bd419ea9c01b3ecd1aa894b55874718a3ff0aon macOS arm64 with CPython 3.9.6. The compatibility report includes exact commands, binary hashes, per-version results, and sanitized supplemental evidence. This covers every release family, not every patch release or operating system.uv 0.0 has no native lock format and rejects local wheel paths. Portable native vendoring explicitly requires uv 0.2 or newer; hosted rewriting supports the historical uv 0.1 distribution format. Exports, script locks, and PEP 751 are tested only on versions that implement those formats. Global uv sources cannot safely represent competing conditional versions, so those paired rewrites are explicitly refused.
Companion scanner annotation PR: https://github.com/SocketDev/depscan/pull/26136. Both changes are developed in separate worktrees; the scanner keeps fixtures from the released CLI alongside output from this change.