Harden uv lockfile patch preservation and verify every uv release family - #239
Merged
Merged
Conversation
Fail-closed revert for unwired Python entries: a ledger entry with no wiring (the shape `repair` re-synthesizes when state.json is lost) used to route into the uv / python-lock revert, iterate zero records, report success, and let the caller delete the vendored wheel and drop the entry while uv.lock, the pylock, the script, or requirements.txt still resolved through it. `revert_pypi_opts` now refuses while any Python project file references the uuid dir, mirroring the npm-family guard. Inventory: a PEP 723 script lock or PEP 751 lock supplements the project's poetry.lock / requirements.txt pins instead of hiding them; uv.lock keeps its exclusive precedence. Discovery: `python_lock_paths` follows symlinks and skips one unreadable entry instead of dropping every lock. Line endings: toml_edit re-emits every newline as LF, so CRLF pylock, script-lock and pyproject rewrites (and document-restore reverts) now restore the input's CRLF convention. Ledger recovery compares the purl name in PEP 503 form like `lookup`. Hosted pyproject edits render uv's own layout — header-less `[tool]` / `[tool.uv]` parents and a `[tool.uv.sources]` table after `[project]` — instead of dotted keys above `[project]`. The PEP 723 script block keeps fully dotted keys: only those vanish when empty after a parse round trip, which the out-of-order multi-package document revert relies on. `redirect_uv_missing_sha256` fires once per dep, not once per lock file; `repair` prefers uv.lock over an exported pylock when stamping flavor. Backtest harness: retry the cold-offline root-build failure with network for any uv version (not just 0.2.37), and pin first, latest, and every observed behaviour-boundary release of each 0.x family. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The experimental `[[distribution]]` grammar (uv 0.1.x–0.2.34) went through
three shapes, and the rewriter keyed everything on the table name:
- string sources + sub-table artifacts (`[distribution.sdist]`,
`[[distribution.wheel]]`) through 0.2.5;
- string sources + inline artifacts (`sdist = {…}`, `wheels = [...]`)
from 0.2.6 through 0.2.17;
- inline-table sources (`source = { registry = … }`) from 0.2.18.
Emitting the string source into a 0.2.18–0.2.34 lock made uv reject it
("data did not match any variant of untagged enum SourceWire") and
silently ignore the lock: an ordinary `uv sync` still installed the patch
through the pyproject source, but `--frozen` / `--locked` failed. Emitting
`[[distribution.wheel]]` into a 0.2.6–0.2.17 lock left the binary with a
direct URL and no wheel, which it tried to build as a source archive
("Unsupported archive type: …whl"). Decide the source shape from the
entry's own `source` syntax and the artifact shape from its own artifact
keys (falling back to any sibling entry), independently.
Vendored native wiring stays refused for the whole `[[distribution]]` era:
every shape records absolute file paths for local artifacts. Reword the
refusal to say so (it claimed "uv 0.1", but 0.2.34 refuses too).
Add scripts/probe-uv-boundaries.py — the bisection tool behind every
boundary pinned in scripts/backtest-uv.py (lock grammar, source and
artifact shape, lock revision, `uv export`, `uv lock --script`, PEP 751
compilation, bare local wheel paths in requirements) — and pin the
0.2.5/0.2.6 and 0.2.17/0.2.18 pairs in the matrix. Document the exact
boundaries.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The first two `[[package]]`-grammar releases wrote no root `[package.metadata]`; the metadata step refused them with `pypi_uv_lock_package_missing` although there was nothing to repoint. Skip the requires-dist rewrite when the root unit has no metadata table at all (a lock that HAS metadata but lacks the entry is stale and still refuses), so vendored native wiring covers every `[[package]]` release. Verified with real `uv sync --frozen` / `--locked` / ordinary installs on 0.2.35, 0.2.36 and 0.2.37. Pin 0.2.36/0.2.37 as the metadata boundary in the matrix and document the exact grammar shapes and coverage. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Regenerate results.json, binaries.json, and the results section of docs/testing/uv-compatibility.md from a full run of scripts/backtest-uv.py against the rebuilt CLI: 41 pinned uv releases (first and latest of every 0.x family plus every observed behaviour boundary), 583 installed-byte comparisons and 240 lock-preservation checks, zero mismatches and zero lock changes. Every non-zero exit is a documented uv boundary or the cold-offline root-build case retried with network. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko)
marked this pull request as ready for review
September 15, 2026 15:54
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Preserve-state fires empty-wiring refusal
- Added !keep_artifact condition to empty-wiring guard in revert_pypi_opts to allow preserve-state mode to succeed without deletion refusal.
Or push these changes by commenting:
@cursor push c7e80e8475
Preview (c7e80e8475)
diff --git a/crates/socket-patch-core/src/vendor/pypi.rs b/crates/socket-patch-core/src/vendor/pypi.rs
--- a/crates/socket-patch-core/src/vendor/pypi.rs
+++ b/crates/socket-patch-core/src/vendor/pypi.rs
@@ -960,7 +960,7 @@
dry_run,
keep_artifact,
} = opts;
- if entry.wiring.is_empty() {
+ if !keep_artifact && entry.wiring.is_empty() {
let uuid_dir_rel = vendor_uuid_dir_rel("pypi", &entry.uuid)
.unwrap_or_else(|| format!(".socket/vendor/pypi/{:?}", entry.uuid));
if let Some(blocked) =
@@ -3824,7 +3824,7 @@
}
}
- // ───────────── uv guard failures surfaced through the orchestrator ────────────���
+ // ───────────── uv guard failures surfaced through the orchestrator ──────���──────
#[tokio::test]
async fn uv_lock_parse_failure_refuses_through_orchestrator() {You can send follow-ups to the cloud agent here.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 3726770. Configure here.
Wenxin Jiang (Wenxin-Jiang)
approved these changes
Sep 15, 2026
Shared FIFO-safe reader (the shape every ecosystem module re-declared privately) and an lstat-based symlink probe, exported so the CLI crate's raw read_to_string sites and the Python lock writers can use them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ry metadata entry classify_dependency scanned project.dependencies, optional-dependencies and [dependency-groups] but not the legacy `[tool.uv] dev-dependencies` array, which uv still honours (0.12.15 warns but records it exactly like `dependency-groups.dev`, in the root unit's [package.metadata.requires-dev]). Such a target classified Transitive: wire_uv took the override branch, [package.metadata.requires-dev] kept `specifier = "==…"`, and after a "successful" vendored scan every uv >= 0.2.37 `uv sync --locked` / `uv lock --check` exited non-zero while a plain `uv sync` rewrote the lock (ledger drift) and on 0.2.37/0.4.30 reinstalled the PRISTINE wheel. rewrite_root_metadata_entries also repointed only the FIRST matching element per array (a `break` in the requires-dist loop and per group in requires-dev) although its doc promised ALL: a package declared in both dependencies and an optional-dependencies extra has two requires-dist entries (the extra's with `marker = "extra == '…'"`) and uv repoints both, so the stale second entry kept --locked red. Fix: extend `declared` with the [tool.uv] dev-dependencies string members (pep508_name); repoint every matching requires-dist element as its own edit/wiring record (revert already replays N records per package), and rebuild each requires-dev group line once with every matching element rewritten so a group naming the package twice never yields overlapping spans. Both new fixtures are byte-exact uv 0.12.15 output. wire_uv now also returns a Vec<VendorWarning> of wiring-time advisories, threaded into the vendor outcome's warnings by pypi.rs (the channel the next commit's override-branch advisory uses). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The module doc's "claim 8" said [tool.uv.sources] applies to
override-dependencies. Against real binaries that holds only from uv
0.5.6: on 0.2.35–0.5.3 a Transitive target wired through the override
branch is silently reinstalled from the registry by a plain `uv sync`
(the lock still names the vendored path, so `--frozen` installs it, which
is why the matrix looked green). No lock-shape marker separates 0.5.3 from
0.5.6, so an offline refusal is impossible.
Fix: the Transitive branch of wire_uv pushes an advisory VendorWarning
with the stable code `pypi_uv_override_requires_uv_0_5_6` ("pin uv or make
the package a direct dependency") through the outcome's warnings, and the
module doc states the >= 0.5.6 boundary. The Direct branch stays silent.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…cted package
When the target is named in `[tool.uv] constraint-dependencies` (or
build-constraint-dependencies), uv.lock carries
`[manifest] constraints = [{ name = "urllib3", specifier = "==1.26.18" }]`
(resp. build-constraints). Once the package has a source, uv >= 0.5.6
re-serializes that element as `{ name, path|url }` with the specifier
dropped (uv 0.12.15 ground truth), so after vendoring or a hosted redirect
`uv lock --check` / `uv sync --locked` exited 2 and a plain sync churned
the lock. Neither backend touched those arrays: the vendored surgery only
knew requires-dist / requires-dev / overrides, and the hosted
rewrite_manifest returned early whenever the (script-lock-only)
`requirements` key was absent.
Vendored: rewrite_manifest_constraints repoints every matching element of
both arrays, one whole-line `<key> = […]` fragment per key recorded as the
new wiring kind `uv_lock_manifest_constraints` (unknown kinds are
warned+skipped on revert, so the arm is added to the replace_fragment
group). The records are pushed last so the reverse-order revert restores
them before searching for a requires-dist element that may be
byte-identical. Hosted: rewrite_manifest repoints matching `constraints` /
`build-constraints` elements before the `requirements` early return, and
still invents no `overrides` array for a direct dependency.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e and revert
wire_uv wrote `doc.to_string()` for pyproject.toml, and toml_edit re-emits
every newline as LF: a pure-CRLF pyproject (git autocrlf on Windows) came
back all-LF — whole-file churn — and revert_uv, which splices fragments
rather than rewriting the file, could never restore it. The lock side had
the same class in three places: rewrite_target_package_unit rebuilt the
target unit with `join("\n")` and appended the reconstructed
[package.metadata] block with "\n\n" (mixed endings inside a CRLF lock);
add_manifest_override created `[manifest]\n…\n\n`, `overrides = […]\n` and
the multi-line `,\n]` tail with LF; and revert_uv removed those fragments
with hardcoded "{new}\n\n" / "{new}\n", so a CRLF lock's created overrides
were left in place with a drift warning.
Fix: render pyproject through preserve_line_endings, record the
uv_override Rewritten old/new array fragments in the file's convention (a
multi-line array renders LF and replace_fragment would miss), detect the
lock's terminator once (newline_of) and build every spliced, appended and
removed lock fragment with it; the target unit's span is trimmed of the
trailing `\r` find_unit_span includes so the rebuilt unit splices back in
front of the same `\r\n`. The regression test CRLF-converts the Direct,
Transitive (created override + created [manifest]), Rewritten-override
(multi-line arrays in both files) and requires-dev fixtures, asserts the
wired pair has no bare LF and equals the LF wiring modulo terminators, and
reverts byte-exactly.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ecords absolute paths"
Both refusals (load_uv_project and the vendored ArtifactSource::Path arm of
rewrite_python_lock) claimed uv < 0.2.35 "records absolute file paths".
Against the real 0.1.45–0.2.34 binaries that is not the limitation:
<= 0.2.6 cannot parse a relative path source at all; 0.2.17–0.2.34 install
one under --frozen / plain sync, but `--locked` rejects any non-canonical
spelling, plain sync (0.2.34) and every `uv lock` absolutize it, resolution
is CWD-relative, and 0.2.17/0.2.18 never verify the wheel hash. A user
reading the old text would look for an absolute path to fix rather than
upgrade or switch to a requirements.txt install.
Both messages now read: uv `[[distribution]]` lockfiles (uv < 0.2.35,
experimental `uv lock`) cannot carry a portable local wheel: `--locked`
rejects relative paths and `uv lock`/`uv sync` rewrite them to absolute
ones; upgrade to uv >=0.2.35 for native vendoring, or use a
requirements.txt installation. The vendored refusal gains a pinning test;
the existing hosted assertion (`contains("0.2.35")`) still holds.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e header
The 0.2.35/0.2.36 early return in rewrite_root_metadata_entries checked
`unit_text.contains("[package.metadata]")`. The requires-dev sub-table
header `[package.metadata.requires-dev]` does NOT contain that substring
(`.` where the bare header has `]`), so a root unit carrying only the
sub-table would have been treated as "no metadata": the requires-dev
entry kept its specifier, `uv lock --check` / `uv sync --locked` went red,
and nothing refused. Every uv release since 0.2.37 happens to write the
empty bare header first, which is why this never fired.
The no-op now requires the absence of BOTH the bare header and any
`[package.metadata.` sub-table; the later scan already works from the
sub-table header alone, so such a unit is repointed rather than skipped.
Pinned by a test whose root unit has only [package.metadata.requires-dev].
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
atomic_write_bytes_preserving_mode stages the new content next to the path and renames over it. uv itself writes THROUGH a symlinked uv.lock / pyproject.toml, but our rename REPLACES the link with a regular file: the target keeps the pre-vendor bytes, git shows a typechange, and a revert (which also renames) can never put the link back. wire_uv now refuses fail-closed before either write, and revert_uv keeps the artifact and fails, both with `pypi_uv_symlink_unsupported` naming the offending file (lstat via crate::utils::fs::is_symlink — the link itself, not its target). The orchestrator sweeps the freshly built wheel on a wiring refusal, so no `.socket/vendor` residue is left. The module's private FIFO-safe reader is replaced by the shared crate::utils::fs::read_regular_to_string. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…for the unwired-revert guard The unwired-revert guard added after #238 protects a `repair`-reconstructed pypi ledger entry (no wiring records) from being deleted while a Python project file still resolves through the vendored wheel. Four gaps remained: - `keep_artifact` (`rollback/remove --preserve-state`) was still guarded, so a preserve-state revert of an unwired entry refused with `vendor_wiring_unknown_revert_blocked` although it would have deleted nothing. npm skips the guard there because the refusal exists only to protect the deletion; the pypi guard now does the same. - The lock enumeration failed OPEN: a `read_dir` error on the project root was read as "no Python locks here", `python_lock_paths` silently dropped any lock whose (followed) metadata failed, and the static probe list lacked uv.lock and pylock.toml. On a 0311 (execute-only) root, a listing-denied ACL or a transient EMFILE, `vendor --revert` deleted the referenced wheel and dropped the entry while uv.lock still resolved through it. The guard now lists the root itself (lstat only, so a symlinked lock with an unreadable target is still probed and fails closed on the read), and an unlistable root refuses with its own clause. - Once the guard cleared, the entry was still dispatched by flavor, which made a true orphan unreclaimable forever: flavor `uv` with uv.lock gone failed "cannot read uv.lock", and flavor `None` (what `repair` stamps for requirements/poetry/pdm/pipenv reconstructions) was unknown to the dispatch. An empty-wiring entry that passes the guard now skips the dispatch (it has nothing to replay) and proceeds to the uuid-dir removal. An UNKNOWN non-None flavor still fails closed: a newer backend's files may reference the artifact from a place this guard does not probe. - The requirements planner writes vendored pins into `-r` includes, but the guard probed only the root requirements.txt. `requirements_include_names` (new, exported from the core crate for the CLI's repair/orphan sweep) walks the same include tree as the planner — FIFO-safe, `Err` when a reached include exists but cannot be read — and the guard probes every in-root include, failing closed on an unreadable tree. Verified against real uv 0.9.30 and 0.5.31 through the built CLI: the refusal on a 0311 root keeps the wheel and `uv sync --frozen --offline` still installs from it; after `uv remove` the orphan is reclaimed and the ledger entry dropped. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`repair`'s ledger reconstruction and `vendor --revert`'s orphan sweep share `scan_vendor_references`, which read every wiring-file candidate with a plain `tokio::fs::read_to_string`. A FIFO under one of those names — most easily the paired `<script>.py` derived from a `*.py.lock`, which the lock lister never stats — blocked forever in open(2) waiting for a writer, so `repair` never returned. The same raw read sat in `detect_reference_flavor` (the Python lock loop and the npm lock closure). All three now go through `read_regular_to_string`, keeping the existing skip-on-error tolerance. The scan also consulted the root requirements.txt alone, while the requirements planner writes a vendored pin where the original pin was — possibly inside a `-r`/`--requirement` include. After a state.json loss `repair` could not re-adopt such a wheel, and the orphan sweep deleted it as `vendor_orphan_removed` while `pip install -r` still resolved through it. The scan now appends the planner's in-root include tree via the new core `requirements_include_names` (an unreadable tree degrades to the root file, matching the per-file tolerance), so repair and the sweep see include-hosted pins. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…and sibling-shape rewrites The `[[distribution]]` shape comments disagreed with the PR's own probe matrix and docs: the inline-table-source era was labelled "0.2.20 through 0.2.34" (it begins at 0.2.18, the string→inline-table flip is 0.2.17→0.2.18) and the string-source/inline-artifact era "0.2.14–0.2.17" (it begins at 0.2.6, the sub-table→inline artifact flip is 0.2.5→0.2.6). The test doc for the latter also attributed bare `[[distribution.dependencies]]` edges to the whole era, but 0.2.6–0.2.15 still write source-qualified edges; only 0.2.16–0.2.17 write bare ones. Anyone deciding from the comments which real uv binary to reproduce a report with would pick the wrong one. Fix the four sites to the verified boundaries and pin two PR-introduced behaviours that had no test of their own: - `legacy_sdist_redirect_emits_sub_table`: a ≤0.2.5 lock (string source, `[distribution.sdist]` + `[[distribution.wheel]]` tables) redirected to a .tar.gz artifact gets a `[distribution.sdist]` sub-table with url/hash, loses the stale `[[distribution.wheel]]`, follows the source-qualified edge, and is byte-stable on rerun. - `artifactless_target_follows_sibling_shape`: an entry with no artifact keys takes its grammar from a sibling — `[[distribution.wheel]]` siblings yield a sub-table, inline `wheels = […]` siblings yield an inline array — and both are byte-stable on rerun. Both were GREEN on HEAD; they guard the shape selection against regression. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…uccess; pin poetry/script union and corrupt-uv.lock fall-through The comment above the pypi fallback chain said "uv.lock stays the EXCLUSIVE project inventory", which reads as presence-keyed. The code keys exclusivity on `python_lock_inventory` SUCCEEDING: an unparseable uv.lock contributes nothing and falls through to poetry.lock / requirements.txt exactly like a package-less poetry.lock does. That is the right behaviour — presence-keying would hide every requirements pin behind a corrupt lock and diverge from hosted, which skips the same file with `redirect_uv_lock_unsupported` and still reads the other pins — but a reader taking the comment at face value would "fix" it the wrong way. Reword the comment to say PARSEABLE and spell out why, and pin the two untested legs of the chain in `python_lock_union_tests`: - `poetry_lock_unions_with_script_lock_and_hides_requirements`: with no uv.lock, poetry.lock is the tool lock (hides requirements.txt) while a `tool.py.lock` still unions with it. - `unparseable_uv_lock_falls_through_to_requirements`: garbage-TOML uv.lock + requirements.txt yields the requirements pin. Both were GREEN on HEAD. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…in where toml_edit places [tool.uv.sources]
The SourcesLayout doc claimed an "out-of-order multi-package document
revert converges" for the dotted script layout. That holds for DIRECT
deps, each of which owns one `sources.<name>` key; TRANSITIVE pairs share
a single `override-dependencies` array and `restore_value`'s array branch
flags drift as soon as the entries stop lining up, so their revert is
order-dependent whatever the layout. It also promised "a real
`[tool.uv.sources]` table AFTER `[project]`", but toml_edit places a new
table right after the last pre-existing table it visits in depth-first
key order: after a trailing `[build-system]`, right after an existing
`[tool.uv]`, and — with `[tool.ruff]` as the only tool table ahead of
`[project]` — inside the tool block BEFORE `[project]`. Both statements
would mislead whoever next touches revert or placement.
Scope the doc to direct deps, describe the placement rule as observed, and
pin the renderings with idempotency (a second rewrite returns None) after
every assertion, including the two pre-existing rendering tests:
- trailing `[build-system]` after `[project]`, LF and CRLF;
- `[tool]` with `uv = { … }` as an inline table: the source lands as a
dotted `sources.alpha` key inside the inline table, parses, settles;
- `[tool.ruff]` before `[project]` (direct + transitive) and `[tool.uv]`
followed by `[tool.ruff]`;
- upgrade path from an older CLI's root-body dotted
`tool.uv.sources.other = {…}`: the new source extends the dotted table
instead of adding a conflicting header;
- PEP 723 block with a user-authored `# [tool.uv.sources]` / `# [tool.uv]`
header: our insertion joins the table, the extracted block parses, and
every line is re-commented.
All GREEN on HEAD; no rendering bug surfaced.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…hrough them Since Python lock discovery started following symlinks (#239), `read_file`'s lstat-based `symlink_metadata().is_file()` pre-check rejected every linked pylock.toml / *.py.lock with `pypi_lock_read_failed: … is not a regular file`. `contains_target` and `load_python_locks` bubble the FIRST unreadable sibling, so a stray non-referencing `pylock.dev.toml` link beside a REGULAR pylock.toml blocked vendoring of the whole project — on main the link was simply invisible. Reading through a link is safe (`open_regular_file`'s fstat already accepts a link to a regular file and still rejects FIFOs/directories), so `read_file` now delegates to the shared `read_regular_to_string`. Writing through one is not: every writer here stages next to the path and renames over it, which replaces the link with a detached regular copy — the target goes stale (uv itself writes THROUGH the link) and a later revert restores bytes but never the link (git: 120000→100644 typechange). The hosted replay side already refuses linked files fail-closed; `wire_python_locks` and `revert_python_locks` now check `is_symlink` on every file they are about to write (the lock AND the paired PEP 723 script) BEFORE the first write and refuse with `pypi_lock_symlink_unsupported` naming the file. A refused revert fails outright so the artifact and ledger entry stay for a retry. Tests (RED-first for the four symlink cases): stray sibling link no longer blocks the regular lock; dangling link is ignored by discovery; symlinked lock and symlinked script are refused before any write (link intact, target byte-identical, no .socket/); the revert twin fails and keeps the vendored bytes. Also pins `restore_document`'s CRLF re-application (LF ledger + CRLF live, the mirror, and the two-package out-of-order CRLF path that does not short-circuit) and documents the known order-dependence of reverting two TRANSITIVE script deps sharing one override-dependencies array. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… candidate reads The hosted write loop renames a staged copy over every `rewrite.files` key. Since discovery follows links, a symlinked uv.lock / pylock*.toml / *.py.lock / pyproject.toml (and, all along, a symlinked package-lock.json) was silently converted into a detached regular copy: the shared target stayed unpatched (uv writes THROUGH the link), and `--revert` could restore bytes but never the link. Reproduced against uv 0.12.15. The revert side (replay.rs) already refuses linked files; the write side now matches: before `save_redirect_state` and before any file write, every planned file is checked with the new `utils::fs::first_symlink`, and a hit fails the WHOLE rewrite (hosted rewrites are transactional) with exit 1, stable code `redirect_symlinked_file_unsupported` naming the file, `errorCode` in the JSON envelope, no ledger and no writes. Applies to every ecosystem and to dry runs, so a dry run predicts the refusal rather than a rewrite that will never happen. Separately, six candidate reads (REDIRECT_CANDIDATE_FILES, the Python locks and their paired scripts, the rush common/subspace locks, and the sync pnpm-workspace.yaml trust probe) used a plain `read_to_string`: a FIFO planted under any of those names wedged `scan --mode hosted` / `get --mode hosted` forever in open(2) waiting for a writer (reproduced). They now go through `read_regular_to_string` (non-blocking open + fstat), plus a new blocking twin `read_regular_to_string_sync` for the sync probe that keeps its NotFound/PermissionDenied/InvalidData classification. A non-regular file reads as unreadable and is skipped like a missing one. Tests (RED-first): `hosted_refuses_symlinked_lock_before_ledger_write` (real 0.12.15 uv.lock linked into a shared dir, wheel served by wiremock: exit 1, link intact, pyproject untouched, no .socket/), the package-lock.json twin with the human-output check, a regular-file control that redirects, and `hosted_scan_returns_with_fifo_candidate` (mkfifo pyproject.toml beside a real uv.lock, 90 s deadline with FIFO release on timeout — wedged before the fix, returns in ~1 s after). fs.rs unit tests cover the sync reader's error kinds and FIFO fail-fast and `first_symlink`'s ordering/dangling semantics. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The real-uv matrix's only native-project fixture is
`dependencies = ["urllib3==1.26.18"]`, which never reaches the
`[tool.uv] dev-dependencies` / PEP 735 requires-dev paths, a duplicate
requires-dist entry, a `[manifest]` constraints entry, or the transitive
override branch — exactly the shapes where `uv sync --locked` was found
to exit 2 after a "successful" scan. Add `variant_matrix`, run on every
`[[package]]`-grammar binary: five pyproject shapes (tool-uv-dev,
dependency-groups, extras-duplicate, constraints, transitive) locked
fresh, scanned hosted and vendored, then installed into a fresh venv
with --frozen, --locked (where listed) and a plain `uv sync`, recording
exit code, lockUnchanged and the installed response.py hash. A plain
sync that rewrites the lock is recorded, not raised: on uv < 0.5.6 the
transitive fixture is expected to reinstall the registry wheel. Fixtures
the binary cannot lock (no [dependency-groups] before 0.4.27, no
[manifest] constraints on older uv) record formatSupported: false. The
transitive cut-off lives in `[tool.uv] exclude-newer` rather than on the
`uv lock` command line: a CLI flag is stored under `[options]` and its
absence on sync makes uv re-resolve ("removal of global exclude newer"),
failing --locked for a reason unrelated to the patch.
Against the pre-fix CLI on 0.12.15 and 0.2.37 the lane is red where the
review said it would be: vendored tool-uv-dev and extras-duplicate and
both modes of constraints fail --locked on 0.12.15; on 0.2.37
tool-uv-dev and transitive fail --locked and plain sync reinstalls the
pristine wheel.
The export lane passed `uv export --output-file`, which only exists from
0.4.7, so the pinned export-boundary release 0.4.1 recorded four exit-2
rows and the doc showed "— / —" for a release whose export installs the
patch. Read the export from stdout and write the file only on a
successful exit (an empty file would pass for a lock in format_matrix);
0.4.1 now records Pass / Pass for the requirements export.
write_summary carries the variant observations and gives each version
row a `variants` verdict, plus `pythonVersion`. `--render-doc-table
<results.json>` prints the generated section of
docs/testing/uv-compatibility.md (run header, per-release table,
project-variant table) so the doc is regenerated after a run instead of
hand-edited; rendering the committed results.json reproduces the current
41 rows byte-for-byte.
probe-uv-boundaries.py separates "the `uv lock` subcommand exists"
(0.1.42) from "it writes a lock" (0.1.45; 0.1.42–0.1.44 panic "not yet
implemented"), reports the panic line, and checks whether `uv export`
accepts `--output-file` (0.4.7). Verified on real 0.1.41/0.1.44/0.1.45
and 0.4.6/0.4.7 binaries.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
docs/testing/uv-compatibility.md carried three inaccuracies the real binaries contradict: `[[distribution]]` locks start at 0.1.45, not "0.1.x" (the `uv lock` subcommand appears in 0.1.42, panics through 0.1.44 and first writes a lock at 0.1.45); the root-build note blamed "uv 0.2.x and 0.3.0" when the cold --offline failure spans 0.2.35–0.3.5, 0.3.3 adds `--no-install-project` (which the harness passes), and 0.4.0 stops building a build-system-less root; and the vendored refusal for `[[distribution]]` locks claimed uv "records absolute file paths" when the real reasons are that a relative path source is unparseable through 0.2.6 and, on 0.2.17–0.2.34, rejected by --locked and absolutized by `uv lock` / plain `uv sync` with CWD-relative resolution and unverified hashes. Fix all three, add the `uv export --output-file` (0.4.7) boundary, define "lock-preservation check" precisely (every install attempt against a patched lock, failed installs included, with the --locked rows as the ones that measure preservation), and stop calling every non-zero exit a documented uv boundary: the 0.4.1 export rows were a harness gap. Document the hardening landing in #239: [tool.uv] dev-dependencies as a direct dependency with every duplicate requires-dist/requires-dev entry repointed, [manifest] constraints repointed, the pypi_uv_override_requires_uv_0_5_6 advisory (uv applies sources to overrides only from 0.5.6), symlinked project files refused before any write in both modes, vendored requirements' hash-enforcement boundaries (--require-hashes from 0.1.32, default from 0.5.x), and the CRLF-preserving vendored uv backend. Describe the new project-variant lane and `--render-doc-table`, and fence the results section with GENERATED markers so the merger regenerates it after the full rerun; the recorded numbers are unchanged and the rerun replaces the variant-table placeholder. docs/ecosystems.md and CHANGELOG.md: hosted uv coverage is requirements from 0.0.5 and native uv.lock from 0.1.45, not "every release since 0.1"; symlinked locks are discovered and refused before any write; the CRLF bullet covers the vendored backend; new bullets for the --locked fixes, the constraints repoint, the override advisory, the reworded [[distribution]] refusal, the unwired-revert guard fixes and the FIFO-safe CLI reads. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…acket the uv lock boundary Hosted twin of the vendored classifier fix: rewrite_project_metadata decided "direct" from project.dependencies / optional-dependencies / dependency-groups only, so a package declared in the legacy [tool.uv] dev-dependencies array got an override-dependencies entry plus a trailing [manifest] overrides section it never needed — and uv < 0.5.6 does not apply [tool.uv.sources] to overrides at all. uv records that array under [package.metadata.requires-dev], which complete_python_lock_metadata already repoints, so the source alone carries the redirect. Pin uv 0.1.44 in the backtest matrix so the `uv lock` boundary (first lock written by 0.1.45; the subcommand panics through 0.1.44) is bracketed like every other one, and refresh the stale CRLF premise in the toml_surgery removal-helper test now that the uv wire renders in the file's convention. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… 0.5.6
The 42-release matrix's new constraints variant showed the boundary the
override branch already had: uv 0.2.37–0.5.3 serialize [manifest]
constraints as { name, specifier } regardless of sources, so the repointed
entry (the shape uv >= 0.5.6 writes and requires) makes `uv sync --locked`
fail there, while --frozen and a plain sync still install the patch (the
plain sync rewrites the entry back). No lock marker separates 0.5.3 from
0.5.6, so emit the advisory pypi_uv_constraints_require_uv_0_5_6 next to
the repoint instead of refusing.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Regenerate results.json, binaries.json and the generated results section of docs/testing/uv-compatibility.md from a full scripts/backtest-uv.py run against the rebuilt CLI at 17d0dcb: 42 pinned uv releases (0.1.44 added to bracket the `uv lock` boundary), 1403 installed-byte comparisons and 1086 lock-preservation checks including the new project-variant lane ([tool.uv] dev-dependencies, [dependency-groups], extras duplicate, constraint-dependencies, transitive) on every [[package]] binary. Every mismatch and lock change is a documented uv boundary below 0.5.6: the 18 mismatches are the transitive variant's plain `uv sync` on 0.2.35–0.5.0 (sources not applied to override-dependencies), the 32 lock changes / non-zero --locked rows are those plus the constraints variant on 0.2.37–0.5.0 ({ name, specifier } serialization regardless of sources); the patch still installed in every constraints row. From 0.5.16 every shape passes in both modes with the lock byte-unchanged. The 0.4.1 export lane now records real installs (stdout export). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…c-reader test CreateFileW refuses to open a directory with ERROR_ACCESS_DENIED before the fstat regular-file check can classify it as InvalidInput; the callers skip either error kind, so the test now accepts both off Unix. Windows CI leg on 18d4046 was the only failure. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Follow-up to #238 (uv / PEP 751 / PEP 723 Python lock patch preservation). Fixes the findings from its review, fixes the uv-version-specific defects the extended matrix exposed, and re-runs the real-uv compatibility matrix against every 0.x release family — first release, latest release, and the releases on either side of every behaviour boundary — for both hosted and vendored modes, now including a project-variant lane (dev groups, duplicate requires-dist entries, constraints, transitive overrides) on every
[[package]]binary.Fixes from the #238 review
vendor --revertno longer deletes a vendored Python wheel a lock still references. A ledger entry with no wiring records (the shapesocket-patch repairre-synthesizes whenstate.jsonis lost) was routed into the uv / python-lock revert, iterated zero records, reported success, and the caller then removed the artifact directory and dropped the entry whileuv.lock/pylock.toml/ the script /requirements.txtstill resolved through it.revert_pypi_optsnow refuses withvendor_wiring_unknown_revert_blockedwhile any Python project file references the uuid directory, mirroring the npm-family guard.poetry.lock/requirements.txtpins instead of hiding them.uv.lockkeeps its exclusive precedence.DirEntry::file_typedoes not follow symlinks); one unreadable directory entry no longer drops every lock.pylock.toml, script locks andpyproject.tomlkeep their convention through rewrites and document-restore reverts.pyproject.tomledits render uv's own[tool.uv.sources]layout;redirect_uv_missing_sha256fires once per dependency;repairprefersuv.lockover an exported pylock when stamping flavor.uv-version defects found by the extended matrix
The
[[distribution]]lock grammar (uv 0.1.45–0.2.34) went through three shapes, and the hosted rewriter keyed everything on the table name:source = "registry+…"strings[distribution.sdist]/[[distribution.wheel]]sub-tablessdist = {…}/wheels = [ … ]{ registry = … }The rewriter now decides the source shape and the artifact shape independently from the entry's own syntax; uv 0.2.35/0.2.36 locks without a root
[package.metadata]vendor natively instead of being refused.Second review pass (ultracode: 10 review dimensions, 3-lens adversarial verification, 4 real-binary probes)
Twenty-two findings survived verification; all are fixed here, each with a RED-first regression test. The two Cursor Bugbot findings are among them.
Correctness (uv end to end)
[tool.uv] dev-dependenciesis a direct dependency. It was classified transitive, wired throughoverride-dependencies, and its[package.metadata.requires-dev]entry left stale, souv sync --locked/uv lock --checkfailed on every uv ≥ 0.2.37 and a plainuv syncon 0.2.37/0.4.30 reinstalled the pristine wheel. Both the vendored classifier and the hostedrewrite_project_metadatanow count it.requires-dist/requires-deventry is repointed (dependencies + an extra, markers), not only the first —--lockedaccepted the lock only by accident before.[tool.uv] constraint-dependencies/build-constraint-dependencies: the lock's[manifest]constraints/build-constraintsentries are repointed (hosted and vendored), the shape uv ≥ 0.5.6 writes; a new wiring kind with a revert arm. uv 0.2.37–0.5.3 reject the repointed entry under--locked, so the repoint emits the advisorypypi_uv_constraints_require_uv_0_5_6.pypi_uv_override_requires_uv_0_5_6: uv applies[tool.uv.sources]tooverride-dependenciesonly from 0.5.6; on 0.2.35–0.5.3--frozeninstalls the patch but a plainuv syncreinstalls the registry wheel. No lock marker separates those releases, so this is an advisory, not a refusal.[[package]]unit, appended[manifest]/[package.metadata]fragments, and their revert); previously a CRLF pair came back LF / mixed.[[distribution]]refusal names the real reasons (probed with real binaries: through 0.2.6 a relative path source does not parse; on 0.2.17–0.2.34--lockedrejects it anduv lock/ plainuv syncabsolutize it) instead of "records absolute paths". The[package.metadata]no-op gate also considers[package.metadata.*]sub-tables.Symlinks (Bugbot finding 2 and its consequence)
pypi_lock::read_fileaccepted only regular files bylstat, so a symlinkedpylock.toml/*.py.lock— and, since discovery started following links, a stray non-referencing sibling symlink — refused vendoring of the whole project. It now reads through the link like every other reader.scan, vendored uv, python-lock) stages next to the path and renames over it, which replaced a symlinked lock with a detached regular file (uv itself writes through the link). Symlinked rewrite targets are now refused before any write —redirect_symlinked_file_unsupported(hosted, before the ledger is written),pypi_uv_symlink_unsupported,pypi_lock_symlink_unsupported— matching the hosted revert side, which already refused them.Unwired-revert guard (Bugbot finding 1 and three adjacent holes)
--preserve-state(keep_artifact): nothing is deleted, so nothing needs protecting (npm's precedent).-r/--requirementincludes ofrequirements.txtare probed, andrepair/ the orphan sweep see include-hosted pins too.Robustness
scan/get,repairand ledger-lessrollbackread candidate lockfiles through the FIFO-safe reader; a named pipe at a lockfile name no longer wedges the command inopen(2).uv exportis read from stdout (--output-fileonly exists from 0.4.7, so the 0.4.1 export lane had been recording a harness error as "unavailable"); uv 0.1.44 pinned to bracket theuv lockboundary;--render-doc-tableregenerates the doc's results tables fromresults.json.Probes that settled coverage questions (real binaries, evidence in the doc's Limits section)
[[distribution]]-era vendored native stays refused: ≤ 0.2.6 cannot parse a relative path source at all; 0.2.17–0.2.34 install one under--frozen/ plain sync but--lockedrejects it,uv lockand plain sync absolutize it, resolution is CWD-relative, and hashes are unverified before 0.2.34.name @ file:./…) breaks pip < 24.2, so the vendored requirements writer stays as is.--hashis silently ignored through 0.1.29;--require-hashesexists from 0.1.32; in-file hashes are verified by default only from 0.5.x. Documented.[[package]]release writes the bare[package.metadata]header even for dev-only projects, so the metadata no-op gate never fires for them.Validation
cargo clippy --workspace --all-features -- -D warningsclean;cargo test --workspace --no-fail-fastgreen (one pre-existing timing test in the update notifier flaked under the concurrent matrix load and passes on rerun).scripts/backtest-uv.py) on 42 uv releases spanning every family 0.0 through 0.12 at source commit17d0dcb: 1403 installed-byte comparisons, 1086 lock-preservation checks. Every mismatch (18) and lock change (32) is the documented uv < 0.5.6 boundary in the transitive and constraints variant lanes — from 0.5.16 every shape passes in both modes with the lock byte-unchanged; on 0.2.35–0.5.3--frozeninstalls the patch everywhere and the advisories name the limitation. Core lanes: 0 mismatches, 0 lock changes. The last commit after that run adds only the constraints advisory and the regenerated docs.[tool.uv] dev-dependencies,[dependency-groups], dependencies + extra duplicate,constraint-dependencies, and a transitive dependency on every[[package]]binary, hosted and vendored, with--frozen,--lockedand plain installs.Hosted mode covers requirements from uv 0.0.5 and native
uv.lockfrom 0.1.45 (the first release whoseuv lockwrites one); vendored native covers every[[package]]release (uv ≥ 0.2.35, with the ≥ 0.5.6 advisories for overrides and constraints); vendored requirements cover uv ≥ 0.1.24.🤖 Generated with Claude Code