diff --git a/CHANGELOG.md b/CHANGELOG.md index 8153865a..ac44cccc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -51,6 +51,75 @@ into the new version's section — see docs/releasing.md. ### Added +- **Python patches survive uv lockfiles in both hosted and vendored modes.** + `scan --mode hosted|vendored` now rewrites native `uv.lock` together with + the paired `pyproject.toml` source and metadata, PEP 723 script locks + (`*.py.lock` plus the script's inline metadata), PEP 751 `pylock*.toml`, + and uv-compiled hashed `requirements.txt`, so `uv sync --frozen|--locked`, + `uv run --script`, and `uv pip sync --require-hashes` install the patched + wheel instead of the registry artifact. Verified against real uv binaries + from every 0.x release family (0.0 through 0.12) — first and latest release + of each plus every observed behaviour boundary — see + `docs/testing/uv-compatibility.md`: hosted mode covers requirements from + uv 0.0.5 and native `uv.lock` from 0.1.45 — the first release whose + `uv lock` writes one — through all three `[[distribution]]` lock shapes + and `[[package]]`; vendored native covers every `[[package]]` release + (uv ≥ 0.2.35), vendored requirements cover uv ≥ 0.1.24 (hash-enforced by + `uv pip sync --require-hashes` from 0.1.32 and by default from 0.5.x). + Follow-up hardening: + `vendor --revert` refuses to delete a vendored Python wheel a lock still + references when the ledger entry has no wiring to replay (the shape + `repair` rebuilds), a script or PEP 751 lock supplements rather than hides + `poetry.lock`/`requirements.txt` pins, symlinked locks are discovered, and + refused before any write (never rewritten in place — uv writes through the + link, an atomic rename would replace it; hosted + `redirect_symlinked_file_unsupported`, vendored + `pypi_uv_symlink_unsupported` / `pypi_lock_symlink_unsupported`), CRLF + locks keep their line endings in both the hosted rewriter and the vendored + uv backend (`pyproject.toml`, the rewritten `[[package]]` unit and the + appended `[manifest]` / `[package.metadata]` fragments, plus their revert), + and the hosted `[tool.uv.sources]` edit renders as a header after + `[project]` the way uv writes it. (#238, #239) +- **uv `--locked` survives the project shapes the plain fixture never + reached.** A package listed only in `[tool.uv] dev-dependencies` is now + classified as a direct dependency (it was wired as a transitive override + and its `requires-dev` entry left stale), and every duplicate + `requires-dist` / `requires-dev` entry for the package — extras, markers — + is repointed rather than only the first, so `uv sync --locked` and + `uv lock --check` accept the patched lock instead of exiting 2 and a plain + `uv sync` no longer rewrites it. `[tool.uv] constraint-dependencies` / + `build-constraint-dependencies` naming the package have their `[manifest]` + `constraints` / `build-constraints` entries repointed too (uv ≥ 0.5.6 + serializes them with the package's source; 0.2.37–0.5.3 reject the + repointed entry under `--locked`, so the repoint emits the advisory + `pypi_uv_constraints_require_uv_0_5_6`). The transitive + (override-dependencies) branch emits the advisory + `pypi_uv_override_requires_uv_0_5_6`: uv applies `[tool.uv.sources]` to + overrides only from 0.5.6, so on 0.2.35–0.5.3 `--frozen` installs the + patch but a plain `uv sync` reinstalls the registry wheel. The + `[[distribution]]`-grammar vendoring refusal now names the real reasons + (relative path sources unparseable through 0.2.6, rejected by `--locked` + and absolutized by `uv lock` / `uv sync` on 0.2.17–0.2.34) instead of + "records absolute paths". `scripts/backtest-uv.py` gains a project-variant + lane covering these shapes on every `[[package]]` binary and a + `--render-doc-table` mode that prints the doc's results tables from + `results.json`; its export lane reads `uv export` from stdout because + `--output-file` only exists from 0.4.7. (#239) +- **Unwired Python vendor entries revert safely.** `vendor --revert` / + `rollback` on a ledger entry without wiring to replay (the shape `repair` + reconstructs) skips the lock-reference guard under `--preserve-state` + (nothing is deleted, so nothing needs protecting), refuses fail-closed when + the project root cannot be listed or a lock's symlink target cannot be + read (instead of treating "could not enumerate locks" as "no lock + references it" and deleting the wheel), probes `-r` / `--requirement` + includes of `requirements.txt` alongside the root file — the orphan sweep + and `repair` see include-hosted pins too — and reclaims a genuinely + orphaned artifact directory whose lock is gone or whose ledger flavor is + unknown instead of failing forever. Hosted `scan` / `get`, `repair`, and + ledger-less `rollback` read candidate lockfiles through the FIFO-safe + reader, so a named pipe at a lockfile name no longer wedges the command in + `open(2)`. (#239) + - **Path targeting on `scan` and `rollback`.** `scan [PATHS]...` scopes discovery to packages with an installed copy under a matching glob (ancestor rule: `scan packages/foo` covers the subtree; `*` never crosses diff --git a/crates/socket-patch-cli/src/commands/repair_vendor.rs b/crates/socket-patch-cli/src/commands/repair_vendor.rs index 3445d40b..be72ca41 100644 --- a/crates/socket-patch-cli/src/commands/repair_vendor.rs +++ b/crates/socket-patch-cli/src/commands/repair_vendor.rs @@ -15,9 +15,9 @@ //! from the lockfile path itself (the contract's uuid-in-path rule), the //! record from the manifest (or the patch API, yielding a detached entry), //! and a fresh ledger entry is re-synthesized so sweep/GC/revert know the -//! artifact again — stamped with the npm lockfile FLAVOR the reference was -//! found in, so a later `vendor --revert` routes to the backend whose -//! unwired-revert guard probes the right lockfile. WIRING reconstruction is +//! artifact again — stamped with the lockfile FLAVOR the reference was +//! found in (npm family and pypi), so a later `vendor --revert` routes to the +//! backend whose unwired-revert guard probes the right lockfile. WIRING reconstruction is //! per-ecosystem: gem recognizes //! its own Gemfile/lock wiring and rebuilds full revert-capable records //! ([`socket_patch_core::vendor::gem::reconstruct_gem_wiring`]); the other @@ -55,6 +55,7 @@ use socket_patch_core::api::client::{get_api_client_with_overrides, ApiClient}; use socket_patch_core::crawlers::CrawlerOptions; use socket_patch_core::manifest::schema::{PatchManifest, PatchRecord}; use socket_patch_core::patch::copy_tree::remove_tree; +use socket_patch_core::utils::fs::read_regular_to_string; use socket_patch_core::utils::purl::{ normalize_purl, percent_decode_purl_component, strip_purl_qualifiers, }; @@ -98,6 +99,8 @@ struct Candidate { /// Files the vendor backends rewire — the search space for /// `.socket/vendor///` references when the ledger is gone. +/// The Python locks the root LISTS (`pylock*.toml`, `*.py.lock` + script) +/// and the requirements `-r` include tree are appended at scan time. const WIRING_FILES: &[&str] = &[ "package-lock.json", "npm-shrinkwrap.json", @@ -143,10 +146,23 @@ pub(crate) async fn scan_vendor_references(project_root: &Path) -> Vec<(String, files.push(path); } } + // The requirements planner writes a vendored pin where the original pin + // was — possibly inside a `-r` include — so the root requirements.txt + // alone would miss it (and the orphan sweep, which reuses this scan, + // would delete the include-referenced wheel). An unreadable include + // tree degrades to the root file, matching the per-file tolerance + // below. + if let Ok(includes) = + socket_patch_core::vendor::requirements_include_names(project_root).await + { + files.extend(includes); + } files.sort(); files.dedup(); for file in files { - let Ok(text) = tokio::fs::read_to_string(project_root.join(file)).await else { + // FIFO-safe: a pipe under a wiring-file name must be skipped, not + // waited on forever in open(2). + let Ok(text) = read_regular_to_string(&project_root.join(file)).await else { continue; }; let mut rest = text.as_str(); @@ -221,8 +237,14 @@ fn synth_entry(eco: &str, uuid: &str, artifact_path: &str, base_purl: &str) -> V async fn detect_reference_flavor(project_root: &Path, eco: &str, uuid: &str) -> Option { if eco == "pypi" { let needle = format!(".socket/vendor/pypi/{uuid}/"); - for file in socket_patch_core::utils::python_lock::python_lock_paths(project_root).ok()? { - if tokio::fs::read_to_string(project_root.join(&file)) + let mut files = + socket_patch_core::utils::python_lock::python_lock_paths(project_root).ok()?; + // uv.lock outranks the standalone locks (the vendor backend's own + // precedence): a pylock EXPORTED from the wired project lock must not + // relabel the entry `python-lock`. Alphabetical order would. + files.sort_by_key(|file| file != "uv.lock"); + for file in files { + if read_regular_to_string(&project_root.join(&file)) .await .ok() .is_some_and(|text| text.contains(&needle)) @@ -244,7 +266,7 @@ async fn detect_reference_flavor(project_root: &Path, eco: &str, uuid: &str) -> } let needle = format!(".socket/vendor/npm/{uuid}/"); let read = |name: &'static str| async move { - tokio::fs::read_to_string(project_root.join(name)) + read_regular_to_string(&project_root.join(name)) .await .ok() }; @@ -1446,6 +1468,96 @@ fn npm_coords(base_purl: &str) -> Option<(String, String)> { mod tests { use super::*; + /// A FIFO under a wiring-file name (here the paired `