Support hosted and legacy Poetry patches - #241
Merged
Merged
Conversation
Wire hosted and vendored wheels across native Poetry lock formats. Preserve independent rollback and verify installer compatibility. Assisted-by: Codex:gpt-6-astra
Preserve the lock when a legacy patch fragment has drifted, so rollback does not restore upstream hashes while retaining a patched source. Assisted-by: Codex:gpt-6-astra
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Normalize checked-out fixtures before explicitly exercising LF and CRLF, so Windows auto-conversion cannot double carriage returns. Assisted-by: Codex:gpt-6-astra
Collaborator
Author
|
Claude (@claude) review |
Collaborator
Author
|
BugBot review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
1 issue from previous review remains unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 2ac2436. Configure here.
Wenxin Jiang (Wenxin-Jiang)
approved these changes
Sep 17, 2026
…s with uv - Never panic on a user-editable lock: a `[metadata.files]` / `[metadata.hashes]` value that is not a table (array of tables, scalar, `[[metadata]]`) used to hit toml_edit's IndexMut and abort `scan --mode hosted|vendored` with exit 101 and no JSON; every table access is now guarded and refuses with a warning. - Accept any `lock-version = "2.<n>"` in the shared rewriter (the vendored loader already accepted newer minors with an advisory), so the same lock no longer succeeds LF-vendored, hard-fails CRLF-vendored and silently no-ops hosted. - Supersede an earlier Socket hosted URL for the same wheel in place (same origin + filename, fragment ignored) instead of refusing it as a foreign source, so a grant-token rotation or republished patch does not strand the pin — the bun / requirements / cargo rewriters make the same call. Foreign sources and vendored file sources are still refused. - Hosted rewriter: gate the missing-SHA-256 warning once per dep (not per lock), warn `redirect_poetry_entry_not_found` when a lock has no entry at the granted version (uv parity), and emit `redirect_poetry_stale_install_risk` when the lock was written by Poetry < 1.4 — measured on real 0.12.17–2.4.3: those releases never replace an already-installed same-version package after the redirect, 1.4+ do. - Write the SHA-256 lowercase (Poetry compares hexdigest strings) and anchor the legacy integrity fragment at its line break so a suffix-named sibling entry with the same value cannot make the splice ambiguous. - Unit tests for the shared rewriter (it had none) and hosted integration tests for the new warnings, the 2.x minor, and token rotation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`pypi_poetry_integrity_unverified` fired for every lock-version 2.0 project, telling Poetry 1.4–1.8 users their installer verifies nothing. Lock 2.0 is written by 1.3 through 1.8; releases from 1.4 stamp `@generated by Poetry X.Y.Z`, so a stamped 2.0 lock is no longer flagged. The advisory now also names the second pre-1.4 behaviour measured on real releases: an already-installed same-version package is not replaced after the lock is rewired. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
scripts/backtest-poetry.py mirrors backtest-uv.py: it bootstraps every Poetry release (0.12.17 … 2.4.3) with uv, generates a native lock for a one-dependency project with a public free-tier patch, and drives the real CLI through hosted, vendored, agent and out-of-tree-venv agent mode plus `setup`, checking applied counts, idempotent re-scans, lock-driven installs into an emptied virtualenv, fresh-clone installs, `poetry check --lock`, `vex`, tampered-hash rejection, warm-virtualenv reinstall, Poetry's own relock, and `rollback` restoring every byte. 108 cases pass on the PR head; the results file feeds the docs table. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nd the matrix - CHANGELOG [Unreleased] entry for hosted + legacy Poetry support (the uv precedent had one; this PR did not). - poetry-compatibility.md rewritten around the measured matrix: per-generation rewrite shapes, the Poetry < 1.4 warm-virtualenv and hash-verification boundaries, which Poetry commands drop the patch source (with content-hash unchanged, so `poetry check --lock` cannot tell), the 0.12/1.0 cwd-relative file-source caveat, agent-mode discovery of Poetry's out-of-tree virtualenv, the lock-only vendored limitation, the harness recipe and the generated results table. - CLI_CONTRACT: poetry.lock joins the hosted candidate files; the vendored rows cover every lock generation; the new `redirect_poetry_*` and `pypi_poetry_integrity_unverified` codes are listed. - hosted-production-e2e.md no longer says Poetry locks are not rewritten by hosted mode; README no longer says hosted mode has no CLI revert (`rollback` unwinds it since the scan↔rollback duality change). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Poetry keeps a project's virtualenv OUTSIDE the project by default
(`{cache-dir}/virtualenvs/<name>-<hash>-py<X.Y>`), so after a plain `poetry
install` the crawler found no VIRTUAL_ENV / .venv / venv and fell through to
the global interpreter's site-packages: `scan --mode agent` patched nothing
for the project's dependencies (or the wrong interpreter) and reported
success, and a bare `rollback` pruned the manifest while the venv stayed
patched — measured on real Poetry 1.1.15, 1.8.5 and 2.4.3.
The crawler now reproduces Poetry's own placement without running Poetry:
`virtualenvs.create` / `in-project` / `path` and `cache-dir` from the
`POETRY_*` environment, the project's `poetry.toml` and the user
`config.toml` (Poetry's precedence), the platform default cache dir, the
`{cache-dir}` / `{project-dir}` / `~` placeholders, and
`EnvManager.generate_env_name` (lowercased sanitized name, 42-char cap,
first 8 chars of url-safe base64 sha256 of normcase(realpath(cwd))) —
unchanged from Poetry 1.0 through 2.x and pinned by known-answer vectors.
Every `-py<X.Y>` sibling is scanned. A project-local venv still wins; a
project that opted into in-project venvs or disabled creation is left to the
existing paths.
The matrix harness's out-of-tree leg now applies and rolls back BARE when the
crawler sees the venv (falling back to `poetry run` otherwise) and records
which path it took.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`scan --mode vendored` on a fresh clone (pyproject + poetry.lock, nothing installed) was skipped with `vendor_fetch_unverifiable` + `package_not_installed` although the lock records the wheel's sha256 — the poetry.lock inventory was discovery-only (`LockIntegrity::None`), so the fetch gate refused before the vendor engine ever ran, while the identical uv.lock scenario vendored fine. That broke the CI story for Poetry: the machine that vendors had to have the package installed. The inventory now carries the pure-Python (`-none-any.whl`) wheel's sha256 from `files` (lock 2.x) or `[metadata.files]` (lock 1.0/1.1), lowercased, and the pypi fetcher resolves a hash-only entry through PyPI's JSON API (`urls[].digests.sha256`, `SOCKET_PYPI_JSON_API` overrides the endpoint) and verifies the download against the same digest, exactly like uv's lock-only path. Poetry 0.12's bare `[metadata.hashes]` names no wheel, and platform-only wheels offer no platform-independent choice, so those stay discovery-only. Measured on the fixed CLI: lock-only vendoring now applies on Poetry 1.0 (populated lock), 1.2, 1.8 and 2.4 fixtures (`vendor_fetched_missing` + `vendor_prebuilt_downloaded`), and the resulting checkout installs the patched wheel with every release. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…endoring CHANGELOG Fixed bullets for the two pre-existing gaps the Poetry matrix surfaced (out-of-tree venv discovery, lock-only vendoring), the contract's pypi discovery-root sentence and `vendor_fetched_missing` row, and the compatibility doc's mode notes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…the pip window - A lock-version 1.0 hosted rewrite only carried the patched hash in `[metadata.files]` and the URL fragment. Poetry >= 1.2 (which still reads 1.0 locks) verifies url sources against the package's own `files` and ignores `metadata.files` hashes, so it installed the hosted wheel with no hash check at all (measured on 1.2.2: every hash tampered, install exit 0). Write the package `files` entry for 1.0 too — Poetry 1.0.10 ignores the extra key (measured), 1.2.2 then rejects a tampered hash. - Poetry 1.0 installs url sources through pip, reading the hash from the URL fragment that Poetry suffixes with `#egg=<name>`. Sweeping pip 20.3–25.0 with real Poetry 1.0.10: the `#sha256=<hex>&` spelling installs and verifies on pip <= 22.2 and >= 23.1; pip 22.3 and 23.0 (what Python 3.8's ensurepip seeds) take the rest of the fragment as part of the digest and refuse (fail-closed); without the `&` every pip >= 22.3 refuses. The spelling stays; the lock-1.0 hosted advisory now names the pip window and the compatibility doc records the sweep. - Pin each generation's hosted shape in `poetry_hosted` (1.0 fragment + reference + dual files, 1.1 dual files, 2.x metadata untouched) and drop duplicated lines from a vendored test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…a qualifier `scan --mode hosted --vex` exempts the purls the run just confirmed from installed-tree verification and attests them from the redirect ledger (`assume_applied`). The confirmed purls come from the grant reference unqualified (`pkg:pypi/urllib3@1.26.18`) while the ledger records the API's artifact-qualified purl (`…?artifact_id=py2-py3-none-any-whl`), so for pypi redirects nothing matched: a lock-only Poetry (or uv) checkout redirected the lock and then exited 1 with `no_applicable_patches`. Match on the qualifier-stripped purl on both sides. Adds the first CLI-level Poetry hosted test: a lock-only project is redirected, attested in the same run, re-scanned idempotently, and rolled back byte for byte against a mocked API. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`revert_lock_fragment_splice_atomic` held the write whenever ANY warning was raised, so a record this build cannot replay (a newer ledger's unknown kind, or a foreign file skipped by the allowlist) blocked restoring the fragments it does understand — the lock stayed fully wired while the revert reported success. Track drift separately: a drifted fragment still holds the whole write (a half-restored legacy lock is uninstallable), forward-compat skips only warn. Also documents that `poetry lock --no-update` on Poetry 1.1/1.2 reshapes a hosted entry and drops the package-level files Poetry >= 1.2 verifies against. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
108/108 cases pass on the fix-branch head. The rerun records the fixes' effects: lock-only vendoring applies wherever the lock names a pure-Python wheel hash (1.0/1.1 populated, 1.2+), the out-of-tree agent leg applies and rolls back BARE on every 1.x/2.x release (the crawler finds Poetry's venv), and the pre-1.4 hosted advisory fires for 1.0–1.3 locks only. The table renderer reports the lock-only column per shape. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two defects with one root cause — the recorded `[[package]]` fragment ended at
the unit's last line while the rewrite APPENDS `[package.source]` after it,
so the pristine fragment was a prefix of every rewritten (or relocked) unit:
- Replay's "already converged" check (`content.contains(original) &&
!new.contains(original)`) fired when a relock (Poetry 1.1/1.2 `poetry lock
--no-update`) or a hand edit dropped the inserted `files` line but kept the
Socket source block: rollback reported success, deleted the ledger and
restored `[metadata.files]` to upstream hashes while the lock still
redirected — an inconsistent lock with no record left (lock 1.1 and 2.x).
For lock 1.0 the guard's other arm meant a hand-restored pristine lock could
never converge and rollback refused with a spurious drift.
- A re-scan after such a relock appended edits recorded against the RELOCKED
text; replay inverted the newer links back to the relocked state and then
found neither `new` nor `original` of the older ones, so `rollback` and
`remove` refused forever — and the refusal's remedy ("re-run scan") was
what lengthened the chain (reproduced with real Poetry 1.1.15 and 1.2.2).
The fragment now extends through the next top-level header (`[[package]]`,
`[metadata]`, …), so a unit that grew a source block never contains the
pristine fragment: the drifted shape is refused, the hand-restored lock
converges. The hosted ledger merge REBASES `redirect_poetry_lock_package`
edits whose (path, kind, key) already exists when the pre-run file no longer
carried their `new` fragments — keeping the oldest `original`, adopting the
fresh `new` — instead of appending, so one invertible link survives any
number of relock/re-scan rounds. Idempotent re-scans and token-rotation
supersedes still append as before.
Tests: fragment boundary property per generation and for adjacent units;
dropped-files-line-with-source-kept is refused not converged (1.1, 2.x);
lock-1.0 hand-restored lock converges; CLI relock → re-scan → rollback chain
lands on the pristine lock with a two-edit ledger.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
When Poetry's own relock keeps the patch source but re-lays the unit (1.1/1.2), the harness now re-scans instead of restoring the CLI's lock and lets the final rollback prove the rebased ledger still lands on pristine bytes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… setup's Poetry 108/108 cases on the fix-branch head (0.12.17 … 2.4.3 × hosted / vendored / agent / out-of-tree agent / setup × direct / populated / crlf / pep621), including the live relock → re-scan → rollback chain on Poetry 1.1 and 1.2. The `setup` leg now hands the CLI-spawned `poetry lock` the case-isolated HOME as well (Poetry <= 1.1's shared HTTP-cache lock wedged one parallel run). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Collaborator
Author
Review follow-ups (2026-09-17)Verified end-to-end against real Poetry 0.12.17, 1.0.10, 1.1.15, 1.2.2, 1.3.2, 1.4.2, 1.5.1, 1.6.1, 1.7.1, 1.8.5, 2.0.1, 2.1.4, 2.2.1, 2.3.4 and 2.4.3 with Fixes on top of the original three commits:
Known, not changed here: 🤖 Generated with Claude Code |
Preserve LF/CRLF in the relock simulator and exercise both newline forms. Probe installed Python package hashes after hosted redirects, warn on stale bytes, and exclude them from same-run VEX even when another interpreter is patched. Cover re-scans, ledger fallback, missing files, dry runs, variants, and custom prefixes. Assisted-by: Codex:gpt-6-astra
Treat ERROR_ACCESS_DENIED from an open racing the deliberate lock-file deletion as a benign test interleaving on Windows. Retry within the existing bound and still require a clean mutually exclusive acquisition. Production lock acquisition and error handling are unchanged. Assisted-by: Codex:gpt-6-astra
Mikola Lysenko (mikolalysenko)
deleted the
codex/poetry-patch-compat
branch
September 17, 2026 22:19
Mikola Lysenko (mikolalysenko)
added a commit
that referenced
this pull request
Sep 18, 2026
…a qualifier `scan --mode hosted --vex` exempts the purls the run just confirmed from installed-tree verification and attests them from the redirect ledger (`assume_applied`). The confirmed purls come from the grant reference unqualified (`pkg:pypi/urllib3@1.26.18`) while the ledger records the API's artifact-qualified purl (`…?artifact_id=py2-py3-none-any-whl`), so for pypi redirects nothing matched: a lock-only Pipenv checkout redirected the lock and then exited 1 with `no_applicable_patches`. Match on the qualifier-stripped purl on both sides. Measured on a lock-only Pipenv 2026.8.0 checkout: exit 1 / 0 statements before, exit 0 / 1 `not_affected` statement after. Same change as b7a4254 on fix/poetry-compat-review (PR #241); identical so the two merge cleanly. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko)
added a commit
that referenced
this pull request
Sep 18, 2026
Conflicts resolved: the crawler runs Poetry's then Pipenv's out-of-tree venv discovery (both gated on nothing found above); replay keeps the PipenvEntry inverse next to main's HatchDocument; the hosted confirmed-uuid filter confirms Pipfile.lock rewrites through `confirmed_pipenv_uuids` alongside main's python-lock / hatch / requirements sets and still vetoes `refused_pipenv_uuids`; main's generic Python stale-install guard (`redirect_pypi_stale_install`, `known_stale`) replaces the Pipenv-specific hosted probe — it now judges only the project's own venvs (no global interpreter fallback) and names the Pipenv-specific remedy (`pipenv run pip uninstall -y <pkg> && pipenv sync`, never `pipenv uninstall`) for Pipfile.lock redirects; the pypi fetcher keeps the digest-set resolver (superset of main's single-hash one); the candidate-file list, the ecosystem matrix row, the hosted-e2e note, CHANGELOG and CLI_CONTRACT carry both sides once. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LLM Description written by Codex:gpt-6-astra
Poetry projects can now install hosted patches on Poetry 1.x/2.x and vendored patches across Poetry 0.12/1.x/2.x. Hosted scans previously missed
poetry.lock, and vendoring refused legacy lock formats and CRLF files.The shared rewriter preserves dependency metadata and the pyproject content hash, supports legacy hashes/files and modern package files, and records independent rollback fragments. Either of two patches can be removed first while unrelated edits survive. Source and integrity edits stay together when rollback encounters drift; re-scanning after Poetry relocks rebases the rollback fragments.
The Python crawler also discovers Poetry's out-of-tree virtualenvs. Lock-only vendoring uses the recorded pure-Python wheel digest to find and verify the upstream artifact. Unsupported formats, ambiguous package forks, foreign sources, and mismatched wheels are refused.
Hosted redirects now probe installed Python file hashes. Readable upstream or modified bytes produce
redirect_pypi_stale_installand are excluded from same-run VEX, including with--vex-no-verifyor a healthy copy in another interpreter. The check is read-only, repeats on re-scans, and falls back to persisted records if fetching fresh records fails. Lock-only checkouts retain their lock-based redirect attestations.Compatibility boundaries
socket-patch vex.Validation
cargo clippy --workspace --all-features -- -D warningspassed.106645c41c687823719a40c9eaba90a8f2dd156b, including Linux/macOS/Windows, release tests, coverage, and the dependent installer suites (CI run).Native lock fixtures, the live backtest harness, captured results, and compatibility documentation are included. The depscan stack is SocketDev/depscan#26287 → core Poetry support #26348 → tests and fixtures #26358.
Note
Medium Risk
Changes how
poetry.lockis rewritten for installs (URLs, hashes, integrity tables); mistakes could break Poetry installs, though inputs are validated and rollback is fragment-based with drift guards.Overview
Hosted PyPI patches now rewrite
poetry.lock(included in hosted scans) to point patched wheels at Socket patch URLs, with recordedredirect_poetry_lock_packagefragments that roll back with other PyPI redirect edits. Poetry 0.12 URL redirects are refused because that installer ignores URL sources.A shared
poetry_lockrewriter drives both hosted URL wiring and vendored local file sources across legacy (metadata.hashes/metadata.files) and modern lock formats, preserves pyproject content-hash, handles CRLF locks, and refuses ambiguous forks, foreign sources, and bad wheels/SHA-256.Vendored Poetry expands beyond 2.x-only to 0.x/1.x/2.x (with integrity warnings before Poetry 1.4). Legacy and CRLF paths use the shared rewriter; atomic revert skips partial writes when lock fragments have drifted so source and hash splices stay consistent.
Fixtures and tests cover many Poetry versions, idempotent redirect, independent multi-patch rollback, and updated docs.
Reviewed by Cursor Bugbot for commit 2ac2436. Configure here.