Skip to content

Use PyPI trusted publishing, pin the e2e PDP, add latest and cloud PDP jobs - #136

Draft
zeevmoney wants to merge 23 commits into
per-16222/strict-toolingfrom
per-16676/release-ci-hardening
Draft

zeevmoney wants to merge 23 commits into
per-16222/strict-toolingfrom
per-16676/release-ci-hardening

Conversation

@zeevmoney

@zeevmoney zeevmoney commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Linear issues

  • PER-16676: Harden the release and CI workflows: trusted publishing, release-tag protection, pinned PDP, latest-PDP and cloud-PDP e2e jobs.

Stacked on #128.

Why

  • Releases uploaded to PyPI with a long-lived API token stored as a repository secret.
  • The required e2e jobs ran against permitio/pdp-v2:latest, so a new PDP release could fail every pull request.
  • No CI job ran the cloud-PDP tests in tests/test_abac_pdp.py. They assumed the cloud PDP answers 501 to ABAC calls, which it no longer does, and set up no policy of their own.
  • Three gaps on the release path:
    • The release scan could take its Trivy binary from the Actions cache.
    • Nothing checked that only the permit package ships.
    • No job had a timeout.

What changed

Release workflow (.github/workflows/python-sdk-publish.yml)

  • Publish to PyPI uploads by PyPI trusted publishing (OIDC).
    • It passes no password, keeps environment: pypi, and its only permission is id-token: write.
    • The zizmor: ignore[use-trusted-publishing] and its TODO are removed.
    • The job comment names the three things PyPI matches: repository, workflow file and environment. It also says PyPI does not check the ref, so the pypi environment's deployment rules are what limit uploads to release tags.
  • Security Gate runs the Trivy action with cache: false, so every release downloads Trivy and checks its checksum.
  • Build distribution, step "Check the wheel and sdist contents", also fails when:
    • the wheel's top level holds anything other than permit/ and permit-<version>.dist-info, or
    • the sdist holds a directory other than permit/.
  • Timeouts: build 10, scan 15, publish 10 minutes.

Already in place and unchanged:

  • One publish path: build → scan → publish, joined by needs:. The scan gates the publish and keeps its report for 90 days.
  • py.typed and _sync_types.pyi are asserted in both artifacts.
  • The tag is validated as a whole string, passed through env.
  • The workflow triggers on release: published.
  • No setup-uv cache, and the build uv is pinned by version and checksum.
  • persist-credentials: false, top-level contents: read, ubuntu-24.04, and SHA-pinned actions.

CI (.github/workflows/test.yml, .github/workflows/security.yml)

  • Every PR gets the full checks. The Test and Security workflows ran only on PRs into main/master, so a stacked PR, whose base is another PR's branch, got no tests, dependency audit or workflow checks. The base filter is dropped. Push runs are unchanged.
  • Pinned PDP. The required pytest jobs run PINNED_PDP_IMAGE, permitio/pdp-v2:0.9.16@sha256:e3cf30794ec2d256636b4714641df46e51ee58a3f1f0d24c606e214e0bf8669a, the multi-arch index digest. Job name, matrix and step names are unchanged.
  • New job e2e-unpinned-pdp. It is not a required check and starts after both pytest lanes pass. It has two legs:
    • e2e (latest PDP image) runs the whole suite on pydantic 2 against permitio/pdp-v2:latest and logs the digest :latest resolved to.
    • e2e (cloud PDP) runs tests/test_cloud_pdp_e2e.py with PDP_URL=https://cloudpdp.api.permit.io and no container. It fails if any test is skipped, which a check on the junit report enforces.
  • Each new leg creates its own scratch environment, python-sdk-ci-<run_id>-<run_attempt>-<leg>. It masks the key before export and deletes the environment in an always() step. The leg keeps the 300 s PDP wait and the filtered PDP log, and has the same permissions, pinned actions, persist-credentials: false and env-passing as pytest.
  • compatibility job. Its artifact check is now the same script as the release check.
  • Timeouts: pytest 30, compatibility 15, e2e-unpinned-pdp 30 minutes.

Tests

  • tests/test_abac_pdp.py is replaced by tests/test_cloud_pdp_e2e.py. It creates its own resource type (actions read and write), a role granting read, two tenants, a user and a role assignment, waits until the cloud PDP allows read, then asserts exact decisions from check, bulk_check, get_user_permissions and filter_objects, including the denials.
  • A new offline test checks that a PDP answering 501 to check, get_user_permissions or filter_objects makes the SDK raise PermitConnectionError with the status in the message.

Docs

  • CONTRIBUTING.md has a new "Releasing" section.
  • "End-to-end tests" now describes:
    • the four e2e jobs;
    • how to reproduce the required jobs on the pinned image;
    • the cloud-PDP command;
    • "Moving the PDP pin".

Behaviour changes

  • Releases upload through OIDC and no longer read PYPI_TOKEN. The trusted publisher must be registered on pypi.org first.
  • The release scan no longer restores Trivy from the Actions cache.
  • A release, or a pull request's compatibility run, fails if the wheel or sdist ships anything besides permit. Today's build passes.
  • The required e2e jobs run PDP 0.9.16. On 2026-09-28 that was the same digest as :latest.
  • There are two new non-required check runs. A Test run can create up to four scratch environments, two at a time.
  • The build, scan, publish, pytest, compatibility and new e2e jobs have timeouts.
  • e2e (cloud PDP) passes only if the cloud PDP returns the expected allow and deny decisions for the policy the tests create.

How it was tested

  • uv lock --check passes.
  • pre-commit run --all-files: all hooks pass.
  • mypy reports no issues in 89 files, on both pydantic 2 and pydantic 1 (1.10.26).
  • pytest -m "not e2e": 308 passed, 3 skipped, no warnings, on pydantic 2 and on pydantic 1.
  • actionlint reports nothing.
  • zizmor .github/ with online audits reports no findings and no ignores. Before this PR it had one ignore, for trusted publishing.
  • Check names. Expanding every matrix gives 20 check runs before and 22 after. None were removed; the two added are e2e (latest PDP image) and e2e (cloud PDP). Both required pytest (Pydantic …) contexts are unchanged.
  • Cloud-PDP selection. pytest --collect-only collects the 3 tests. --setup-plan -rs skips all 3 with PDP_URL=http://localhost:7766 and skips none with PDP_URL=https://cloudpdp.api.permit.io.
  • 501 handling (offline). One test per method (check, get_user_permissions, filter_objects) against pytest-httpserver answering 501. Six mutations of the enforcer, treating 501 as success or dropping the status from the message, each fail exactly the case they target.
  • No-skip gate. On a junit report with 3 skipped tests it exits 1; with 12 tests run it exits 0. A mutant without the check passes the skipped report.
  • Contents check. It passes a local build of this branch and fails a wheel with a planted tests/ package. A mutant without the exit lets that wheel through. The release and compatibility copies are identical.
  • PDP digest. The Docker Hub tag API returns the pinned digest for 0.9.16 as an OCI image index.
  • Not run locally. The e2e suite needs CI secrets; it runs in this PR's CI. e2e (cloud PDP) is not a required check.

Owner actions before merge

  1. On pypi.org, under project permit, Manage, Publishing, add a GitHub Actions trusted publisher: owner permitio, repository permit-python, workflow python-sdk-publish.yml, environment pypi.

  2. Create a tag ruleset that lets only admins create, move or delete tags matching refs/tags/v[0-9]* and refs/tags/[0-9]*. Save the JSON below as release-tag-ruleset.json and run gh api repos/permitio/permit-python/rulesets -X POST --input release-tag-ruleset.json.

    release-tag-ruleset.json
    {
      "name": "Release tags: admins only",
      "target": "tag",
      "enforcement": "active",
      "bypass_actors": [
        { "actor_id": 5, "actor_type": "RepositoryRole", "bypass_mode": "always" },
        { "actor_id": 1, "actor_type": "OrganizationAdmin", "bypass_mode": "always" }
      ],
      "conditions": {
        "ref_name": {
          "include": ["refs/tags/v[0-9]*", "refs/tags/[0-9]*"],
          "exclude": []
        }
      },
      "rules": [
        { "type": "creation" },
        { "type": "update", "parameters": { "update_allows_fetch_and_merge": false } },
        { "type": "deletion" }
      ]
    }
  3. Protect the pypi environment:

    • allow deployments only from tags matching v[0-9]* and [0-9]*;
    • require a release reviewer;
    • turn off admin bypass.

After the first release published by OIDC succeeds, delete the PYPI_TOKEN secret and revoke that token on pypi.org.

🤖 Generated with Claude Code

zeevmoney and others added 18 commits October 1, 2026 05:00
The publish job authenticated with the long-lived PYPI_TOKEN secret.
It now passes no password, so pypa/gh-action-pypi-publish exchanges the
job's OIDC token for a short-lived upload token. The zizmor ignore for
use-trusted-publishing and its TODO are gone.

PyPI must list permitio/permit-python, workflow python-sdk-publish.yml
and environment pypi as a trusted publisher of the permit project
before this is released, or the upload is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trivy-action caches by default, and on a cache hit setup-trivy puts the
cached Trivy binary on PATH without checking it. The release gate then
scans with whatever binary the Actions cache held. With the cache off,
each release downloads Trivy and checks it against its release's
checksums, as the build and scan jobs already do for uv.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pytest jobs ran permitio/pdp-v2:latest, so a new PDP release could
fail a required check with no change in this repository. They now run
0.9.16 by the digest of its multi-arch index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both ran with GitHub's default limit of six hours. A run of pytest takes
5-6 minutes and one of compatibility under one, so 30 and 15 minutes
leave room for a slow PDP start or rate-limit retries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The build job checked that the artifacts carry py.typed and
_sync_types.pyi, but not what else they hold. permit 2.8.3's wheel
installed a top-level `tests` package into site-packages. The same step
now also fails when the wheel's top level holds anything but permit/
and its own .dist-info, or the sdist holds a directory other than
permit/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The build, scan and publish jobs had none, so a hung step would hold
the release for GitHub's six-hour default. Each usually finishes in
under a minute; they now stop after 10, 15 and 10 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new job, e2e-unpinned-pdp, which is not a required check, runs after
both pytest lanes pass, with a scratch environment per leg:

- e2e (latest PDP image): the suite against permitio/pdp-v2:latest on
  pydantic 2, so a PDP release that breaks the SDK shows up before the
  pin moves to it. It prints the digest :latest resolved to.
- e2e (cloud PDP): tests/test_abac_pdp.py against the hosted cloud PDP.
  Those three tests skip everywhere else, so until now no CI job ran
  them. The leg fails if any of them is skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in per-16676/impl-publish: trusted publishing for the PyPI
upload, no Trivy cache in the release scan, a check that the wheel and
sdist ship only permit, and a timeout on each release job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in per-16676/impl-ci: the required e2e jobs run a PDP image
pinned by version and digest, two jobs that are not required checks
run the e2e tests on the latest PDP image and on the cloud PDP, and
the pytest and compatibility jobs get timeouts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The compatibility job in test.yml now runs the same artifact check as
the release build, so a packaging change that ships a package beside
permit fails a pull request, not the next release. The two scripts are
identical, and each workflow points at the other.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The comment read as if PyPI took uploads for permit only from this
publisher, which is not so while a project API token exists. It now
says what makes this job's own upload accepted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CONTRIBUTING.md gets a Releasing section: which tags the release
workflow accepts, what each of its three jobs checks, that the upload
uses PyPI trusted publishing, and which names on pypi.org must change
with the workflow file or the environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CONTRIBUTING.md now says which e2e jobs CI runs and which are
required, how to reproduce them locally on the pinned PDP image, the
latest one or the cloud PDP, and how to move the PDP pin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The module comment said CI only ever skips these tests. The new
e2e (cloud PDP) job runs them and fails if any is skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three tests in tests/test_abac_pdp.py passed on any
PermitConnectionError, which the SDK also raises for a rejected key, a
server error and a PDP it cannot reach. The e2e (cloud PDP) job was
then green without the cloud PDP ever answering. Each test now
requires the status code 501 in the error message, and the workflow
comment and CONTRIBUTING.md say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The text said three, but the list under it names four: the two pytest
lanes, e2e (latest PDP image) and e2e (cloud PDP).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PyPI trusted publishing matches the repository, the workflow file name
and the environment, not the ref, so a branch that edits the publish
workflow to run on push could upload. The publish job's comment and
the Releasing section now say that the pypi environment's deployment
rules, a repository setting, are what prevent that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both workflows ran only on PRs into main or master. A stacked PR, whose
base is another PR's branch, got no tests, no dependency audit and no
workflow checks until it was retargeted. Drop the base filter so every
PR gets the same checks before it merges. Push runs are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Oct 1, 2026

Copy link
Copy Markdown

PER-16676

@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown

Dependency Security Audit

Scanned: pyproject.toml dependencies + dev group, resolved at Python 3.10 (the current resolution, and the lowest versions the published specs permit under each pydantic major)

✅ No known vulnerabilities found.

Both the resolved dependency set and the lowest versions the published specs permit are clean at HIGH and CRITICAL.

zeevmoney and others added 5 commits October 1, 2026 12:53
A PDP that does not implement check, get_user_permissions or
filter_objects answers 501. The SDK must then raise
PermitConnectionError with that status in the message, not return a
decision. The cloud PDP tests asserted this against the hosted PDP,
which now answers these calls. This test keeps the contract covered
against a local pytest-httpserver PDP.

The status is matched where each message reports it, since the message
also holds the PDP's URL, whose random port can contain 501.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cloud PDP tests expected the hosted PDP to answer check,
get_user_permissions and filter_objects with 501. It now answers them
with 200, so all three failed the first time CI ran them.

tests/test_cloud_pdp_e2e.py replaces tests/test_abac_pdp.py. Each test
creates a small RBAC policy with per-run keys in the scratch
environment: a resource type with two actions, a role that grants one
of them, a tenant where the user has that role and a tenant where it
has none. It waits, with a bounded poll, until the cloud PDP allows the
granted action, then asserts the exact answers of check, bulk_check,
get_user_permissions and filter_objects. Teardown deletes every object
it created and treats a 404 as success.

The module still skips unless PDP_URL is the cloud PDP. The
e2e (cloud PDP) job now runs the new path and still fails if any of
its tests is skipped. The workflow comments and CONTRIBUTING.md
describe what the job now tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The cloud PDP lists its built-in "tenant-association" role after the
roles assigned to a user who belongs to the tenant. The first CI run
against it returned ["<role>", "tenant-association"] on every poll, so
the expected value now includes it. check, bulk_check and
filter_objects already matched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The invites target a resource instance, and the API now refuses to
approve an invite whose role belongs to another resource (PER-15743).
The test gave them a tenant role; it now creates a role on the invited
resource and deletes it before the resource.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The PDP can allow a check before its list of role assignments shows
the grant, so the tests read that list once and sometimes saw none.
They now poll for it, as they do for decisions, before asserting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant