Migrate end-to-end tests to pytest Uranium

.agents/skills/apply-pr-comments/SKILL.md

@@ -11,5 +11,5 @@ should drive code changes and which should be answered without changes.
 - Do not post replies on GitHub. Propose suggested replies in your report.
 - Apply code changes for valid review feedback.
 - Build using `../build-pv/SKILL.md`.
-- Run relevant tests via `../run-unit-tests/SKILL.md` and `../run-autests/SKILL.md` when applicable.
+- Run relevant tests via `../run-unit-tests/SKILL.md` and `../run-urtests/SKILL.md` when applicable.
 - Do not commit or push. Leave changes unstaged for manual review.

.agents/skills/plan-issue/SKILL.md

@@ -8,8 +8,8 @@ description: Read a GitHub issue with gh and produce an implementation and verif
 Using `gh`, read the issue and produce an implementation plan.
 
 - Understand the issue description and inspect relevant code paths.
-- Plan tests for the fix (Catch2 and/or autest as appropriate). See `../writing-autests/SKILL.md`.
+- Plan tests for the fix (Catch2 and/or Uranium as appropriate). See `../writing-urtests/SKILL.md`.
 - Plan the production code changes.
-- Include build and verification steps using `../build-pv/SKILL.md` and `../run-autests/SKILL.md`.
+- Include build and verification steps using `../build-pv/SKILL.md` and `../run-urtests/SKILL.md`.
 
 Unless asked otherwise, do not post comments to the issue.

.agents/skills/run-urtests/SKILL.md

@@ -0,0 +1,46 @@
+---
+name: run-urtests
+description: Run Proxy Verifier end-to-end tests with the build-generated pytest wrapper.
+---
+
+# Uranium Tests
+
+Uranium tests are pytest-based end-to-end tests. Run them through the
+build-generated `urtest.sh` so the correct binaries, plugins, and sandbox are
+selected. Tests are ordinary `test_*.py` modules under `tests/uranium_tests`.
+
+# Run Uranium Tests
+
+After building (see `../build-pv/SKILL.md`), run tests from the build tree:
+
+```bash
+
+if [ "`uname`" = "Linux" ]
+then
+  num_threads=$(nproc)
+else
+  num_threads=$(sysctl -n hw.logicalcpu)
+fi
+./build/dev/urtest.sh -n "${num_threads}"
+```
+
+An optional `--sandbox /tmp/sbpv` can be used to specify a custom sandbox
+location rather than the default. The sandbox contains process logging output
+which can help diagnose process behavior. Passed-case sandboxes are removed;
+failed-case sandboxes are retained.
+
+The `-n` option runs tests in parallel. Omit it for a sequential run.
+
+Use pytest's `-k` option to select tests by name. For example, to run the HTTPS
+and HTTP/2 modules:
+
+```bash
+./build/dev/urtest.sh -k 'https or http2'
+```
+
+The `-k` and `-n` options can be combined when you want a smaller parallel
+smoke test:
+
+```bash
+./build/dev/urtest.sh -n 4 -k 'http or https'
+```

.agents/skills/writing-urtests/SKILL.md

@@ -0,0 +1,45 @@
+---
+name: writing-urtests
+description: Guidance for adding Proxy Verifier pytest-based end-to-end tests.
+---
+
+# Write Uranium Tests
+
+Uranium is Proxy Verifier's pytest-based end-to-end test framework.
+
+- Put tests under `tests/uranium_tests` in ordinary `test_*.py` modules.
+- Use `CaseSuite` and the process helpers from `tools.uranium`.
+- Treat replay YAML, JSON, and gold files as data referenced by Python tests;
+  they are not collected directly.
+- Use existing modules under `tests/uranium_tests` for examples.
+- Run tests through the build-generated `urtest.sh` wrapper described in
+  `../run-urtests/SKILL.md`.
+
+## Process Expectations
+
+Register output expectations through the explicit stream API. Both regex
+methods require an explanation that describes the intended behavior and is
+reported when the expectation fails.
+
+```python
+server.stdout.contains(
+    "Ready with 3 transactions",
+    "The server should parse all three transactions.",
+)
+server.stdout.excludes(
+    "Violation:",
+    "The server should not report verification errors.",
+)
+server.stdout.matches_gold("gold/server.gold")
+```
+
+Use `server.stdout.reset()` to intentionally discard all expectations already
+registered for that stream. The reset affects expectations only; it does not
+change the captured output path.
+
+The `stdout` and `stderr` properties are read-only. Do not use `=` or `+=` to
+register expectations. Configure acceptable process exit statuses explicitly:
+
+```python
+client.expect_return_codes(0, 1)
+```

.dockerignore

@@ -2,5 +2,5 @@
 .github
 .venv
 build
-tests/autests/.venv
+tests/.venv
 **/__pycache__

.github/workflows/ci.yml

@@ -52,7 +52,7 @@ jobs:
             --input-exclude "**/*.lock" \
             --input-exclude "**/*.md" "**/*.json" "**/*.yaml" "**/*.gold" "**/*.pem" \
             --input-exclude "tests/unit_tests/catch.hpp" \
-            --input-exclude "tests/autests/gold_tests/autest-site/default_url_file" \
+            --input-exclude "tests/tools/uranium/default_url_file" \
             --input-exclude-std GIT HIDDEN_DIR HIDDEN_FILE -- \
             "${GITHUB_WORKSPACE}" | tee "${RUNNER_TEMP}/RAT.txt"
 
@@ -113,8 +113,8 @@ jobs:
           path: build/dev/Testing
           if-no-files-found: ignore
 
-  autests:
-    name: AuTests (Ubuntu 26.04, GCC)
+  urtests:
+    name: Uranium tests (Ubuntu 26.04, GCC)
     runs-on: ubuntu-24.04
     timeout-minutes: 60
     container:
@@ -127,28 +127,28 @@ jobs:
         uses: actions/checkout@v6
 
       - name: Configure
-        run: cmake --preset dev
+        run: cmake --preset dev -DURTEST_SANDBOX=/tmp/proxy-verifier-urtest
 
       - name: Build
         run: cmake --build --preset dev --parallel
 
-      - name: Run AuTests
-        run: ./build/dev/autest.sh -v --clean=none -j"$(nproc)"
+      - name: Run Uranium tests
+        run: ./build/dev/urtest.sh -v -n "$(nproc)"
 
-      - name: Collect AuTest sandbox
+      - name: Collect Uranium sandbox
         if: failure()
         run: |
           mkdir -p artifacts
-          if [[ -d /tmp/proxy-verifier-autest ]]; then
+          if [[ -d /tmp/proxy-verifier-urtest ]]; then
             tar --create --gzip \
-              --file="${GITHUB_WORKSPACE}/artifacts/autest-sandbox.tar.gz" \
-              --directory=/tmp proxy-verifier-autest
+              --file="${GITHUB_WORKSPACE}/artifacts/urtest-sandbox.tar.gz" \
+              --directory=/tmp proxy-verifier-urtest
           fi
 
-      - name: Upload AuTest sandbox
+      - name: Upload Uranium sandbox
         if: failure()
         uses: actions/upload-artifact@v6
         with:
-          name: autest-sandbox
-          path: artifacts/autest-sandbox.tar.gz
+          name: urtest-sandbox
+          path: artifacts/urtest-sandbox.tar.gz
           if-no-files-found: ignore

.gitignore

@@ -67,7 +67,7 @@ share/
 
 *.lock
 !uv.lock
-!tests/autests/uv.lock
+!tests/uv.lock
 
 # vim temporary files
 *.sw[a-p]

AGENTS.md

@@ -16,9 +16,9 @@ existing project style and preserve current CLI and YAML behavior.
   as compatibility-sensitive.
 - Update `README.md` when behavior, flags, workflows, or replay-file semantics
   change.
-- For new features and bug fixes, always consider whether unit tests and AuTests
-  should be added or updated. See .agents/skills/write-autests/SKILL.md for how to
-  write AuTests.
+- For new features and bug fixes, always consider whether unit tests and
+  Uranium tests should be added or updated. See
+  `.agents/skills/writing-urtests/SKILL.md` for how to write Uranium tests.
 
 ## Build And Test
 
@@ -32,14 +32,14 @@ cmake --build --preset dev --parallel
 ctest --preset dev
 ```
 
-For end-to-end coverage, use the generated AuTest wrapper from the build tree:
+For end-to-end coverage, use the generated Uranium wrapper from the build tree:
 
 ```bash
-./build/dev/autest.sh
+./build/dev/urtest.sh
 ```
 
-When a change is localized, prefer running the relevant AuTest subset with
-`-f ...` instead of the full suite.
+When a change is localized, prefer running the relevant Uranium subset with
+`-k ...` instead of the full suite.
 
 ## Formatting And File Structure
 
@@ -55,10 +55,10 @@ When a change is localized, prefer running the relevant AuTest subset with
   it.
 - Almost all new files should contain the Apache license preface with this year
   referenced as the copyright, except of course files that cannot contain such
-  comments, such as `.json` files and autest `.gold` files.
-- Replay YAML files, including AuTest replay files under `tests/autests`, also
-  need the same commented file prologue (`# @file`, copyright, SPDX). Do not
-  add new replay YAMLs without that header.
+  comments, such as `.json` and `.gold` files.
+- Replay YAML files, including Uranium replay files under
+  `tests/uranium_tests`, also need the same commented file prologue (`# @file`,
+  copyright, SPDX). Do not add new replay YAMLs without that header.
 
 ## C++ Style
 
@@ -130,7 +130,7 @@ do_something(!IGNORE_CASE);
   naming with descriptive behavior-oriented titles.
 - Table-driven tests and designated initializers are already used and are good
   patterns to continue when they improve clarity.
-- End-to-end behavior belongs in AuTests under `tests/autests/gold_tests`.
+- End-to-end behavior belongs in Uranium tests under `tests/uranium_tests`.
 - For parser, protocol, replay, CLI, or compatibility fixes, prefer adding a
   regression test alongside the code change.
 
@@ -142,4 +142,4 @@ Before finishing a change, sanity-check:
 - Does it preserve existing YAML schema and replay semantics?
 - Does it keep current diagnostics and docs accurate enough?
 - Should `README.md` change?
-- Should a unit test or AuTest be added?
+- Should a unit test or Uranium test be added?

CMakeLists.txt

@@ -154,5 +154,15 @@ include(HttpDependencies)
 pv_find_http_dependencies(PV_HTTP_DEPENDENCY_TARGETS)
 
 add_subdirectory(src)
-add_subdirectory(tests/autests)
+string(SHA256 _pv_source_hash "${CMAKE_SOURCE_DIR}")
+string(SUBSTRING "${_pv_source_hash}" 0 8 _pv_source_hash)
+set(URTEST_SANDBOX
+    "/tmp/proxy-verifier-urtest-${_pv_source_hash}"
+    CACHE PATH "Location for Uranium test output")
+set(URTEST_OPTIONS
+    ""
+    CACHE STRING
+          "Additional pytest options supplied by the Uranium test target")
+
+add_subdirectory(tests)
 add_subdirectory(tests/unit_tests)

CONTRIBUTING.md

@@ -20,7 +20,7 @@ When your code is ready to be submitted, read github's article about how to [sub
 
 1. Run `tools/pr-prep.sh` to format the source code and update copyright years
    according to the repository's standards.
-1. Verify that all AuTests pass (see the [README](https://github.com/yahoo/proxy-verifier/blob/master/README.md) for instructions on how to run the AuTests).
+1. Verify that all Uranium tests pass (see the [README](https://github.com/yahoo/proxy-verifier/blob/master/README.md) for instructions).
 1. Verify that all unit tests pass (see the [README](https://github.com/yahoo/proxy-verifier/blob/master/README.md) for instructions on how to run the unit tests).
 
 We only seek to accept code that you are authorized to contribute to the project. We have added a pull request template on our projects so that your contributions are made with the following confirmation: 

NOTICES

@@ -7,7 +7,7 @@ SPDX-License-Identifier: Apache-2.0
 
 proxy_http1.py
 
-The test HTTP/1.x proxy named tests/autests/gold_tests/autest-site/proxy_http1.py
+The test HTTP/1.x proxy named tests/tools/uranium/proxy/proxy_http1.py
 was initially copied from:
 https://github.com/inaz2/proxy2/tree/50ae4b9ef7fbc51799f95e405d4d7adf1af239f7
 
@@ -54,7 +54,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 proxy_http2.py
 
-The test HTTP/2 proxy named tests/autests/gold_tests/autest-site/proxy_http2.py
+The test HTTP/2 proxy named tests/tools/uranium/proxy/proxy_http2.py
 was initially copied from:
 https://github.com/python-hyper/hyper-h2/blob/e0a3e2eb5449b78ab811f32b7672bfd4e1033cf0/examples/eventlet/eventlet-server.py