From 0ef9690822c45923eede8a0ef769405b7492f4e9 Mon Sep 17 00:00:00 2001 From: Omry Yadan Date: Sun, 25 Oct 2026 23:29:25 +0800 Subject: [PATCH] [node-maintenance-repair/7] Validate the sealed candidate on a separate runner Validate the sealed preparation candidate on a separate read-only runner. Reuse native Git patch/index checks, the strict request and audit envelope, frozen installation and the configured project checks. Reject dependency/configuration drift, remaining selected advisories and new findings. Keep validation logs and results separate from the immutable preparation artifact. The five workflow inputs stay unchanged; publication and normal-audit retention remain later slices. Validation: 1,455 Python tests, Ruff lint/format, Pyrefly, and real Node 24.22.1 with pnpm 11.20.0/11.21.0 success, failure, drift and artifact-tamper cases. Current-agent and fresh independent reviews are clean. Hosted validation remains pending regular PR approval. --- .github/workflows/node-repair.yml | 25 ++- docs/node-maintenance-operator.md | 25 ++- node/repair/action.yml | 48 +++++- node/runtime/repair.py | 168 ++++++++++++++++++- node/tests/test_selected_repairs.py | 245 +++++++++++++++++++++++++++- node/tests/verify_node_project.py | 127 +++++++++++++- repair-app/pilot/README.md | 20 ++- 7 files changed, 632 insertions(+), 26 deletions(-) diff --git a/.github/workflows/node-repair.yml b/.github/workflows/node-repair.yml index b700746..cd16681 100644 --- a/.github/workflows/node-repair.yml +++ b/.github/workflows/node-repair.yml @@ -1,4 +1,4 @@ -name: Prepare selected Node repairs +name: Validate selected Node repairs 'on': workflow_call: inputs: @@ -19,7 +19,7 @@ name: Prepare selected Node repairs required: true outputs: candidate-artifact-id: - description: Immutable preparation artifact ID in this run; no validation or publication occurred. + description: Immutable preparation artifact ID in this run; validation never replaces it. No publication occurred. value: ${{ jobs.prepare.outputs.candidate-artifact-id }} permissions: contents: read @@ -40,6 +40,27 @@ jobs: expected-head: ${{ inputs.expected-head }} expected-base: ${{ inputs.expected-base }} selections: ${{ inputs.selections }} + validate: + needs: prepare + runs-on: ubuntu-latest + timeout-minutes: 50 + permissions: + contents: read + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 + with: + ref: ${{ inputs.expected-head }} + persist-credentials: false + - name: Validate the sealed candidate using trusted workflow code + uses: $/node/repair + with: + operation: validate + candidate-artifact-id: ${{ needs.prepare.outputs.candidate-artifact-id }} + request-id: ${{ inputs.request-id }} + pull-request: ${{ inputs.pull-request }} + expected-head: ${{ inputs.expected-head }} + expected-base: ${{ inputs.expected-base }} + selections: ${{ inputs.selections }} prepare: needs: guard runs-on: ubuntu-latest diff --git a/docs/node-maintenance-operator.md b/docs/node-maintenance-operator.md index 3a8525b..6fedb1a 100644 --- a/docs/node-maintenance-operator.md +++ b/docs/node-maintenance-operator.md @@ -128,8 +128,8 @@ these instructions. The local/native tests cannot replace that acceptance. ## Candidate preparation boundary -The reusable `.github/workflows/node-repair.yml` currently enables only `guard` -and `prepare`. It accepts the existing five request inputs and returns +The reusable `.github/workflows/node-repair.yml` enables `guard`, `prepare` +and `validate`. It accepts the existing five request inputs and returns `candidate-artifact-id` for the immutable artifact created in that run. It does not install a caller or enable repairs in an existing repository. @@ -158,8 +158,25 @@ The envelope rejects missing/extra fields, mismatched request or project, unsupported versions, symlinked project parents, unconfirmed selected fixes and new findings. Later consumers must download by the returned artifact ID from the same run. A preparation success is not proof of a validated or published -repair. Separate validation, publication and normal-audit retention remain -later slices, followed by the isolated live repair gate and general rollout. +repair. + +Validation checks out the same expected head on a separate read-only runner, +without persisted Git credentials or App credentials. It downloads only the +preparation artifact ID from the current run. It rejects extra members and +unapproved patch paths or file modes before applying the candidate. Frozen +installation and the configured `validationScript` run against that candidate; +include any project-specific local-patch tests in this script. Changes to +dependency files or configuration fail validation. A fresh audit must confirm +all selected advisories are removed and no new finding is present. + +The separate `node-repair-validation-` artifact contains human +evidence only: install/check logs, the fresh audit and a validation result. +It never replaces the preparation artifact. Project code runs with the +repository write maintainer as its trust principal; validation cannot attest +intentionally dishonest tests. A later publisher must use the original +preparation artifact and actual validation job success, never validation-local +patches, reports or logs. Publication and normal-audit retention remain later +slices, followed by the isolated live repair gate and general rollout. ## Check the receiver locally diff --git a/node/repair/action.yml b/node/repair/action.yml index a424012..607d73d 100644 --- a/node/repair/action.yml +++ b/node/repair/action.yml @@ -1,8 +1,8 @@ name: Node repair candidate -description: Guard or prepare a selected pnpm candidate without project scripts or publication. +description: Guard, prepare or validate a selected pnpm candidate without publication. inputs: operation: - description: guard or prepare. + description: guard, prepare or validate. required: true request-id: required: true @@ -19,6 +19,9 @@ inputs: selections: required: true description: Canonical selected package/version triples. + candidate-artifact-id: + required: false + description: Single immutable preparation artifact ID from this run, required for validate. outputs: candidate-artifact-id: description: Immutable preparation artifact ID from this run. @@ -32,9 +35,23 @@ runs: OPERATION: ${{ inputs.operation }} run: | case "$OPERATION" in - guard|prepare) ;; - *) echo '::error::Expected guard or prepare'; exit 1 ;; + guard|prepare|validate) ;; + *) echo '::error::Expected guard, prepare or validate'; exit 1 ;; esac + - name: Validate the artifact ID + if: ${{ inputs.operation == 'validate' }} + shell: bash + env: + ARTIFACT_ID: ${{ inputs.candidate-artifact-id }} + run: | + [[ "$ARTIFACT_ID" =~ ^[1-9][0-9]*$ ]] || { echo '::error::Expected one preparation artifact ID'; exit 1; } + - name: Download only the preparation artifact from this run + if: ${{ inputs.operation == 'validate' }} + uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 + with: + artifact-ids: ${{ inputs.candidate-artifact-id }} + path: ${{ runner.temp }}/node-repair-candidate + digest-mismatch: error - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 with: python-version: '3.13' @@ -53,18 +70,18 @@ runs: REPAIR_SELECTIONS: ${{ inputs.selections }} run: python "$GITHUB_ACTION_PATH/../runtime/repair.py" guard - name: Resolve the declared preparation toolchain - if: ${{ inputs.operation == 'prepare' }} + if: ${{ inputs.operation != 'guard' }} shell: bash run: | set -euo pipefail python "$GITHUB_ACTION_PATH/../runtime/project.py" --github-env "$GITHUB_ENV" --resolved-config "$RUNNER_TEMP/node-repair-config.json" - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 - if: ${{ inputs.operation == 'prepare' }} + if: ${{ inputs.operation != 'guard' }} with: node-version: ${{ env.NODE_AUDIT_NODE_VERSION }} check-latest: true - name: Enable the declared package manager - if: ${{ inputs.operation == 'prepare' }} + if: ${{ inputs.operation != 'guard' }} shell: bash run: corepack enable - name: Prepare and seal the selected candidate @@ -88,3 +105,20 @@ runs: ${{ runner.temp }}/node-repair-candidate/candidate.patch ${{ runner.temp }}/node-repair-candidate/audit.json ${{ runner.temp }}/node-repair-candidate/result.md + - name: Validate the selected candidate + if: ${{ inputs.operation == 'validate' }} + shell: bash + env: + REPAIR_REQUEST: ${{ inputs.request-id }} + REPAIR_PR: ${{ inputs.pull-request }} + REPAIR_HEAD: ${{ inputs.expected-head }} + REPAIR_BASE: ${{ inputs.expected-base }} + REPAIR_SELECTIONS: ${{ inputs.selections }} + run: python "$GITHUB_ACTION_PATH/../runtime/repair.py" validate --artifact "$RUNNER_TEMP/node-repair-candidate" --evidence "$RUNNER_TEMP/node-repair-validation" + - name: Retain human validation evidence only + if: ${{ always() && inputs.operation == 'validate' }} + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a + with: + name: node-repair-validation-${{ inputs.request-id }} + if-no-files-found: warn + path: ${{ runner.temp }}/node-repair-validation/ diff --git a/node/runtime/repair.py b/node/runtime/repair.py index 2bd35e6..9e73f5c 100644 --- a/node/runtime/repair.py +++ b/node/runtime/repair.py @@ -1,4 +1,4 @@ -"""Guard and seal one selected pnpm repair candidate; run no project scripts.""" +"""Prepare a sealed pnpm candidate and validate it without publication.""" import argparse import copy @@ -7,6 +7,7 @@ import re import runpy import subprocess +import tempfile import time from collections import Counter from contextlib import contextmanager @@ -484,6 +485,162 @@ def export_patch(paths): ) +def apply_candidate(expected, artifact, directory, snapshot): + """Consume the preparation copy; never produce a new candidate artifact.""" + root = Path.cwd().resolve() + if ( + artifact.is_symlink() + or set(p.name for p in artifact.iterdir()) + != {"candidate.patch", "audit.json", "result.md"} + or any(p.is_symlink() or not p.is_file() for p in artifact.iterdir()) + ): + raise ValueError("Invalid sealed artifact members") + envelope = read_envelope( + artifact / "audit.json", expected, directory.relative_to(root).as_posix() + ) + paths = { + (directory / name).relative_to(root).as_posix() + for name in ("pnpm-workspace.yaml", "pnpm-lock.yaml") + } + # Let Git parse quoted names, binary diffs and path safety. A temporary index + # proves the complete patch's paths and modes before it touches the worktree. + with tempfile.TemporaryDirectory(prefix="node-repair-index-") as temporary: + environment = dict(os.environ, GIT_INDEX_FILE=str(Path(temporary) / "index")) + + def indexed(*args): + return subprocess.check_output(["git", *args], env=environment, text=True) + + indexed("read-tree", "HEAD") + indexed( + "apply", + "--cached", + "--whitespace=nowarn", + str(artifact / "candidate.patch"), + ) + records = indexed( + "diff", + "--no-ext-diff", + "--no-textconv", + "--cached", + "--raw", + "--no-abbrev", + "-z", + "HEAD", + ).split("\0") + if not records[0] or indexed( + "diff", "--no-ext-diff", "--no-textconv", "--cached", "--summary", "HEAD" + ): + raise ValueError("Candidate must modify existing dependency files only") + for index in range(0, len(records) - 1, 2): + fields = records[index].split() + if ( + len(fields) != 5 + or fields[:2] != [":100644", "100644"] + or fields[4] != "M" + or records[index + 1] not in paths + ): + raise ValueError("Candidate patch changed an unapproved path or mode") + git("apply", "--whitespace=nowarn", str(artifact / "candidate.patch")) + candidate = ENGINE["valid_snapshot"](directory) + check_scope(snapshot, candidate, expected["selections"]) + return envelope, candidate + + +def validation_state(root, directory): + """Capture declaration bytes and modes before running project code.""" + inputs = PROJECT["configuration_inputs"](root, directory) + paths = set(inputs) | {(directory / "pnpm-lock.yaml").relative_to(root).as_posix()} + for parent in (directory, *directory.parents): + if parent.is_relative_to(root): + paths.update( + (parent / name).relative_to(root).as_posix() + for name in (".npmrc", ".pnpmfile.cjs") + ) + state = {} + for name in paths: + path = root / project_path(root, name) + if path.exists(): + if path.is_symlink() or not path.is_file(): + raise ValueError("Validation declarations must remain regular files") + state[name] = (path.stat().st_mode, path.read_bytes()) + else: + state[name] = None + return state + + +def validate(expected, artifact, evidence): + root = Path.cwd().resolve() + if ( + git("rev-parse", "HEAD").strip() != expected["head"] + or git("status", "--porcelain", "--untracked-files=all").strip() + ): + raise ValueError("Validation requires a clean checkout of the expected head") + if artifact.resolve() != artifact or artifact.is_relative_to(root): + raise ValueError("Candidate artifacts must be outside the project checkout") + if ( + evidence.resolve() != evidence + or evidence.is_relative_to(root) + or evidence.is_relative_to(artifact) + or artifact.is_relative_to(evidence) + ): + raise ValueError( + "Validation evidence must be separate from the checkout and candidate" + ) + evidence.mkdir(parents=True, exist_ok=False) + try: + with suppress_hooks(): + directory, selected, snapshot = project_context() + envelope, candidate = apply_candidate( + expected, artifact, directory, snapshot + ) + state = validation_state(root, directory) + deadline = time.monotonic() + 40 * 60 + + def unchanged(): + if ( + validation_state(root, directory) != state + or ENGINE["valid_snapshot"](directory) != candidate + ): + raise ValueError("Validation changed dependency files or configuration") + + for label, arguments in ( + ("Frozen install", ("install", "--frozen-lockfile")), + ( + "Configured project/local-patch checks", + ("run", selected["validationScript"]), + ), + ): + result = ENGINE["run"](directory, *arguments, deadline=deadline) + ( + evidence + / ("install.log" if label == "Frozen install" else "checks.log") + ).write_text(result.stdout + "\n" + result.stderr) + unchanged() + if result.returncode: + raise ValueError(f"{label} failed (exit {result.returncode})") + with suppress_hooks(): + after = ENGINE["audit"](directory, deadline=deadline) + unchanged() + # Reuse the strict selected-fix and no-new-findings checks against a fresh + # audit. The in-memory preparation envelope stays the comparison source. + validate_envelope( + dict(envelope, candidate=after), expected, envelope["project_directory"] + ) + (evidence / "audit.json").write_text( + json.dumps(after, ensure_ascii=False) + "\n" + ) + message = "Validation passed: frozen install, configured project/local-patch checks and fresh advisory checks. No dependency change has been published." + except Exception as error: + (evidence / "result.md").write_text( + "Validation failed. No dependency change has been published.\n\n" + + str(error) + + "\n" + ) + raise + (evidence / "result.md").write_text(message + "\n") + return after + + def prepare(expected, artifact): with suppress_hooks(): return prepare_candidate(expected, artifact) @@ -572,14 +729,19 @@ def prepare_candidate(expected, artifact): def main(): parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("operation", choices=("guard", "prepare")) + parser.add_argument("operation", choices=("guard", "prepare", "validate")) parser.add_argument("--artifact", type=Path) + parser.add_argument("--evidence", type=Path) args = parser.parse_args() expected = request() if args.operation == "guard": guard(expected) elif args.artifact is None: - parser.error("prepare requires --artifact") + parser.error("prepare and validate require --artifact") + elif args.operation == "validate": + if args.evidence is None: + parser.error("validate requires --evidence") + validate(expected, args.artifact, args.evidence) else: prepare(expected, args.artifact) diff --git a/node/tests/test_selected_repairs.py b/node/tests/test_selected_repairs.py index 3483e85..c5a18d1 100644 --- a/node/tests/test_selected_repairs.py +++ b/node/tests/test_selected_repairs.py @@ -1,4 +1,4 @@ -"""Selection/report contract tests; no repair execution or publication.""" +"""Selection, candidate and read-only validation contracts; no publication.""" import copy import json @@ -845,7 +845,7 @@ def git(*args): assert not (tmp_path / "helper-executed").exists() -def test_repair_workflow_uses_trusted_code_and_has_only_read_only_preparation(): +def test_repair_workflow_keeps_preparation_separate_from_read_only_validation(): workflow = yaml.safe_load((ROOT / ".github/workflows/node-repair.yml").read_text()) action = yaml.safe_load((ROOT / "node/repair/action.yml").read_text()) assert set(workflow["on"]["workflow_call"]["inputs"]) == { @@ -855,7 +855,7 @@ def test_repair_workflow_uses_trusted_code_and_has_only_read_only_preparation(): "expected-base", "selections", } - assert set(workflow["jobs"]) == {"guard", "prepare"} + assert set(workflow["jobs"]) == {"guard", "prepare", "validate"} for job in workflow["jobs"].values(): assert set(job["permissions"].values()) == {"read"} assert any(step.get("uses") == "$/node/repair" for step in job["steps"]) @@ -875,10 +875,33 @@ def test_repair_workflow_uses_trusted_code_and_has_only_read_only_preparation(): "audit.json", "result.md", ] - assert not any( - "validate" in step.get("run", "") or "publish" in step.get("run", "") + validation = workflow["jobs"]["validate"] + assert validation["needs"] == "prepare" + assert validation["steps"][0]["with"] == checkout["with"] + assert ( + validation["steps"][1]["with"]["candidate-artifact-id"] + == "${{ needs.prepare.outputs.candidate-artifact-id }}" + ) + assert ( + workflow["on"]["workflow_call"]["outputs"]["candidate-artifact-id"]["value"] + == "${{ jobs.prepare.outputs.candidate-artifact-id }}" + ) + download = next( + step + for step in action["runs"]["steps"] + if step.get("uses", "").startswith("actions/download-artifact@") + ) + assert set(download["with"]) == {"artifact-ids", "path", "digest-mismatch"} + assert download["with"]["artifact-ids"] == "${{ inputs.candidate-artifact-id }}" + assert download["with"]["digest-mismatch"] == "error" + assert upload["if"] == "${{ inputs.operation == 'prepare' }}" + human = next( + step for step in action["runs"]["steps"] + if step.get("name") == "Retain human validation evidence only" ) + assert human["with"]["path"] == "${{ runner.temp }}/node-repair-validation/" + assert "publish" not in workflow["jobs"] def test_parser_install_isolated_from_checkout_module_shadowing(tmp_path): @@ -923,3 +946,215 @@ def test_parser_install_isolated_from_checkout_module_shadowing(tmp_path): ) assert result.returncode == 0, result.stderr assert marker.exists() is shadowed + + +@pytest.fixture +def sealed_checkout(monkeypatch, tmp_path): + root = tmp_path / "checkout" + root.mkdir() + monkeypatch.chdir(root) + before = repair_files(root) + (root / ".github").mkdir() + (root / ".github/node-maintenance.json").write_text('{"validationScript":"check"}') + (root / "unrelated.txt").write_text("unchanged\n") + subprocess.run(["git", "init", "-q"], check=True) + subprocess.run(["git", "add", "."], check=True) + subprocess.run( + [ + "git", + "-c", + "core.hooksPath=/dev/null", + "-c", + "user.name=Fixture", + "-c", + "user.email=fixture@example.com", + "commit", + "-qm", + "fixture", + ], + check=True, + ) + expected = dict(repair_request(), head=REPAIR["git"]("rev-parse", "HEAD").strip()) + repair_files(root, ITEM["to"]) + artifact = tmp_path / "candidate" + artifact.mkdir() + (artifact / "candidate.patch").write_bytes( + REPAIR["export_patch"](["pnpm-workspace.yaml", "pnpm-lock.yaml"]) + ) + (artifact / "audit.json").write_text(json.dumps(dict(envelope(), request=expected))) + (artifact / "result.md").write_text("Original preparation report\n") + subprocess.run(["git", "reset", "--hard", "-q", "HEAD"], check=True) + module = REPAIR["validate"].__globals__ + monkeypatch.setitem( + module, "project_context", lambda: (root, {"validationScript": "check"}, before) + ) + return root, artifact, before, expected + + +@pytest.mark.parametrize( + "problem", + [ + None, + "extra-member", + "missing-member", + "symlink", + "directory", + "wrong-request", + "empty-patch", + "extra-path", + "mode", + "delete", + "rename", + "copy", + "absolute", + "traversal", + ], +) +def test_candidate_consumption_uses_native_git_scope_before_execution( + sealed_checkout, problem +): + root, artifact, before, expected = sealed_checkout + patch = artifact / "candidate.patch" + if problem == "extra-member": + (artifact / "extra").touch() + elif problem == "missing-member": + (artifact / "result.md").unlink() + elif problem in ("symlink", "directory"): + (artifact / "result.md").unlink() + if problem == "symlink": + (artifact / "result.md").symlink_to(root / "unrelated.txt") + else: + (artifact / "result.md").mkdir() + elif problem == "wrong-request": + data = json.loads((artifact / "audit.json").read_text()) + data["request"]["id"] = "0" * 36 + (artifact / "audit.json").write_text(json.dumps(data)) + elif problem == "empty-patch": + patch.write_text("") + elif problem == "extra-path": + (root / "unrelated.txt").write_text("changed\n") + patch.write_bytes( + patch.read_bytes() + REPAIR["export_patch"](["unrelated.txt"]) + ) + subprocess.run(["git", "checkout", "--", "unrelated.txt"], check=True) + elif problem == "mode": + patch.write_text( + patch.read_text().replace( + "index ", "old mode 100644\nnew mode 100755\nindex ", 1 + ) + ) + elif problem == "delete": + (root / "pnpm-lock.yaml").unlink() + patch.write_bytes(REPAIR["export_patch"](["pnpm-lock.yaml"])) + subprocess.run(["git", "checkout", "--", "pnpm-lock.yaml"], check=True) + elif problem in ("rename", "copy"): + patch.write_text( + f"diff --git a/pnpm-lock.yaml b/other.yaml\nsimilarity index 100%\n{problem} from pnpm-lock.yaml\n{problem} to other.yaml\n" + ) + elif problem in ("absolute", "traversal"): + name = "/tmp/outside.yaml" if problem == "absolute" else "../outside.yaml" + patch.write_text( + patch.read_text() + .replace("a/pnpm-lock.yaml", "a/" + name) + .replace("b/pnpm-lock.yaml", "b/" + name) + ) + if problem: + with pytest.raises((ValueError, subprocess.CalledProcessError)): + REPAIR["apply_candidate"](expected, artifact, root, before) + assert ENGINE["valid_snapshot"](root) == before + else: + _, candidate = REPAIR["apply_candidate"](expected, artifact, root, before) + REPAIR["check_scope"](before, candidate, expected["selections"]) + + +@pytest.mark.parametrize( + "problem", + [ + None, + "install-failure", + "check-failure", + "install-drift", + "check-drift", + "policy-mode", + "lock-symlink", + "config-drift", + "new-config", + "audit-drift", + "malformed-audit", + "selected-remains", + "new-finding", + "artifact-tamper", + ], +) +def test_validation_fails_closed_and_never_reseals_candidate( + monkeypatch, sealed_checkout, tmp_path, problem +): + import shutil + + root, original, _, expected = sealed_checkout + original_bytes = {p.name: p.read_bytes() for p in original.iterdir()} + artifact = tmp_path / "downloaded" + shutil.copytree(original, artifact) + module = REPAIR["validate"].__globals__ + commands = [] + + def command(directory, *args, **kwargs): + commands.append(args) + install = args[0] == "install" + if problem == ("install-drift" if install else "check-drift"): + with (root / "pnpm-lock.yaml").open("a") as stream: + stream.write("\n# changed by project\n") + if not install: + if problem == "config-drift": + (root / ".github/node-maintenance.json").write_text("{}") + elif problem == "new-config": + (root / ".npmrc").write_text("ignore-scripts=true") + elif problem == "policy-mode": + (root / "pnpm-workspace.yaml").chmod(0o755) + elif problem == "lock-symlink": + lock = root / "pnpm-lock.yaml" + saved = tmp_path / "lock-copy" + saved.write_bytes(lock.read_bytes()) + lock.unlink() + lock.symlink_to(saved) + elif problem == "artifact-tamper": + (artifact / "candidate.patch").write_text("forged patch\n") + (artifact / "result.md").write_text("forged success\n") + return SimpleNamespace( + returncode=int( + problem == ("install-failure" if install else "check-failure") + ), + stdout="test output", + stderr="", + ) + + def audit(*args, **kwargs): + if problem == "audit-drift": + (root / "pnpm-workspace.yaml").write_text("{}") + if problem == "malformed-audit": + return "unavailable" + if problem in ("selected-remains", "new-finding"): + result = repair_audit() + if problem == "new-finding": + result["advisories"]["1"]["url"] += "-new" + result["vulnerabilities"][ITEM["name"]]["via"][0]["url"] += "-new" + return result + return empty_audit() + + monkeypatch.setitem(module["ENGINE"], "run", command) + monkeypatch.setitem(module["ENGINE"], "audit", audit) + evidence = tmp_path / "human-evidence" + if problem in (None, "artifact-tamper"): + REPAIR["validate"](expected, artifact, evidence) + assert commands == [("install", "--frozen-lockfile"), ("run", "check")] + assert "Validation passed" in (evidence / "result.md").read_text() + assert "forged" not in (evidence / "result.md").read_text() + else: + with pytest.raises(ValueError): + REPAIR["validate"](expected, artifact, evidence) + assert "Validation failed" in (evidence / "result.md").read_text() + if problem.startswith("install-"): + assert len(commands) == 1 + assert not (evidence / "candidate.patch").exists() + assert {p.name: p.read_bytes() for p in original.iterdir()} == original_bytes + assert REPAIR["git"]("rev-parse", "HEAD").strip() == expected["head"] diff --git a/node/tests/verify_node_project.py b/node/tests/verify_node_project.py index f5ebcb3..ca2285b 100644 --- a/node/tests/verify_node_project.py +++ b/node/tests/verify_node_project.py @@ -12,7 +12,7 @@ import yaml -ROOT = Path(__file__).parents[1] +ROOT = Path(__file__).resolve().parents[1] def check_pnpm_repair(project, command, version): @@ -217,6 +217,130 @@ def check_npm_policy(version): return "passed (project, user and environment configuration)" +def check_selected_validation(version): + """Exercise the real prepare/consume/frozen-check path in disposable Git.""" + repair = runpy.run_path(str(ROOT / "runtime/repair.py")) + with tempfile.TemporaryDirectory(prefix="selected-validation-") as temporary: + root = Path(temporary) + checkout = root / "checkout" + checkout.mkdir() + (checkout / ".github").mkdir() + (checkout / ".github/node-maintenance.json").write_text( + '{"directory":".","validationScript":"check"}' + ) + (checkout / ".gitignore").write_text("node_modules/\n") + (checkout / ".nvmrc").write_text( + subprocess.check_output(["node", "--version"], text=True).strip() + ) + (checkout / "package.json").write_text( + json.dumps( + { + "name": "selected-validation", + "private": True, + "packageManager": f"pnpm@{version}", + "dependencies": {"launch-editor": "^2.14.1"}, + "scripts": {"check": "node check.cjs"}, + } + ) + ) + (checkout / "pnpm-workspace.yaml").write_text( + yaml.safe_dump( + { + "packages": ["."], + "minimumReleaseAge": 14400, + "overrides": {"launch-editor": "2.14.1"}, + } + ) + ) + (checkout / "check.cjs").write_text( + "const fs=require('node:fs');\n" + "require('node:assert/strict').equal(typeof require('launch-editor'),'function');\n" + "fs.writeFileSync(process.env.REPAIR_TEST_MARKER,'checked');\n" + "if(process.env.REPAIR_TEST_MODE==='fail')process.exit(1);\n" + "if(process.env.REPAIR_TEST_MODE==='drift')fs.appendFileSync('pnpm-lock.yaml','\\n# project changed lock\\n');\n" + "if(process.env.REPAIR_TEST_MODE==='tamper')for(const name of ['candidate.patch','result.md'])" + "fs.writeFileSync(process.env.REPAIR_TEST_ARTIFACT+'/'+name,'forged');\n" + ) + subprocess.run( + ["corepack", "pnpm", *repair["CANDIDATE_INSTALL"]], + cwd=checkout, + check=True, + timeout=180, + ) + for args in ( + ["init", "-q"], + ["add", "."], + [ + "-c", + "core.hooksPath=/dev/null", + "-c", + "user.name=Fixture", + "-c", + "user.email=fixture@example.com", + "commit", + "-qm", + "fixture", + ], + ): + subprocess.run(["git", *args], cwd=checkout, check=True) + expected = dict( + id="12345678-1234-1234-1234-123456789abc", + repository="example/fixture", + pr=1, + head=subprocess.check_output( + ["git", "rev-parse", "HEAD"], cwd=checkout, text=True + ).strip(), + base="b" * 40, + selections=[{"name": "launch-editor", "from": "2.14.1", "to": "2.14.2"}], + ) + previous = Path.cwd() + try: + os.chdir(checkout) + with patch.dict(os.environ, {"RUNNER_TEMP": str(root)}): + artifact = root / "prepared" + repair["prepare"](expected, artifact) + sealed = {p.name: p.read_bytes() for p in artifact.iterdir()} + for mode in ("pass", "fail", "drift", "tamper"): + subprocess.run(["git", "reset", "--hard", "-q", "HEAD"], check=True) + downloaded = root / f"download-{mode}" + shutil.copytree(artifact, downloaded) + marker = root / f"checked-{mode}" + with patch.dict( + os.environ, + { + "REPAIR_TEST_MODE": mode, + "REPAIR_TEST_MARKER": str(marker), + "REPAIR_TEST_ARTIFACT": str(downloaded), + }, + ): + try: + repair["validate"]( + expected, downloaded, root / f"evidence-{mode}" + ) + except ValueError as error: + assert mode in ("fail", "drift"), str(error) + assert ( + "checks failed" + if mode == "fail" + else "changed dependency" + ) in str(error) + else: + assert mode in ("pass", "tamper") + assert marker.exists(), "Configured project check did not run" + assert { + p.name: p.read_bytes() for p in artifact.iterdir() + } == sealed + assert ( + subprocess.check_output( + ["git", "rev-parse", "HEAD"], text=True + ).strip() + == expected["head"] + ) + finally: + os.chdir(previous) + return "passed (real frozen install, configured checks, failure, drift and immutable original artifact)" + + def check(node, manager, version): with tempfile.TemporaryDirectory(prefix="node-maintenance-") as temporary: root = Path(temporary) @@ -326,6 +450,7 @@ def check(node, manager, version): result["selected_repair_preparation"] = check_pnpm_repair( project, command, version ) + result["selected_repair_validation"] = check_selected_validation(version) if manager == "npm": result["inherited_npm_security_policy"] = check_npm_policy(version) return result diff --git a/repair-app/pilot/README.md b/repair-app/pilot/README.md index 219aff9..df8a933 100644 --- a/repair-app/pilot/README.md +++ b/repair-app/pilot/README.md @@ -29,7 +29,7 @@ Restore the original PR description after temporary probe selections, checking that no concurrent edit would be overwritten. Record actual source/deployment identities and distinguish local intercepted API tests from real GitHub events. -## Isolated candidate-preparation acceptance +## Isolated candidate preparation and validation acceptance After reviewing the preparation slice, use a separately authorized isolated repository. Replace its existing harmless caller's job with a call to the @@ -64,6 +64,18 @@ unselected dependencies and a truthful preparation-only report. Also record that no project scripts ran and no repository dependency files were published. Stale consent and unavailable evidence must fail without a candidate artifact. -Only guard and preparation are enabled now. This gate does not pass G-REPAIR, -upgrade the PR or prove the later validation/publisher jobs. Those are added and -tested in their owning slices before the final Hydra rollout. +Validation runs on a separate read-only runner after preparation. Record a +successful frozen install and configured project/local-patch checks, followed +by a fresh audit with all selected advisories removed and no new finding. +Use a deliberately failing check or dependency-file mutation to prove rejection +without changes to the PR. Keep those test changes in the isolated repository. + +Also exercise project code that changes the validation runner's downloaded +`candidate.patch` or `result.md`. Download the original preparation artifact by +its retained ID again and prove its bytes did not change. The validation +artifact is human evidence only; its files cannot become publication inputs. +Keep both artifact IDs and their separate contents in the acceptance record. + +Publication is not enabled yet. These gates do not pass G-REPAIR or upgrade the +PR. Publication and retention are tested in their owning slices before the +final Hydra rollout.