build: make Python baseline validation truthful (#43)

REQUIREMENTS.md

@@ -2,7 +2,7 @@
 
 > **Note:** This document is automatically generated and verified against the live test suite by `scripts/generate_requirements.py` and `tests/backend/test_requirements_sync.py`.
 
-**Test Verification Baseline:** **1115 Automated Tests** (757 Pytest Backend + 308 Vitest Frontend + 50 Playwright E2E).
+**Test Verification Baseline:** **1129 Automated Tests** (771 Pytest Backend + 308 Vitest Frontend + 50 Playwright E2E).
 
 ---
 
@@ -874,6 +874,10 @@ Asserts both Markdown and PDF docs are matched under doc_type='doc'._
 - `test_delete_by_repo`
 - `test_get_stats_and_health_check`
 
+#### `tests/test_approved_mirror.py` (2 tests)
+- `test_checked_in_mcp_pin_resolves_from_approved_mirror` - _Catch an unavailable exact MCP pin even when the declared range is satisfiable._
+- `test_direct_requirement_has_compatible_approved_wheel` - _Distinguish unavailable project artifacts from overly restrictive candidate pins._
+
 #### `tests/test_auth_endpoints.py` (9 tests)
 - `test_oauth_protected_resource_metadata` - _Verify GET /.well-known/oauth-protected-resource returns valid RFC 9728 JSON._
 - `test_local_dev_bypass_allows_unauthenticated_requests` - _When AUTH_ENABLED=false, endpoints allow access without authorization header._
@@ -953,7 +957,7 @@ Asserts both Markdown and PDF docs are matched under doc_type='doc'._
 - `test_runtime_ca_update_controls_application_start` - _Verifies that runtime CA bundle updates copy certificates and fail closed when update-ca-certificates errors._
 - `test_runtime_ca_empty_or_missing_dir_skips_update_and_starts` - _Verifies that an empty or missing EXTRA_CA_CERTS_DIR skips CA trust updates and launches the command cleanly._
 
-#### `tests/test_docker_orchestration.py` (42 tests)
+#### `tests/test_docker_orchestration.py` (36 tests)
 - `TestDockerComposeConfig::test_compose_version_and_services`
 - `TestDockerComposeConfig::test_postgres_service_config`
 - `TestDockerComposeConfig::test_contextcortex_service_config`
@@ -972,9 +976,6 @@ Asserts both Markdown and PDF docs are matched under doc_type='doc'._
 - `TestStartupDatabaseInitialization::test_init_application_database_without_env`
 - `TestStartupDatabaseInitialization::test_init_application_database_with_admin_key_env`
 - `TestStartupDatabaseInitialization::test_wait_for_db_retry_success` - _Ensures wait_for_db returns True upon successful database connectivity._
-- `TestPythonConstraintsBaseline::test_constraints_file_exists_and_contains_metadata`
-- `TestPythonConstraintsBaseline::test_constraints_covers_all_requirements_with_exact_pins`
-- `TestPythonConstraintsBaseline::test_pip_check_clean_dependency_tree`
 - `test_compose_version_and_services`
 - `test_postgres_service_config`
 - `test_contextcortex_service_config`
@@ -993,9 +994,6 @@ Asserts both Markdown and PDF docs are matched under doc_type='doc'._
 - `test_init_application_database_without_env`
 - `test_init_application_database_with_admin_key_env`
 - `test_wait_for_db_retry_success` - _Ensures wait_for_db returns True upon successful database connectivity._
-- `test_constraints_file_exists_and_contains_metadata`
-- `test_constraints_covers_all_requirements_with_exact_pins`
-- `test_pip_check_clean_dependency_tree`
 
 #### `tests/test_embedding_cache.py` (4 tests)
 - `test_embedding_cache_set_and_get`
@@ -1253,6 +1251,26 @@ over the PostgresConnectionAdapter using an initialized test engine._
 - `test_large_file_auto_summarization`
 - `test_large_file_disabled_summarization`
 
+#### `tests/test_python_baseline.py` (18 tests)
+- `TestPythonBaselineProbe::test_resolves_cleanly_with_constraints_and_emits_only_candidate_inventory`
+- `TestPythonBaselineProbe::test_failed_resolver_does_not_expose_private_output_or_retry`
+- `TestPythonBaselineProbe::test_rejects_unpinned_or_mismatched_transitive_artifacts`
+- `TestPythonBaselineProbe::test_rejects_missing_or_invalid_artifact_hash`
+- `TestPythonBaselineProbe::test_rejects_non_exact_constraints_before_contacting_pip`
+- `TestPythonBaselineProbe::test_rejects_invalid_or_empty_report`
+- `TestPythonBaselineProbe::test_timeout_does_not_expose_captured_private_output`
+- `TestPythonBaselineProbe::test_cli_writes_no_inventory_when_resolution_fails`
+- `TestPythonBaselineProbe::test_cli_emits_json_inventory_for_successful_probe`
+- `test_resolves_cleanly_with_constraints_and_emits_only_candidate_inventory`
+- `test_failed_resolver_does_not_expose_private_output_or_retry`
+- `test_rejects_unpinned_or_mismatched_transitive_artifacts`
+- `test_rejects_missing_or_invalid_artifact_hash`
+- `test_rejects_non_exact_constraints_before_contacting_pip`
+- `test_rejects_invalid_or_empty_report`
+- `test_timeout_does_not_expose_captured_private_output`
+- `test_cli_writes_no_inventory_when_resolution_fails`
+- `test_cli_emits_json_inventory_for_successful_probe`
+
 #### `tests/test_rest_role_permissions.py` (4 tests)
 - `test_anonymous_and_invalid_key_requests_denied` - _Anonymous or invalid/revoked requests receive 401 Unauthorized._
 - `test_viewer_role_permissions` - _Viewer key can read resources but is denied content mutations & administrative settings (403)._

docs/guide/enterprise-build.md

@@ -103,40 +103,104 @@ artifacts/lockfiles or mirror support for those tarball requests. Do not disable
 preparation scripts or fall back to public npm to hide the failure. The local
 frontend-only build currently reaches this blocker in a Git helper dependency.
 
-### The 30-day quarantine & Approved Mirror Baseline
+### The 30-day quarantine and candidate validation (issue #43 remains open)
 
-Let the internal mirror enforce its package-age policy; a package being
-available on public PyPI is not authorization to fetch it. Package age is defined
-and measured from the official release publication date on public PyPI (or artifact upload timestamp)
-prior to admission into the approved private mirror. All dependencies must satisfy the environment's
-30-day quarantine period before admission.
+**There is currently no verified approved Python baseline in this repository.**
+The former approval and cross-platform verification claims in
+`python-constraints.txt` were unsupported. Its MCP candidate now uses
+`mcp==1.28.1`, which the configured approved mirror provides and the declared
+`>=1.20.0,<2.0.0` requirement permits. Other pins remain unverified.
 
-For repeatability across supported platforms (Linux / Python 3.11 in Docker builds and Windows / Python 3.12 on enterprise hosts),
-the repository provides a complete reviewed exact-version baseline in `python-constraints.txt` (and `.secrets\python-constraints.txt`).
-This file records exact pins (`package==version`) for all direct, transitive, and build dependencies required by `requirements.txt`.
-It includes provenance comments, approval timestamps, and quarantine verification status.
+The updated real-mirror checks pass for MCP on Windows / Python 3.12.10 and
+Linux / Python 3.11. Each target has 17 passing checks and four failures.
+Even without exact constraints, these direct project artifacts remain missing:
 
-A constraints file enforces exact pinned versions during resolution without altering lower bounds in `requirements.txt`.
-Do not lower minimum version requirements or attempt to use public extra-index fallbacks to bypass mirror quarantine.
-Selecting `pgvector` or another vector store backend at runtime does **not** remove `qdrant-client` or `fastembed` from backend imports;
-all direct and transitive packages must be approved and present on the private mirror.
+- `qdrant-client>=1.9.0`
+- `fastembed>=0.3.0`
+- `tree-sitter-language-pack>=0.21.0`
+- `chromadb>=0.5.0`
 
-You can verify clean resolution against the approved mirror baseline on a configured host without modifying installed packages:
+Independent index queries also find no distributions for those four projects.
+These failures do not justify requesting newer MCP versions or lowering floors.
+A complete fresh requirements resolution stops at `qdrant-client`.
+Direct wheel availability does not prove transitive dependency resolution.
+
+Installing MCP 1.28.1 into a disposable cached runtime passes `pip check`,
+12 focused MCP tests, and an actual initialize/list-tools/catalog-call roundtrip.
+This tests compatibility with existing runtime dependencies, not a clean backend
+installation. The running deployment remains unchanged.
+
+Run the opt-in availability checks only with pip configured for the approved
+mirror and no public extra index:
 
 ```powershell
-python -m pip install --no-cache-dir --dry-run --ignore-installed `
-  --constraint .secrets\python-constraints.txt -r requirements.txt
+$env:RUN_APPROVED_MIRROR_TESTS = "1"
+python -m pytest --confcutdir=tests tests\test_approved_mirror.py
 ```
 
-> [!TIP]
-> Always pass `--no-cache-dir` when testing mirror resolution to avoid reusing locally cached wheels, ensuring every dependency is verified against the private repository quarantine.
+The tests resolve each declared direct requirement independently, without
+installing it, and separately check the exact MCP pin. They ignore installed
+packages and caches, use bounded wheel-only lookups, and withhold private pip
+diagnostics. They skip unless explicitly enabled. They do not prove a complete
+baseline or replace the constrained full-graph probe below.
+
+Let the internal mirror enforce its package-age policy; public availability or
+successful resolution is not approval. The mirror owner must agree whether the
+30-day clock starts at release publication, artifact upload or mirror admission.
+That decision and per-artifact evidence are still missing. Code cannot grant
+approval or import missing mirror artifacts.
 
-After resolution, confirm dependency tree consistency by running:
+A constraints file enforces exact pinned versions during resolution without altering lower bounds in `requirements.txt`.
+Do not lower minimum version requirements or attempt to use public extra-index fallbacks to bypass mirror quarantine.
+Selecting `pgvector` or another vector store backend at runtime does **not** remove `qdrant-client` or `fastembed` from backend imports;
+all direct and transitive packages must be approved and present on the private mirror.
+
+For a proposed replacement, put reviewed candidate exact pins in an ignored
+local file such as `.secrets\python-constraints.txt`. Configure pip to use only
+the approved mirror (including removing any public extra index from local pip
+configuration/environment). The probe inherits that configuration; it does not
+configure or authorize a registry. Never publish pip configuration or raw
+resolver reports: they can contain credentials and private endpoints.
+
+Run this stdlib-only probe on each actual target: Windows / Python 3.12 and
+Linux / Python 3.11. It does not install application packages:
 
 ```powershell
-pip check
+python scripts\probe_python_baseline.py `
+  --constraints .secrets\python-constraints.txt
 ```
 
+The probe runs pip with `--no-cache-dir --dry-run --ignore-installed`, exact
+constraints, zero retries and bounded timeouts. It fails on missing/mismatched
+transitive pins or absent SHA-256 hashes. It emits only a sanitized **unreviewed
+candidate** inventory (versions, selected artifact hashes, target and normalized
+UTF-8 constraints-text hash), never download URLs or approval claims. Failure
+output is withheld; inspect detailed pip errors privately if necessary.
+It requires wheels (`--only-binary=:all:`) to avoid executing source-build
+hooks during the probe. Source-only packages require separate approved build
+dependency/artifact review, not removing this guard to bypass quarantine.
+This inventory is not a hash-enforced lock or evidence of installed backend
+behavior, source-build dependency completeness, approval or 30-day age.
+
+Before promoting a candidate, the mirror owner/reviewer must record privately:
+
+- Every target's direct, transitive and build artifacts, exact versions,
+  filenames, SHA-256 hashes and provenance; a constraints file alone does not
+  record artifact identity or enforce those hashes during installation.
+- The agreed age basis, underlying timestamps, age of at least 30 days at
+  approval, approval timestamp and accountable reviewer.
+- Clean constrained resolution **and installation** into a new isolated native
+  environment on each supported target, followed by that environment's
+  `python -m pip check` with exit code zero (no filtered exceptions). `pip check`
+  in an existing environment after a dry-run does not validate the candidate.
+- Backend smoke/focused tests and a clean Docker **source** build using those
+  same reviewed constraints, followed by `python -m pip check` in the resulting
+  image. An existing image/overlay or mocked probe test is not this validation.
+
+These are remaining blockers for #43, not completed checks. No replacement
+baseline is endorsed here; preserve `requirements.txt` minimums while arranging
+approved mirror artifacts and review.
+
 A "No matching distribution" result indicates missing package mirror approval, quarantine hold, or incompatible target architecture wheels.
 Arrange mirror import and approval for missing direct or transitive packages (`qdrant-client`, `fastembed`, `tree-sitter`, `tree-sitter-language-pack`, `mcp`, etc.) before initiating a restricted-network build.
 

python-constraints.txt

@@ -1,8 +1,13 @@
-# ContextCortex Approved Python Mirror Constraints Baseline
-# Approval Timestamp: 2026-10-15T00:00:00Z
-# Target Environments: Linux / Python 3.11 (Docker Runtime), Windows / Python 3.12 (Host)
-# Quarantine Policy: 30-Day Quarantine satisfied; age measured from PyPI release publication date
-# Verification: Resolved against approved mirror baseline; verified via pip dry-run and pip check
+# ContextCortex UNVERIFIED Python constraints candidate - NOT an approved baseline
+# Intended targets: Linux / Python 3.11 and Windows / Python 3.12.
+# No approval, provenance, artifact hashes, quarantine-age evidence or clean
+# installation evidence accompanies the full graph. Previous verification claims
+# were unsupported. The MCP candidate now uses mirror-available 1.28.1; direct
+# wheel checks pass for that pin on Windows / Python 3.12 and Linux / Python 3.11.
+# Full resolution remains blocked by missing project artifacts (issue #43).
+# Do not bypass constraints.
+# Other pins remain unverified, NOT endorsed for use.
+# See docs/guide/enterprise-build.md for review and validation prerequisites.
 
 aiohappyeyeballs==2.7.1
 aiohttp==3.14.5
@@ -51,7 +56,7 @@ jsonschema-specifications==2025.9.1
 kubernetes==37.0.1
 loguru==0.7.3
 markdown-it-py==4.2.0
-mcp==1.30.0
+mcp==1.28.1
 mdurl==0.1.2
 mmh3==5.3.1
 multidict==7.2.1

scripts/probe_python_baseline.py

@@ -0,0 +1,97 @@
+#!/usr/bin/env python3
+"""Probe exact wheel candidates; never certify mirror approval or quarantine age."""
+
+import argparse
+import hashlib
+import json
+import re
+import subprocess
+import sys
+from pathlib import Path
+
+
+def canonical_name(name):
+    return re.sub(r"[-_.]+", "-", name).lower()
+
+
+def resolve_candidate(requirements, constraints):
+    text = constraints.read_text(encoding="utf-8")
+    pins = {}
+    for line in text.splitlines():
+        line = line.split("#", 1)[0].strip()
+        if not line:
+            continue
+        match = re.fullmatch(r"([A-Za-z0-9][A-Za-z0-9._-]*)==([A-Za-z0-9][A-Za-z0-9.!+_-]*)", line)
+        if not match or canonical_name(match[1]) in pins:
+            raise ValueError("Constraints must contain unique exact pins only.")
+        pins[canonical_name(match[1])] = match[2]
+    if not pins:
+        raise ValueError("Constraints must contain unique exact pins only.")
+
+    command = [
+        sys.executable, "-m", "pip", "install", "--quiet",
+        "--disable-pip-version-check", "--no-cache-dir", "--dry-run",
+        "--ignore-installed", "--only-binary=:all:", "--retries", "0",
+        "--timeout", "15", "--report", "-",
+        "--constraint", str(constraints), "--requirement", str(requirements),
+    ]
+    try:
+        completed = subprocess.run(
+            command, capture_output=True, text=True, check=False, timeout=180,
+        )
+    except subprocess.TimeoutExpired:
+        raise ValueError("Constrained resolution timed out; pip output withheld.") from None
+    if completed.returncode:
+        # Resolver output and pip reports can contain authenticated private URLs.
+        raise ValueError(
+            f"Constrained resolution failed (exit {completed.returncode}); "
+            "pip output withheld. Investigate locally without publishing credentials."
+        )
+    try:
+        report = json.loads(completed.stdout)
+        target = {
+            key: report["environment"][key]
+            for key in ("python_full_version", "sys_platform")
+        }
+        if not all(isinstance(value, str) and re.fullmatch(r"[A-Za-z0-9._-]+", value)
+                   for value in target.values()):
+            raise ValueError("Invalid pip candidate report.")
+        if not report["install"]:
+            raise ValueError("Invalid pip candidate report.")
+        artifacts = []
+        for item in report["install"]:
+            name = canonical_name(item["metadata"]["name"])
+            version = item["metadata"]["version"]
+            if pins.get(name) != version:
+                raise ValueError("Resolved artifact is not exactly constrained.")
+            digest = item["download_info"]["archive_info"].get("hashes", {}).get("sha256", "")
+            if not re.fullmatch(r"[a-fA-F0-9]{64}", digest):
+                raise ValueError("Resolved artifact is missing a valid SHA-256.")
+            artifacts.append({"name": name, "version": version, "sha256": digest.lower()})
+    except (KeyError, TypeError, AttributeError, json.JSONDecodeError):
+        raise ValueError("Invalid pip candidate report.") from None
+    return {
+        "status": "unreviewed-candidate",
+        "scope": "wheel-only dry-run; not installation, approval or age verification",
+        "target": target,
+        "constraints_text_sha256": hashlib.sha256(text.encode("utf-8")).hexdigest(),
+        "artifacts": sorted(artifacts, key=lambda artifact: artifact["name"]),
+    }
+
+
+def main():
+    root = Path(__file__).resolve().parent.parent
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--constraints", type=Path, default=root / "python-constraints.txt")
+    args = parser.parse_args()
+    try:
+        candidate = resolve_candidate(root / "requirements.txt", args.constraints)
+    except (ValueError, OSError) as error:
+        print(f"Candidate probe failed: {error}", file=sys.stderr)
+        return 1
+    print(json.dumps(candidate, indent=2))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())