docs: publish this project's exit-code contract, and flag the family collision

docs/EXIT-CODES.md

@@ -1,20 +1,37 @@
-# Exit Codes (stable contract)
-
-Onboarding scripts and CI gates may rely on these codes. Meanings never
-change; new codes are only appended.
-
-| Code | Meaning |
-| --- | --- |
-| `0` | READY — scan succeeded, no blocking problems |
-| `1` | READY_WITH_WARNINGS — warnings/unknowns found |
-| `2` | BLOCKED — one or more BLOCKED/ERROR findings |
-| `3` | INTERNAL_ERROR — DevRepro itself failed |
-| `4` | USAGE_ERROR — invalid arguments or unreadable policy |
-
-`devrepro preflight` and `devrepro check --policy` are the intended CI
-entry points:
-
-```yaml
-# GitHub Actions example
-- run: python -m devrepro preflight --policy .devrepro.toml
-```
+# Exit codes
+
+`devrepro-doctor` treats its exit codes as a public contract: CI gates and onboarding
+scripts branch on them, so an existing code never changes meaning -- new ones
+are appended.
+
+Defined in [`devrepro/core/exit_codes.py`](https://github.com/webdevsamran/devrepro-doctor/blob/main/devrepro/core/exit_codes.py).
+
+| Code | Name | Meaning |
+|---|---|---|
+| `0` | `READY` | The machine meets every checked requirement. |
+| `1` | `READY_WITH_WARNINGS` | Usable, with non-blocking findings. This is a SUCCESS. |
+| `2` | `BLOCKED` | At least one blocking problem. This is the verdict to act on. |
+| `3` | `INTERNAL_ERROR` | An unexpected error inside the tool. |
+| `4` | `USAGE_ERROR` | The command was invoked incorrectly. |
+
+## If you use more than one of these tools
+
+These four projects are independent and their exit codes are **not** a shared
+vocabulary. Only `0` means the same thing in all of them (success). Every other
+code differs, and two collisions are worth knowing before you write a wrapper:
+
+| Code | api-verity-lab | devrepro-doctor | tooltrace-bench | local-ai-hardware-bench |
+|---|---|---|---|---|
+| 1 | findings detected | **ready, with warnings** | error | validation error |
+| 2 | usage error | **machine blocked** | task validation error | usage error |
+
+The dangerous one is `1`. In devrepro-doctor it means *the machine is usable*;
+in the other three it means something went wrong. A wrapper that treats any
+non-zero status as failure will block on a DevRepro run that reported success.
+
+The second is `2`: an operator mistake in two of them, and devrepro-doctor's
+most important verdict -- the machine cannot build this project -- in the third.
+
+These are not being unified. A shared exit-code library would couple four
+independent release cycles, and one of these projects deliberately ships with
+no dependencies at all. Knowing the difference is cheaper than removing it.

requirements-docs.txt

@@ -1,2 +1,11 @@
-# Pinned below 9.7: later lines changed licensing terms; 9.6.x remains ISC.
+# Pinned to a minor range so a docs build is reproducible, not for licensing
+# reasons. The previous comment here read "Pinned below 9.7: later lines changed
+# licensing terms; 9.6.x remains ISC" and was wrong three ways: mkdocs-material
+# is MIT at 9.5, 9.6, 9.7.0 and 9.7.7 (checked against PyPI classifiers), there
+# is no ISC line, and the pin it annotated had already been moved to >=9.7.7 by
+# the Dependabot bump in #17 -- across the exact boundary the comment declared.
+#
+# That is the part worth remembering: an automated bump walked through a stated
+# constraint because the constraint lived in a comment nothing enforced. If a
+# licence boundary ever does matter here, it needs a check, not a sentence.
 mkdocs-material>=9.7.7,<9.8

tests/test_exit_codes_documented.py

@@ -0,0 +1,68 @@
+"""The documented exit codes must match the ones the code defines.
+
+Every project in this family calls its exit codes a public contract -- CI
+gates and onboarding scripts branch on them, so a code that changes meaning
+breaks something silently, somewhere else. A *document* that disagrees with the
+code is worse than no document, because wrappers get written from the document.
+
+Checking this is cheap, so it is checked.
+"""
+
+from __future__ import annotations
+
+import re
+from pathlib import Path
+
+_ROOT = Path(__file__).resolve().parent.parent
+# Spelled exactly as git tracks it. This repo's file is EXIT-CODES.md and
+# mkdocs.yml references that name; a lowercase path resolves on Windows and
+# fails on Linux, which is how it reached CI green locally and red there.
+_DOC = _ROOT / "docs" / "EXIT-CODES.md"
+
+
+def _documented() -> dict[int, str]:
+    rows = re.findall(
+        r"^\|\s*`(\d+)`\s*\|\s*`([^`]+)`\s*\|", _DOC.read_text(encoding="utf-8"), re.M
+    )
+    return {int(code): name for code, name in rows}
+
+
+def _defined() -> dict[int, str]:
+    src = (_ROOT / "devrepro/core/exit_codes.py").read_text(encoding="utf-8")
+    return {
+        int(m.group(2)): m.group(1) for m in re.finditer(r"^\s{4}([A-Z_]+)\s*=\s*(\d+)", src, re.M)
+    }
+
+
+def test_every_defined_code_is_documented() -> None:
+    documented, defined = _documented(), _defined()
+    missing = sorted(set(defined) - set(documented))
+    assert not missing, f"exit codes defined in code but absent from docs/exit-codes.md: {missing}"
+
+
+def test_no_documented_code_is_invented() -> None:
+    documented, defined = _documented(), _defined()
+    extra = sorted(set(documented) - set(defined))
+    assert not extra, f"docs/exit-codes.md documents codes the code does not define: {extra}"
+
+
+def test_each_code_is_documented_with_its_real_name() -> None:
+    documented, defined = _documented(), _defined()
+    wrong = {
+        code: (documented[code], defined[code])
+        for code in sorted(set(documented) & set(defined))
+        if documented[code] != defined[code]
+    }
+    assert not wrong, f"docs name codes differently from the code: {wrong}"
+
+
+def test_the_cross_project_warning_is_present() -> None:
+    """The collision is the reason this document exists.
+
+    Only `0` means the same thing across the four sibling projects. In
+    devrepro-doctor `1` means the machine is usable; elsewhere it means
+    failure. A wrapper treating non-zero as failure blocks a successful run.
+    """
+    text = _DOC.read_text(encoding="utf-8")
+    assert "If you use more than one of these tools" in text
+    assert "devrepro-doctor" in text