Repository navigation
feat(assurance): claim register, assurance case and regulation mapping (T17) - #41
nick-developer wants to merge 9 commits into
Conversation
Customers' privacy, legal and audit teams need every claim in the documentation traced to a proof, a test or a measured report (roadmap T17). tools/claims_check.py checks docs/assurance/claims.yaml: quotes must still be in their documents, evidence must resolve (proposition labels numbered as LaTeX prints them, check ids, pytest nodes, report files), claim-like sentences of the documents in scope must be registered or exempted with a reason, and Prop. N references in any Markdown document must name a defined proposition. tests/unit/test_assurance.py runs the check, tests each kind of drift it must catch, and confines network access to the service modules the security guide names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSVqwE13trs5MdpBoE4YyJ Signed-off-by: Claude <noreply@anthropic.com>
… governance guides 60 claims of docs/privacy.md, docs/dp.md, docs/security.md and docs/governance/guide.md, each traced to propositions, checks, tests or the aggregate result files now kept in docs/assurance/evidence/. Registering them found statements in docs/security.md that 2.0 made false: 'formal differential privacy is not part of v1.0', 'privacy checks run by default' (the distance guards are not a privacy check, docs/privacy.md), five core dependencies (seven), subprocess used once (four places) and no network access (the service talks to the OIDC provider and Vault). The guide now says what the code does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSVqwE13trs5MdpBoE4YyJ Signed-off-by: Claude <noreply@anthropic.com>
…d release guidance The deliverables of roadmap T17 for customers' privacy, legal and audit teams: - assurance-case.md: top claim, sub-claims for structure, measured and formal privacy, governance and traceability, assumptions A1-A5 and defeaters, every leaf citing the claim register or a proposition; - regulation-mapping.md: EDPB Guidelines 02/2026 (v1.0, paras. 6-103) and EDPS v SRB (C-413/23 P, paras. 71-112), AI Act Art. 10 as amended by Regulation (EU) 2026/1744 (Art. 10(5) moved to Art. 4a), 45 CFR 164.514 and the HHS 2012 guidance, NIST SP 800-226 and SP 800-188, and ISO/IEC 27559 and 20889 by topic (texts not available), with rows a lawyer must decide marked as such; - dpia-template.md following GDPR Art. 35(7); - release-guidance.md for internal, partner and public recipients, with the Five Safes and an approval checklist. The claim checker now also fails when these documents cite a claim id, a proposition label or a check that does not exist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSVqwE13trs5MdpBoE4YyJ Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSVqwE13trs5MdpBoE4YyJ Signed-off-by: Claude <noreply@anthropic.com>
…ward ranges Sections may name a fixed number in the title ([...; Prop.~950]) while LaTeX prints the running number; references to the named number are valid. A reference such as 'Props. 951-902' is now reported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSVqwE13trs5MdpBoE4YyJ Signed-off-by: Claude <noreply@anthropic.com>
test_settings_reference_is_up_to_date fails on main since the runtime settings (#35) landed after the reference was generated (#39); the unit jobs of every PR are red until it is regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSVqwE13trs5MdpBoE4YyJ Signed-off-by: Claude <noreply@anthropic.com>
|
The unit jobs failed on every platform with one test: Generated by Claude Code |
|
After the settings fix, The cause is a false positive in the test. Line 120 scans every raw file under the data folder for the 3-byte marker Proposed fix (not pushed here, because it belongs to the service tests). Line 117 already checks the decrypted content, so skip the ciphertext in the raw scan: for f in Path(settings.data_dir).rglob("*"):
if f.is_file() and f.suffix != ".enc": # ciphertext is checked decrypted above
assert SECRET_MARK.encode() not in f.read_bytes(), fA second option is a marker of 16 or more bytes, which removes the chance collision. I can't re-run the job from here, so a maintainer re-run should go green. I'll keep watching this PR. Generated by Claude Code |
test_no_source_row_is_persisted searched every file under the data folder, including the encrypted artefacts, for a 3-byte marker. Random ciphertext contains it by chance (macOS unit job on #41); the artefacts are already checked decrypted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PSVqwE13trs5MdpBoE4YyJ Signed-off-by: Claude <noreply@anthropic.com>
…-nka4jq Signed-off-by: Claude <noreply@anthropic.com> # Conflicts: # .github/workflows/ci.yml # CHANGELOG.md
…ing on Windows
Put README.md in the claim register's scope with 36 entries (RM-01..RM-36).
A README sentence that sums up a guide cites that guide's claim with the new
{claim: ID} evidence kind, which is as strong as the claim it cites.
README fixes found while registering: 'No row is copied' becomes 'Rows are
drawn from a fitted model; only tables you declare mode: copy are copied';
'Privacy checks run by default' becomes copy guards; C700-C706 do not exist,
the 2.0 checks are C600-C606 and the generator propositions are unit-tested.
The checker tests wrote their fixture files in the platform encoding, which
broke on Windows (cp1252); they now write UTF-8.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSVqwE13trs5MdpBoE4YyJ
Signed-off-by: Claude <noreply@anthropic.com>
Requested by Cod3r · project thread
What and why
Roadmap T17: give customers' privacy, legal and audit teams the evidence they need to approve a relsynth release. New folder
docs/assurance/:claims.yaml(60 claims) and checkertools/claims_check.py, run in the CI joblint. Every claim-like sentence of docs/privacy.md, docs/dp.md, docs/security.md and docs/governance/guide.md is bound to its quote and to propositions (by label, numbered as LaTeX prints them), checks, pytest nodes or measured reports. CI fails when a quote leaves its document, evidence disappears, a new claim is unregistered, aProp. Nanywhere in the docs names an undefined proposition, or a range runs backwards.docs/assurance/evidence/: aggregate result files of T06 and T07 that claims cite (scores and intervals only, no rows or person labels; datasets all marked allowed).Registering the claims found statements in docs/security.md that 2.0 made false (no formal DP, "privacy checks run by default", five core dependencies, one
subprocess, no network access). The guide now says what the code does. A new test confines network access to the service modules that the guide names.README: one row in the documentation map. The README claims join the register after the 2.0.0 release PR lands, because that PR rewrites the README.
How it was tested
ruff check src tests tools bench: all checks passed;python tools/check_spdx.py: okpython tools/claims_check.py:ok: 60 claims traced, 4 documents in scope, 152 propositionspytest tests/unit/test_assurance.py tests/unit/test_security.py: 57 passed. test_assurance covers the register, LaTeX counter numbering (including save-and-restore sections), each kind of drift, citations in the assurance documents, and network confinementProps. 951–902in docs/testing.md), and the coordinator has been toldsrc/); CI runs themStatistics and privacy
src/)Checklist
git commit -s)UnreleasedGenerated by Claude Code