From 2ce392535ecb464f850e7787d9ab30331a91f4f5 Mon Sep 17 00:00:00 2001 From: stacknil Date: Thu, 28 May 2026 11:10:50 +0800 Subject: [PATCH] docs(sbom): map example evidence types --- .../docs/example-artifact-regeneration.md | 3 +++ .../docs/reviewer-evidence-pack.md | 10 +++++++ .../sbom-diff-and-risk/docs/reviewer-path.md | 27 ++++++++++++++++++- 3 files changed, 39 insertions(+), 1 deletion(-) diff --git a/tools/sbom-diff-and-risk/docs/example-artifact-regeneration.md b/tools/sbom-diff-and-risk/docs/example-artifact-regeneration.md index 71990b1..5b34397 100644 --- a/tools/sbom-diff-and-risk/docs/example-artifact-regeneration.md +++ b/tools/sbom-diff-and-risk/docs/example-artifact-regeneration.md @@ -70,6 +70,9 @@ services, and does not make dependency safety claims. Provenance-aware, Scorecard-aware, and enriched SARIF sample artifacts remain covered by their focused golden tests because those examples include mocked evidence. +For reviewer-facing interpretation of each checked-in example artifact, see +[reviewer-path.md#artifact-evidence-map](reviewer-path.md#artifact-evidence-map). + For `examples/sample-sarif.sarif`, the script normalizes the generated `originalUriBaseIds.%SRCROOT%.uri` value to `file:///__PROJECT_ROOT__/` so checked-in output does not depend on a maintainer's local checkout path. diff --git a/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md b/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md index 1894d23..5be90e2 100644 --- a/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md +++ b/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md @@ -4,6 +4,11 @@ This page is a reproducible evidence checklist for reviewing `sbom-diff-and-risk For the shortest ordered route through these materials, start with [reviewer-path.md](reviewer-path.md). + +For interpreting checked-in examples, use the +[artifact evidence map](reviewer-path.md#artifact-evidence-map). It separates +deterministic no-network examples, mocked enrichment snapshots, and consumer +workflow templates. ## Project Identity @@ -69,6 +74,11 @@ python scripts/regenerate-example-artifacts.py --check For the exact regeneration scope, see [example-artifact-regeneration.md](example-artifact-regeneration.md). +This check covers deterministic no-network examples only. Provenance-aware and +Scorecard-aware checked-in examples are focused rendering snapshots built by +golden tests with constructed evidence; they are not live PyPI or Scorecard +lookups. + Generate the strict-policy JSON sidecar: ```powershell diff --git a/tools/sbom-diff-and-risk/docs/reviewer-path.md b/tools/sbom-diff-and-risk/docs/reviewer-path.md index 80cb310..c229c61 100644 --- a/tools/sbom-diff-and-risk/docs/reviewer-path.md +++ b/tools/sbom-diff-and-risk/docs/reviewer-path.md @@ -49,7 +49,7 @@ Inspect the checked-in examples in this order: | Step | Artifact | What it proves | | --- | --- | --- | -| 1 | [sample-report.json](../examples/sample-report.json) | Full machine-readable diff, risk, policy, and metadata shape. | +| 1 | [sample-report.json](../examples/sample-report.json) | Default machine-readable diff, risk, and metadata shape. | | 2 | [sample-summary.json](../examples/sample-summary.json) | Compact CI-facing `summary` contract. | | 3 | [sample-policy.json](../examples/sample-policy.json) | Policy-only sidecar for CI consumers. | | 4 | [sample-report.md](../examples/sample-report.md) | Human-readable reviewer report. | @@ -73,6 +73,31 @@ Look for these reviewer anchors: Stop here if you need to understand the review outputs without running code. +## Artifact evidence map + +Use this map before interpreting checked-in examples. The example directory +contains deterministic no-network artifacts, mocked enrichment snapshots, and +consumer workflow templates. They answer different review questions. + +| Evidence type | Artifacts | How they are produced | Network status | Reviewer use | +| --- | --- | --- | --- | --- | +| Default diff output | [sample-report.json](../examples/sample-report.json), [sample-summary.json](../examples/sample-summary.json), [sample-report.md](../examples/sample-report.md) | `scripts/regenerate-example-artifacts.py` with the CycloneDX example inputs | No network | Verify the default report, compact summary, and Markdown shapes. | +| Policy output | [sample-policy-warn-report.json](../examples/sample-policy-warn-report.json), [sample-policy-fail-report.json](../examples/sample-policy-fail-report.json), [sample-policy.json](../examples/sample-policy.json) | `scripts/regenerate-example-artifacts.py` with local policy files | No network | Review warning, blocking, and policy-only sidecar semantics. | +| Requirements output | [sample-requirements-report.json](../examples/sample-requirements-report.json), [sample-requirements-report.md](../examples/sample-requirements-report.md) | `scripts/regenerate-example-artifacts.py --only requirements` | No network | Verify the `requirements.txt` parser path and report shape. | +| Strict-policy SARIF | [sample-sarif.sarif](../examples/sample-sarif.sarif) | `scripts/regenerate-example-artifacts.py --only sarif` with normalized source root | No network | Inspect conservative code-scanning output for selected findings. | +| PyPI provenance snapshot | [sample-provenance-report.json](../examples/sample-provenance-report.json), [sample-provenance-report.md](../examples/sample-provenance-report.md), [sample-provenance-report.sarif](../examples/sample-provenance-report.sarif) | Focused golden tests with constructed provenance evidence | No live lookup while testing | Review opt-in provenance rendering and policy semantics, not current PyPI package truth. | +| OpenSSF Scorecard snapshot | [sample-scorecard-report.json](../examples/sample-scorecard-report.json), [sample-scorecard-report.md](../examples/sample-scorecard-report.md), [sample-scorecard-report.sarif](../examples/sample-scorecard-report.sarif) | Focused golden tests with constructed Scorecard evidence | No live lookup while testing | Review opt-in Scorecard rendering and explicit policy gating, not current repository reputation. | +| Consumer workflows | [github-actions-consumer.yml](../examples/github-actions-consumer.yml), [github-actions-policy-consumer.yml](../examples/github-actions-policy-consumer.yml) | Checked-in examples | Not executed by this repository | Copy or inspect CI consumption patterns. | + +Reviewer rule: + +- no-network generated examples prove deterministic local output still matches + the code +- mocked enrichment snapshots prove report rendering and policy interpretation + for opt-in evidence shapes +- release verification docs prove distribution evidence for the tool itself +- none of these artifacts prove that a third-party dependency is safe + ## 15-minute reproduction check From `tools/sbom-diff-and-risk`, run the deterministic example checks: