Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 16 additions & 39 deletions docs/evidence/dropin-certification-verdict.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"schema": "pi.dropin.certification_verdict.v1",
"git_commit": "52e9fbfb24352045985b59df9d7ea63f1f8f2ef8",
"generated_at_utc": "2026-05-18T19:37:26Z",
"overall_verdict": "CERTIFIED",
"git_commit": "ba07485eb12504f52aea9f519bbc8c1ddb50743a",
"generated_at_utc": "2026-06-29T12:09:13Z",
"overall_verdict": "NOT_CERTIFIED",
"hard_gate_results": [
{
"gate_id": "non_mock_unit",
Expand Down Expand Up @@ -30,17 +30,17 @@
},
{
"gate_id": "ext_provider_compat",
"status": "pass",
"status": "skip",
"blocking": false,
"detail": null,
"detail": "Artifact not found: tests/ext_conformance/reports/provider_compat/provider_compat_report.json",
"artifact_path": "tests/ext_conformance/reports/provider_compat/provider_compat_report.json",
"bead": "bd-1f42.4.6"
},
{
"gate_id": "evidence_bundle",
"status": "pass",
"status": "fail",
"blocking": false,
"detail": null,
"detail": "summary.verdict = 'insufficient' (expected one of: [\"complete\", \"partial\"])",
"artifact_path": "tests/evidence_bundle/index.json",
"bead": "bd-1f42.6.8"
},
Expand All @@ -54,9 +54,9 @@
},
{
"gate_id": "conformance_regression",
"status": "pass",
"status": "fail",
"blocking": true,
"detail": null,
"detail": "Artifact not found: tests/ext_conformance/reports/regression_verdict.json; fail-closed policy: blocking gates cannot remain in skip state",
"artifact_path": "tests/ext_conformance/reports/regression_verdict.json",
"bead": "bd-1f42.4"
},
Expand Down Expand Up @@ -163,25 +163,11 @@
"detail": null,
"artifact_path": "tests/full_suite_gate/waiver_audit.json",
"bead": "bd-1f42.8.8.1"
},
{
"gate_id": "G04-cli-parity",
"status": "pass",
"blocking": true,
"detail": "gap-cli-slash-command-surface resolved: tests/dropin_slash_differential.rs now requires real mirrored Rust Pi/pi-mono RPC pass evidence for credential-free slash scenarios, with UI-only, process-exit, invalid parser, and model-dependent compaction scenarios explicitly excluded from pass evidence.",
"artifact_path": "docs/evidence/dropin-differential-evidence-suite.json",
"bead": "bd-7derw"
},
{
"gate_id": "G10-differential-evidence-suite",
"status": "pass",
"blocking": true,
"detail": "dropin-differential-evidence-suite passes with slash_command_differential backed by real mirrored RPC execution.",
"artifact_path": "docs/evidence/dropin-differential-evidence-suite.json",
"bead": "bd-7derw"
}
],
"blocking_reasons": [],
"blocking_reasons": [
"conformance_regression:fail (Artifact not found: tests/ext_conformance/reports/regression_verdict.json; fail-closed policy: blocking gates cannot remain in skip state)"
],
"evidence_index": [
{
"path": "docs/non-mock-rubric.json",
Expand All @@ -197,7 +183,7 @@
},
{
"path": "tests/ext_conformance/reports/provider_compat/provider_compat_report.json",
"exists": true
"exists": false
},
{
"path": "tests/evidence_bundle/index.json",
Expand All @@ -209,7 +195,7 @@
},
{
"path": "tests/ext_conformance/reports/regression_verdict.json",
"exists": true
"exists": false
},
{
"path": "tests/ext_conformance/reports/conformance_summary.json",
Expand Down Expand Up @@ -262,20 +248,11 @@
{
"path": "tests/full_suite_gate/waiver_audit.json",
"exists": true
},
{
"path": "docs/evidence/dropin-differential-evidence-suite.json",
"exists": true
},
{
"path": "docs/evidence/dropin-parity-gap-ledger.json",
"exists": true
}
],
"source": {
"certification_lane_artifact": "tests/full_suite_gate/certification_verdict.json",
"lane_verdict": "pass",
"lane_generated_at": "2026-05-01T06:00:46.624Z",
"claim_integrity_override": "bd-jgnx0 reopened gap-cli-slash-command-surface after detecting that slash_command_differential pass evidence was backed by fail-closed placeholder evidence; bd-7derw restored real mirrored RPC execution and closed the gap."
"lane_verdict": "fail",
"lane_generated_at": "2026-06-29T12:09:13.017Z"
}
}
81 changes: 53 additions & 28 deletions tests/evidence_bundle/bundle_report.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
# Unified CI Evidence Bundle

> Generated: 2026-05-12T19:26:21Z
> Git ref: 80edf7ef9
> CI run: local-20260512T192621Z
> Verdict: **COMPLETE**
> Generated: 2026-06-29T12:09:11Z
> Git ref: ba07485e
> CI run: 28370238357
> Verdict: **INSUFFICIENT**

## Summary

| Metric | Value |
|--------|-------|
| Total sections | 29 |
| Present | 29 |
| Missing | 0 |
| Invalid | 0 |
| Total artifacts | 2004 |
| Total size | 181069.7 KB |
| Present | 18 |
| Missing | 10 |
| Invalid | 1 |
| Total artifacts | 93 |
| Total size | 792.5 KB |
| Required present | 12/12 |

## Conformance (6)
Expand All @@ -23,29 +23,29 @@
|---------|--------|-------|------|------|
| Extension conformance summary | PASS | 1 | 1058 B | `tests/ext_conformance/reports/conformance_summary.json` |
| Conformance baseline | PASS | 1 | 16247 B | `tests/ext_conformance/reports/conformance_baseline.json` |
| Conformance event log | PASS | 1 | 223532 B | `tests/ext_conformance/reports/conformance_events.jsonl` |
| Conformance event log | MISS | 0 | 0 B | `tests/ext_conformance/reports/conformance_events.jsonl` |
| Conformance report (Markdown) | PASS | 1 | 48168 B | `tests/ext_conformance/reports/CONFORMANCE_REPORT.md` |
| Regression gate verdict | PASS | 1 | 1658 B | `tests/ext_conformance/reports/regression_verdict.json` |
| Conformance trend data | PASS | 1 | 3075 B | `tests/ext_conformance/reports/conformance_trend.jsonl` |
| Regression gate verdict | MISS | 0 | 0 B | `tests/ext_conformance/reports/regression_verdict.json` |
| Conformance trend data | MISS | 0 | 0 B | `tests/ext_conformance/reports/conformance_trend.jsonl` |

## Diagnostics (8)

| Section | Status | Files | Size | Path |
|---------|--------|-------|------|------|
| Must-pass gate verdict | PASS | 1 | 1206 B | `tests/ext_conformance/reports/gate/must_pass_gate_verdict.json` |
| Must-pass gate event log | PASS | 1 | 56814 B | `tests/ext_conformance/reports/gate/must_pass_events.jsonl` |
| Per-extension failure dossiers | PASS | 34 | 140366 B | `tests/ext_conformance/reports/dossiers` |
| Health & regression delta report | PASS | 3 | 88320 B | `tests/ext_conformance/reports/health_delta` |
| Provider compatibility matrix | PASS | 753 | 375207 B | `tests/ext_conformance/reports/provider_compat` |
| Sharded extension matrix reports | PASS | 12 | 280518 B | `tests/ext_conformance/reports/sharded` |
| Extension journey report | PASS | 1 | 14742 B | `tests/ext_conformance/reports/journeys/journey_report.json` |
| Auto-repair summary | PASS | 1 | 56700 B | `tests/ext_conformance/reports/auto_repair_summary.json` |
| Must-pass gate verdict | PASS | 1 | 1197 B | `tests/ext_conformance/reports/gate/must_pass_gate_verdict.json` |
| Must-pass gate event log | PASS | 1 | 53944 B | `tests/ext_conformance/reports/gate/must_pass_events.jsonl` |
| Per-extension failure dossiers | MISS | 0 | 0 B | `tests/ext_conformance/reports/dossiers` |
| Health & regression delta report | PASS | 3 | 80613 B | `tests/ext_conformance/reports/health_delta` |
| Provider compatibility matrix | MISS | 0 | 0 B | `tests/ext_conformance/reports/provider_compat` |
| Sharded extension matrix reports | MISS | 0 | 0 B | `tests/ext_conformance/reports/sharded` |
| Extension journey report | PASS | 1 | 1185 B | `tests/ext_conformance/reports/journeys/journey_report.json` |
| Auto-repair summary | MISS | 0 | 0 B | `tests/ext_conformance/reports/auto_repair_summary.json` |

## E2e (1)

| Section | Status | Files | Size | Path |
|---------|--------|-------|------|------|
| E2E test results | PASS | 1176 | 183247322 B | `tests/e2e_results` |
| E2E test results | PASS | 74 | 256474 B | `tests/e2e_results` |

## Quarantine (2)

Expand All @@ -61,28 +61,53 @@
| Performance budget summary | PASS | 1 | 4082 B | `tests/perf/reports/budget_summary.json` |
| PERF-3X comparison report | PASS | 1 | 6350 B | `tests/perf/reports/perf_comparison.json` |
| PERF-3X parameter sweeps report | PASS | 1 | 1076 B | `tests/perf/reports/parameter_sweeps.json` |
| PERF-3X stress triage report | PASS | 1 | 4829 B | `tests/perf/reports/stress_triage.json` |
| Extension load-time benchmark | PASS | 1 | 17566 B | `tests/ext_conformance/reports/load_time_benchmark.json` |
| PERF-3X lineage coherence contract | PASS | 0 | 0 B | `tests/ext_conformance/reports/gate/must_pass_gate_verdict.json | tests/ext_conformance/reports/conformance_summary.json | tests/perf/reports/stress_triage.json` |
| PERF-3X stress triage report | PASS | 1 | 4796 B | `tests/perf/reports/stress_triage.json` |
| Extension load-time benchmark | MISS | 0 | 0 B | `tests/ext_conformance/reports/load_time_benchmark.json` |
| PERF-3X lineage coherence contract | WARN | 0 | 0 B | `tests/ext_conformance/reports/gate/must_pass_gate_verdict.json | tests/ext_conformance/reports/conformance_summary.json | tests/perf/reports/stress_triage.json` |

## Security (2)

| Section | Status | Files | Size | Path |
|---------|--------|-------|------|------|
| Security and licensing risk review | PASS | 1 | 86687 B | `tests/ext_conformance/artifacts/RISK_REVIEW.json` |
| Security and licensing risk review | PASS | 1 | 86698 B | `tests/ext_conformance/artifacts/RISK_REVIEW.json` |
| Extension provenance verification | PASS | 1 | 143396 B | `tests/ext_conformance/artifacts/PROVENANCE_VERIFICATION.json` |

## Traceability (2)

| Section | Status | Files | Size | Path |
|---------|--------|-------|------|------|
| Requirement-to-test traceability matrix | PASS | 1 | 89373 B | `docs/traceability_matrix.json` |
| Requirement-to-test traceability matrix | PASS | 1 | 94477 B | `docs/traceability_matrix.json` |
| High-value suite artifact inventory | PASS | 1 | 11209 B | `docs/evidence/high-value-suite-artifact-inventory.json` |

## Inventory (2)

| Section | Status | Files | Size | Path |
|---------|--------|-------|------|------|
| Extension inventory | PASS | 1 | 89349 B | `tests/ext_conformance/reports/inventory.json` |
| Extension inclusion manifest | PASS | 4 | 406062 B | `tests/ext_conformance/reports/inclusion_manifest` |
| Extension inventory | MISS | 0 | 0 B | `tests/ext_conformance/reports/inventory.json` |
| Extension inclusion manifest | MISS | 0 | 0 B | `tests/ext_conformance/reports/inclusion_manifest` |

## Missing / Invalid Sections

- **Conformance event log** (missing): File not found
Path: `tests/ext_conformance/reports/conformance_events.jsonl`
- **Regression gate verdict** (missing): File not found
Path: `tests/ext_conformance/reports/regression_verdict.json`
- **Conformance trend data** (missing): File not found
Path: `tests/ext_conformance/reports/conformance_trend.jsonl`
- **Per-extension failure dossiers** (missing): Directory not found
Path: `tests/ext_conformance/reports/dossiers`
- **Provider compatibility matrix** (missing): Directory not found
Path: `tests/ext_conformance/reports/provider_compat`
- **Sharded extension matrix reports** (missing): Directory not found
Path: `tests/ext_conformance/reports/sharded`
- **Auto-repair summary** (missing): File not found
Path: `tests/ext_conformance/reports/auto_repair_summary.json`
- **Extension load-time benchmark** (missing): File not found
Path: `tests/ext_conformance/reports/load_time_benchmark.json`
- **Extension inventory** (missing): File not found
Path: `tests/ext_conformance/reports/inventory.json`
- **Extension inclusion manifest** (missing): Directory not found
Path: `tests/ext_conformance/reports/inclusion_manifest`
- **PERF-3X lineage coherence contract** (invalid): PERF-3X lineage span exceeds 14 days for run_id '28370238357' (oldest=2026-05-12 19:26:24 UTC, newest=2026-06-29 12:09:10 UTC)
Path: `tests/ext_conformance/reports/gate/must_pass_gate_verdict.json | tests/ext_conformance/reports/conformance_summary.json | tests/perf/reports/stress_triage.json`

Loading