feat(scorecard): run the first four-way, and add the column it proved was missing (G4)

[henleda]

benchmarks/RESULTS.md now exists, so G4 is checked.

First real sweep of the Nimbus fixture, dgx skipped — the local Ollama box is behind a tunnel this
machine cannot reach. --skip was added for exactly that: an unreachable config is declared as
absent from the table rather than recorded as a model that failed, which would be a false red row.

config	recall	precision	triage	policies (unusable)	wall
claude	9/9 = 1.00	1.00	8/9 = 0.89	9 (0)	195s
openai	8/9 = 0.89	1.00	8/8 = 1.00	7 (0)	53s
gemini	6/9 = 0.67	1.00	6/6 = 1.00	6 (3)	394s

The run proved the table needed a column the acceptance never asked for

Gemini scored precision 1.00 and triage 1.00 while emitting {} as the spec for 3 of its 6
policies. Structured output validated — {} is a valid dict — and the artifact was useless.

Recall, precision and triage all say the routing was right. None of them notice. Published as
specified, this table would have flattered a model that routed correctly and then emitted nothing.

policies (unusable) counts what lint_generated_spec rejects — an empty spec, no DENY rule, an
OpenAPI fragment missing its envelope — reusing machinery the pipeline already runs on every scan.
No new judgement, just a count that reaches the scorecard.

Flagging it as beyond the stated acceptance: I added it because the alternative was publishing a
misleading table.

Which surfaced an older bug in turn

Counting policies by globbing policies/ gave 32 for Claude's 9 generated — a re-scan into an
existing out dir leaves the previous run's artifacts behind. The count now comes from the run's own
policies.json index, so a model is never scored against someone else's leftovers. Pinned by a test.

Also in the table, deliberately

• Not reproducible run to run — stated rather than faked; two providers accept no seed and none
  guarantee determinism.
• Not a cost comparison — token cost absent by decision, and the table says so.
• Not the full config set — dgx named as skipped-because-unreachable.

Six new tests · 268 pass · coverage 73% · ruff clean.

🤖 Generated with Claude Code