[wbreza]

Summary

Separate lint/run/report jobs currently require artifact IDs and digests to be exported through job outputs, plus two conditional reporting steps. This change lets one command: report step discover matching structured artifacts using a shared report-id (default evals), without that wiring.

Discovery uses the authenticated artifact listing for the current workflow run and attempt, verifies SHA-256 digests and downloaded report identities, and rejects ambiguous, expired, or missing evidence. It reports lint only with a warning when no run artifact exists, never claiming an evaluation pass. Explicit primary ID/digest pairs bypass discovery for backward compatibility; lint-only overrides can accompany a discovered run.

The simplified workflow requires actions: read on the reporting job. Comment concurrency remains caller-configured and required; execution permissions and native evaluation behavior are unchanged. README examples cover the simplified workflow, custom identities, and explicit overrides. The distribution is rebuilt.

Tracking issue: N/A.

Verification

• Model-free checks and source-to-bundle verification pass.
• Native integration evidence is distinguished from synthetic controls.
• Credential, process and artifact boundaries were reviewed.
• Documentation and packaged dist match source.

Commands run locally with Node 24 and pnpm 12.8.1:

• pnpm exec vitest run tests\inputs.test.ts tests\artifacts.test.ts tests\action.test.ts tests\reporter.test.ts - passed, 104 tests at the focused checkpoint.
• pnpm build - passed; generated dist/index.cjs committed.
• pnpm check - passed, including type checking, lint, 165 model-free tests, coverage thresholds, source-to-bundle comparison, and the clean consumer-directory packaging control.
• git diff --check - passed.

Real native CLI controls use the pinned Vally CLI, built-in mock executor, and deterministic graders. Artifact discovery, digest failures, permissions, pagination, identity rejection, overrides, and comment handling are covered with synthetic GitHub responses; those tests do not establish hosted API/token acceptance. No models were run.

Limitations

Hosted acceptance of automatic artifact discovery and cross-job reporting with actions: read remains unverified; this PR stays draft pending that integration evidence. The supported Action host remains Linux x64, while local validation was performed on Windows.

Discovery does not reuse prior-attempt artifacts: rerun the producing jobs along with reporting. Older Action uploads require explicit IDs/digests. Duplicate matching producers require distinct report identities or explicit artifact selection. This change does not add a reusable reporting workflow or move the concurrency guard inside the Action.

[github-actions]

Evals: passed

Suite: copilot-smoke

Lint: validated (no model execution) - Evidence

Run: passed - 1 trial - Evidence

Eval	Trials	Result
copilot-skill-smoke	1	passed

Exact revision | Workflow run

[Copilot]

🔵 Needs a closer look

Hosted Linux cross-job discovery and real-token permission acceptance still lack integration evidence.

2 open issues

• Use a safe artifact name when command validation fails · New
• Document actions:read requirement for artifact discovery · New

What changed in this PR

This PR simplifies separate lint, run, and report jobs by discovering structured artifacts through a shared report identity, without changing native evaluation behavior.

Changes:

• Adds current-run and attempt-scoped discovery with digest and report-identity validation.
• Preserves explicit artifact overrides and supports lint-only reporting without claiming an evaluation pass.
• Documents simpler workflow wiring and adds coverage for discovery, rejection paths, and compatibility.

File	Description
tests/​reporter.test.ts	Tests shared-identity lint companions and consistency checks.
tests/​inputs.test.ts	Tests discovery defaults and paired artifact overrides.
tests/​example.test.ts	Validates the simplified three-job README workflow.
tests/​artifacts.test.ts	Covers discovery, pagination, ambiguity, and invalid metadata.
tests/​action.test.ts	Covers automatic reporting, overrides, and evidence validation.
src/​reporting.ts	Allows multiple expected report identities.
src/​reporter.ts	Accepts shared-identity lint companions.
src/​inputs.ts	Makes primary artifact overrides optional but paired.
src/​artifacts.ts	Adds artifact naming and scoped discovery.
src/​action.ts	Integrates discovery, lint fallback, and command validation.
README.md	Documents simplified jobs, permissions, identities, and overrides.
CONTRIBUTING.md	Defines discovery invariants and synthetic testing boundaries.
action.yml	Updates input descriptions for automatic discovery.

_{Review effort: Max}