SOL ExecBench ROCm 4.0

SOL ExecBench ROCm evaluates GPU kernel candidates and derives formal Speed-of-Light bounds for the AMD Radeon RX 9060 XT. The distribution contains two sibling packages with a deliberately narrow boundary:

• sol_execbench owns problem/workload contracts, corpus generation, input generation, candidate correctness and timing, local score formulae, aggregation policy, security, and the CLI.
• solar owns only the three stages described in SOL-ExecBench §4.2: operator graph extraction, validated extended-einsum conversion, and SOL analysis.

The paper is the source of truth when imported implementations disagree: SOL-ExecBench.

Install

uv sync --all-groups

Only Python 3.12 is supported. Linux x86-64 resolves the ROCm 7.2 PyTorch stack declared in pyproject.toml.

Public problem corpus

The hardware-independent, text-backbone corpus is frozen at problems/LLM_CORE/releases/LLM_CORE_V2/manifest.yaml. It contains 36 production-derived Definitions and 36 frozen workload-generation rules across dense, attention, MoE, KV cache, long-context, and quantization profiles. It does not freeze concrete Workloads. Validate it or generate a target view from an isolated measurement of currently allocatable GPU memory:

uv run sol-execbench dataset corpus validate
uv run sol-execbench dataset corpus generate \
  --target-template gfx1200 \
  --device cuda:0 \
  --profile core \
  --output problems/local/LLM_CORE/gfx1200-measured

The bundled gfx1200 and gfx942 files declare capability, dtype, quantization, tensor, and IPC limits. A bounded allocation probe measures usable quota, which is floored to a frozen capacity class. Each Definition then generates exactly nine slots at one common scale or is skipped atomically. Exact free bytes and other audit-only hardware fields cannot change workloads inside one cohort. Leaderboard submission, ranking, and hosted scoring remain out of scope.

The legacy AKA-reviewed selection remains tracked at problems/AMD_AKA/manifest.yaml. Problems are derived from AMD AgentKernelArena (AKA, Apache-2.0) under the SOL-ExecBench paper §3 construction methodology: each problem's PyTorch reference is the AKA task's own correctness oracle (module_fn), lifted into a standalone def run(...). The manifest pins the exact AKA commit and records per-task file checksums so the generated problems are bound to that source revision. The current seed set covers matmul, softmax, norm, elementwise, and conv operations across FP32 / BF16 / FP16.

Authored definitions and workloads are committed under problems/AMD_AKA/<suite>/<name>/. Materialization detects the selected ROCm device, filters incompatible workloads with static target rules plus a trusted reference/harness probe, and writes an auditable target tree under problems/local/AMD_AKA/<gfx-target>/:

bash scripts/fetch_aka_source.sh
uv run sol-execbench dataset materialize
uv run sol-execbench dataset audit problems/local/AMD_AKA/gfx1200

dataset audit also verifies the local AKA clone is pinned to the manifest revision and that every problem's per-task checksums match (the problems-bound-to-commit binding). The public Definition contract requires a non-empty op_type; the AKA importer records it per problem.

Evaluate and analyze

Candidate evaluation remains an outer-project operation:

./scripts/run_docker.sh -- sol-execbench evaluate \
  problems/local/AMD_AKA/gfx1200/torch2hip/l1n1_square_matmul \
  --solution /sol-execbench/path/to/solution.json \
  --trace-output /outputs/matmul.trace.jsonl

The intended formal SOLAR target is RX 9060 XT gfx1200. This port's formal publication policy requires the pinned Orojenesis toolchain; that is an explicit ROCm release constraint, not a claim that the paper mandates this tool for every SOLAR use. The architecture audit is packaged and content-addressed, with its throttled resource measurements explicitly limited to instruction/runtime corroboration. The reproducibly built mapper digest is reviewed and present in the release allowlist, so the formal producer reports ready. Official scoring remains unavailable because the corpus policy is still pending and no repository release bundle has been published. An executable whose digest is only self-declared by its local provenance manifest is still rejected:

uv run sol-execbench solar analyze \
  problems/local/AMD_AKA/gfx1200/torch2hip/l1n36_rmsnorm \
  --workload aka-l1n36_rmsnorm-w0 \
  --orojenesis-home /path/to/pinned/orojenesis \
  --output out/solar/norm_forward_bf16

The isolated worker publishes an atomic directory whose canonical top-level files are operator_graph.yaml, the selected IR graph, conversion-attestation.yaml, solar-analysis.yaml, and manifest.yaml. When tile-aware analysis runs, the directory also contains the recursively content-addressed Orojenesis evidence referenced by the manifest; successful mapper logs are not published. The worker never receives candidate runtimes or computes scores.

Before a full SOLAR release, run the mandatory uniform qualification chain:

uv run sol-execbench solar qualify-static out/release \
  --orojenesis-home /path/to/orojenesis \
  --qualification-root out/solar-qualification
uv run sol-execbench solar qualify-canary out/release \
  --orojenesis-home /path/to/orojenesis \
  --qualification-root out/solar-qualification
uv run sol-execbench solar qualify-full out/release \
  --orojenesis-home /path/to/orojenesis \
  --qualification-root out/solar-qualification

Full qualification derives its 43-problem / 163-workload denominator from the pinned manifest and writes a content-addressed readiness matrix. Formal release-build refuses to start without the complete content-bound gate chain. See large batch GPU qualification.

SOLAR exposes exactly two fixed paths through --backend: torchview_extended_einsum (the default) and make_fx_aten. Extraction and IR selection cannot be combined independently, a run never falls back to the other path, and a corpus audit or release build uses one path for its complete denominator. Both paths preserve exact source-input, output, tensor-metadata, and effect provenance; unsupported tracing, conversion, replay, or resource accounting fails closed.

Extended schema v6 is a native semantic IR, not serialized ATen. Its operations carry ordered operands, public attributes, effects, and bounded dynamic shapes; the default executor calls public PyTorch APIs independently of the ATen path.

Official score

The schema v7 corpus pins the pending rx9060xt-gfx1200-reference-v2 baseline identity. Official scoring remains unauthorized until its v2 release evidence is published. Once authorized, the fail-closed scorer accepts a publisher-authored release bundle whose SHA-256 references bind the corpus, baseline, candidate execution, and pinned SOLAR manifests:

uv run sol-execbench score official RELEASE/release-bundle.json

sol-execbench score status reports the repository-pinned policy and baseline. Raw caller-authored measurements, baselines, or SOLAR JSON are not scorer inputs. Bundle hashes provide artifact integrity; publisher authenticity comes from the repository or release channel that distributes the bundle.

The release workflow includes baseline release-build, baseline candidate-build, the three baseline qualify-* commands, baseline release-run, the three solar qualify-* commands, solar release-build, score build-statement, and score assemble-bundle. It has one baseline run and one candidate run; it does not require role signatures or an independent rerun. See the release and official-score workflow.

Correct candidates must satisfy T_b > T_SOL and T_k >= T_SOL; workloads are averaged within each problem and then across problems with equal weight.

Development

uv run --with ruff ruff check .
uv run ty check
uv run pytest tests/

GPU tests declare their ROCm and architecture prerequisites. Build the optional container with ./scripts/run_docker.sh --build.

See SOLAR boundary and scoring contract for the normative architecture. The recorded GPU engineering evidence is limited to RX 9060 XT gfx1200 on the locked ROCm 7.2 stack; see RDNA4 validation scope for the exact test, toolchain, self-hosted-runner, and publisher-release boundaries.

License

Apache-2.0. Imported SOLAR files retain their original SPDX attribution.