Skip to content

backend: Add MX quant/dequant on-the-fly compute (size-changing datapath) - #167

Open
DanielKellerM wants to merge 2 commits into
backend/w-beat-done-retirefrom
otf/size-changing-compute
Open

backend: Add MX quant/dequant on-the-fly compute (size-changing datapath)#167
DanielKellerM wants to merge 2 commits into
backend/w-beat-done-retirefrom
otf/size-changing-compute

Conversation

@DanielKellerM

@DanielKellerM DanielKellerM commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds size-changing on-the-fly compute (write byte count != read byte count) on top of the transpose/otf_compute framework, with MX (OCP microscaling) quantization and dequantization as the first ops. Stacked on #166 (backend/w-beat-done-retire), which provides the per-beat write retire this depends on.

Mechanism. Per-op source:dest byte ratios in idma_pkg (compute_in_bytes/compute_out_bytes) program the legalizer write length, so the write burst's beat count matches the beats the compute engine emits. The compute seam presents per-lane valid (pack-buffer occupancy) with full strobes; the write manager's own beat mask gates acceptance, and the engine retires lane-exactly against the byte lanes the write port actually consumed (buffer_out_ready), so a partial tail beat pops only its own bytes — no drain FSM, no transport-layer state machine, and back-to-back transfers with an identical compute config pipeline with zero inter-transfer bubbles. The legalizer asserts fence the contract: input-granule alignment, beat-aligned src/dst, per-beat-capable destination protocols (no TileLink), the FP16 width cap, and that the requested op is elaborated (EnableCompute / ComputeOps).

Ops (selected via compute_cfg.compute_op, same programming model as transpose):

Op Direction Ratio Width
COMPUTE_MXQUANT FP32 -> MXFP8 128B -> 33B per 32-elem block up to 1024b (StrbWidth 128)
COMPUTE_MXQUANT_FP16 FP16 -> MXFP8 64B -> 33B up to 512b (StrbWidth 64)
COMPUTE_MXDEQUANT MXFP8 -> FP32 33B -> 128B length must be 33k with k % StrbWidth == 0

MX block layout is inline [1B E8M0 scale][32B E5M2]. The quantizer is RNE with full subnormal, saturation and Inf/NaN handling, bit-exact with the viDMA ALCU reference (the split normal/subnormal rounding bands are load-bearing; do not merge them).

Validation

Byte-exact against a DPI-C golden (test/idma_mxquant_dpi.c, pure integer pipeline) on Questa 2026.1 at every legal StrbWidth (DataWidth 32/64/128/256/512/1024):

  • tb_idma_mxquant: FP16 and FP32 sources, a genuine partial 33B tail beat, a 4K page-crossing write.
  • tb_idma_mxroundtrip: quant -> dequant back-to-back through one backend, byte-exact at both stages (the exact E5M2 identity); FP16 leg at 512b, FP32 leg at 1024b (256 blocks, 32KB legs).
  • tb_idma_mxrand: constrained-random campaign — 40 serialized transfers per run with a random op each (FP16/FP32 quant, dequant, plain unaligned copy), random block counts, writes biased across 4K boundaries, behind an AXI shim injecting 40% random per-channel stalls; canary bytes fence each destination.
  • tb_idma_mxneg: every compute guard assert proven to fire (9 cases, incl. transfer overlap tripping the sub-unit fatal). Note this PR also adds +define+INC_ASSERT to the sim flow — without it ASSERT_NEVER compiles to nothing, so the legalizer guard asserts (including pre-existing ones) were inert in simulation.
  • tb_idma_transpose_b2b / tb_idma_transpose_nd unchanged and green (backward compatibility).
  • tb_idma_mxperf: steady-state throughput vs a plain-copy baseline on ideal memory — the bottleneck channel (quant: R, dequant: W) sustains 100% bus utilization at every StrbWidth, both for single transfers and for pipelined back-to-back streams (aggregate window across 8 transfers including the boundaries); the other channel sits at the compression ratio (e.g. 33/128). Three microarchitectural points make this hold: pack-buffer space only gates block-completing input beats, the dequant pack buffer carries one expanded-block window of refill-during-drain headroom, and lane-exact retire keeps tails sound across transfer boundaries. The pipelined quant stream is also checked byte-exact under the random-stall shim in tb_idma_mxrand.

Known scope / disclosures

  • Compute is functionally validated only on idma_backend_rw_axi. Size-changing compute on non-AXI src/dst is fenced with NOT IMPLEMENTED asserts (ComputeMxSrcProtocol / ComputeMxDstProtocol) until validated; OBI is the natural first candidate (its write ready is already per-beat). Transpose keeps its pre-existing protocol scope.
  • Back-to-back compute transfers with an identical config pipeline freely (verified at 100% utilization). Any config change (op, FP16 flag, params) still requires the engine to be drained first; violations $fatal in the dispatcher (config change while busy) and in the MX sub-units (clear with in-flight state) instead of corrupting silently.
  • Compute requires beat-aligned src/dst (asserted; all guard asserts covered by firing negative tests in tb_idma_mxneg).
  • ComputeOps defaults to '1: existing EnableCompute designs gain the MX engines on regeneration (area); set ComputeOps explicitly to opt out.
  • Synthesized standalone in a 12 nm technology at 1 GHz, timing met: quant 39.5k/71.5k cells (512b/1024b), dequant 37.4k/52.8k. The second commit restructures the engines for synthesis (hoisted completion datapath, generate-gated FP16, circular dequant buffer) — bit-exact and throughput-neutral, re-verified by the full suite. No in-system PnR data yet.
  • Validated at BufferDepth=3, MemSysDepth=0, 1D transfers only.

Copilot AI lite review requested due to automatic review settings August 6, 2026 06:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@DanielKellerM
DanielKellerM force-pushed the otf/size-changing-compute branch 3 times, most recently from 8333342 to fca1db0 Compare August 6, 2026 08:06
…ath)

Support compute ops whose write byte count differs from the read byte count:
per-op source:dest ratios in idma_pkg (compute_in_bytes/compute_out_bytes)
program the legalizer write length, and the compute seam presents per-lane
valid (pack occupancy) with full strobes so the write manager's own beat mask
gates acceptance — a partial beat can only complete the transfer's genuine
tail. The legalizer asserts input-granule alignment, beat-aligned src/dst,
per-beat-capable destination protocols (no TileLink), the FP16 width cap,
and that the requested op is elaborated (EnableCompute / ComputeOps).

Ops (selected via compute_cfg.compute_op like transpose):
- COMPUTE_MXQUANT / COMPUTE_MXQUANT_FP16: FP32/FP16 -> MXFP8, one 33B block
  ([1B E8M0 scale][32B E5M2]) per 32 elements (128B/64B in), RNE with full
  subnormal, saturation and Inf/NaN handling, bit-exact with the viDMA ALCU.
  FP16 requires StrbWidth <= 64; FP32 scales to StrbWidth 128 (1024b).
- COMPUTE_MXDEQUANT: MXFP8 -> FP32 (33B -> 128B); input length must be
  beat-aligned (33k % StrbWidth == 0).

Verified byte-exact against a DPI-C golden at every legal StrbWidth
(DataWidth 32-1024): tb_idma_mxquant (FP16 + FP32 sources, partial tail,
4K page crossing) and tb_idma_mxroundtrip (quant -> dequant, FP16 leg at
512b, FP32 leg at 1024b); transpose regressions unchanged.
@DanielKellerM
DanielKellerM force-pushed the otf/size-changing-compute branch from fca1db0 to aa0bf44 Compare August 6, 2026 09:43
Hoist the block-completion datapath out of the unrolled element loop (64x
replicated at 1024b: 2048 quantizer sites -> 32), generate-gate the FP16
path above 512b, drop output registers that duplicated pack state, reduce
the subnormal shifter to its 3 reachable amounts, and rebuild the dequant
pack as a one-block-per-cycle expansion into a pow2 circular buffer (no
byte shifting). Bit-exact vs the DPI golden and throughput-neutral; the
naive engines did not finish synthesis in 14h, the restructured ones
close 1 GHz timing in a 12 nm technology in ~25 min (quant 39.5k/71.5k
cells at 512b/1024b, dequant 37.4k/52.8k).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants