Skip to content

Add single-dispatch layer-by-layer multi-head attention#91

Draft
andrej wants to merge 4 commits into
amd:develfrom
andrej:mha-lxl-sd
Draft

Add single-dispatch layer-by-layer multi-head attention#91
andrej wants to merge 4 commits into
amd:develfrom
andrej:mha-lxl-sd

Conversation

@andrej

@andrej andrej commented Apr 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

"Naive" alternative implementation for multi-head attention from the currently checked-in data-flow design. This is a simple layer-by-layer implementation, but it uses the single-dispatch mechanism to fuse it all into one MLIR file and save on CPU roundtrips and XRT overheads.

Includes two variants:

  1. "core": Only does the core matmuls and softmax; assumes projected and repeated inputs Q, K, V. This matches the functionality of the checked-in dataflow MHA.
  2. "projected": Performs the Q, K, V projections, applies a RoPE positional embedding and repeats K and V matrices for grouped-query attention. Takes an embedding vector and RoPE angles as input.

andrej
andrej commented Apr 6, 2026
View reviewed changes
Comment thread iron/operators/mha_prefill_lxl/reference.py

@andrej andrej Apr 6, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we reuse the reference from the existing mha? (Note: does not include RoPE and Q, K, V projections, but some code reuse should be possible.)

@github-actions

github-actions Bot commented Apr 7, 2026

Copy link
Copy Markdown
Contributor
📊 Test Results for Test Example Applications

1d87fe8 (2026_04_07_21_05_39)

IRONCLAD

Tested on 2026_04_07_21_05_39 at commit 1d87fe8.

Test Checks TTFT (mean)TPS (mean)
llama_3.2_1b_prompt_1024_tokens_1 ✅ 5/5 2.13 n/a
llama_3.2_1b_prompt_1024_tokens_40 ✅ 5/5 2.18 4.31
llama_3.2_1b_prompt_13_tokens_1 ✅ 5/5 2.09 n/a
llama_3.2_1b_prompt_13_tokens_40 ✅ 5/5 2.09 4.31
📈 Trends (vs main branch) for Test Example Applications

1d87fe8 (2026_04_07_21_05_39)

IRONCLAD Trends

llama_3.2_1b

Commit/Date Num Tokens (max)Num Tokens (mean)Num Tokens (median)Num Tokens (min)Num Tokens (stddev)TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)Total (max)Total (mean)Total (median)Total (min)Total (stddev)
130b6ea — 2025-12-05 21:33:1240.00 (+0.00%)40.00 (+0.00%)40.00 (+0.00%)40.00 (+0.00%)0.00 (n/a)4.71 (-0.42%)4.64 (-0.09%)4.64 (+0.65%)4.55 (-0.22%)0.05 (-17.66%)4.41 (-0.34%)4.39 (-0.19%)4.38 (-0.33%)4.37 (-0.15%)0.01 (-25.90%)12.96 (-0.00%)12.80 (+0.07%)12.80 (-0.23%)12.67 (+0.44%)0.09 (-21.12%)
0a6c11c — 2025-12-03 23:35:1540.00 (n/a)40.00 (n/a)40.00 (n/a)40.00 (n/a)0.00 (n/a)4.73 (n/a)4.64 (n/a)4.61 (n/a)4.56 (n/a)0.06 (n/a)4.42 (n/a)4.40 (n/a)4.40 (n/a)4.37 (n/a)0.02 (n/a)12.96 (n/a)12.79 (n/a)12.83 (n/a)12.62 (n/a)0.12 (n/a)

llama_3.2_1b_prompt_1024_tokens_1

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
1d87fe8 — 2026-04-07 21:00:002.15 (+0.09%)2.13 (+0.08%)2.13 (-0.42%)2.12 (+0.62%)0.01 (-31.21%)
912e6bc — 2026-04-07 19:08:432.15 (n/a)2.13 (n/a)2.13 (n/a)2.11 (n/a)0.02 (n/a)

llama_3.2_1b_prompt_1024_tokens_40

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
1d87fe8 — 2026-04-07 21:00:004.33 (+2.90%)4.31 (+3.44%)4.31 (+3.58%)4.29 (+3.77%)0.01 (-46.93%)2.29 (+0.48%)2.18 (+0.83%)2.15 (+0.80%)2.13 (+0.61%)0.07 (-4.73%)
912e6bc — 2026-04-07 19:08:434.21 (n/a)4.17 (n/a)4.16 (n/a)4.14 (n/a)0.03 (n/a)2.28 (n/a)2.16 (n/a)2.13 (n/a)2.12 (n/a)0.07 (n/a)

llama_3.2_1b_prompt_13_tokens_1

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
1d87fe8 — 2026-04-07 21:00:002.10 (-0.10%)2.09 (+0.11%)2.09 (+0.19%)2.09 (+0.00%)0.01 (+8.87%)
912e6bc — 2026-04-07 19:08:432.10 (n/a)2.09 (n/a)2.09 (n/a)2.09 (n/a)0.01 (n/a)

llama_3.2_1b_prompt_13_tokens_40

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
1d87fe8 — 2026-04-07 21:00:004.36 (+4.23%)4.31 (+3.57%)4.30 (+3.44%)4.29 (+3.23%)0.03 (+128.30%)2.09 (-0.38%)2.09 (-0.04%)2.09 (+0.00%)2.08 (+0.44%)0.01 (-34.93%)
912e6bc — 2026-04-07 19:08:434.18 (n/a)4.16 (n/a)4.16 (n/a)4.15 (n/a)0.01 (n/a)2.10 (n/a)2.09 (n/a)2.09 (n/a)2.07 (n/a)0.01 (n/a)

llama_3.2_1b_prompt_2048_tokens_1

Commit/Date Num_Tokens (max)Num_Tokens (mean)Num_Tokens (median)Num_Tokens (min)Num_Tokens (stddev)TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
897d04e — 2026-03-06 22:56:071.00 (+0.00%)1.00 (+0.00%)1.00 (+0.00%)1.00 (+0.00%)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)2.68 (-1.06%)2.68 (-1.06%)2.68 (-1.06%)2.68 (-1.06%)0.00 (n/a)
84d3478 — 2026-02-17 23:16:231.00 (n/a)1.00 (n/a)1.00 (n/a)1.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)2.70 (n/a)2.70 (n/a)2.70 (n/a)2.70 (n/a)0.00 (n/a)

llama_3.2_1b_prompt_2048_tokens_40

Commit/Date Num_Tokens (max)Num_Tokens (mean)Num_Tokens (median)Num_Tokens (min)Num_Tokens (stddev)TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
897d04e — 2026-03-06 22:56:0740.00 (+0.00%)40.00 (+0.00%)40.00 (+0.00%)40.00 (+0.00%)0.00 (n/a)4.00 (-1.72%)4.00 (-1.72%)4.00 (-1.72%)4.00 (-1.72%)0.00 (n/a)2.70 (-0.44%)2.70 (-0.44%)2.70 (-0.44%)2.70 (-0.44%)0.00 (n/a)
84d3478 — 2026-02-17 23:16:2340.00 (n/a)40.00 (n/a)40.00 (n/a)40.00 (n/a)0.00 (n/a)4.07 (n/a)4.07 (n/a)4.07 (n/a)4.07 (n/a)0.00 (n/a)2.71 (n/a)2.71 (n/a)2.71 (n/a)2.71 (n/a)0.00 (n/a)

@github-actions

github-actions Bot commented Apr 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

CI Test Results

2896c25 (2026_07_13_21_24_51)

IRON - CI Summary

Examples

iron/applications/llama_3.2_1b
Test Krackan Status Krackan Phoenix Status Phoenix
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40] ✅ - - -

Small

iron/operators/axpy
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0] ✅ 185.94 ✅ 427.10
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0] ✅ 200.84 ✅ 438.98
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0] ✅ 204.36 ✅ 497.62
test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0] ✅ 233.76 - -
iron/operators/dequant
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32] ✅ 156.00 ✅ 344.28
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32] ✅ 189.82 ✅ 554.86
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32] ✅ 184.04 ✅ 364.24
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32] ✅ 168.00 ✅ 501.62
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32] ✅ 163.30 ✅ 649.28
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32] ✅ 162.30 ✅ 461.56
test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32] ✅ 179.88 - -
test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32] ✅ 216.96 - -
iron/operators/elementwise_add
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048] ✅ 158.32 ✅ 390.62
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024] ✅ 140.34 ✅ 635.02
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512] ✅ 176.88 ✅ 483.24
test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256] ✅ 214.36 - -
iron/operators/elementwise_mul
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048] ✅ 180.42 ✅ 287.52
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024] ✅ 170.98 ✅ 316.60
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512] ✅ 180.24 ✅ 291.82
test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256] ✅ 193.06 - -
iron/operators/gelu
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 134.60 ✅ 301.14
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 183.68 ✅ 381.06
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 166.02 ✅ 297.34
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 205.54 ✅ 536.48
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 166.26 ✅ 355.40
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 185.58 ✅ 411.54
test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 192.14 - -
test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 201.04 - -
iron/operators/gemm
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1] ✅ 2295.46 - -
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1] ✅ 231.28 ✅ 476.44
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1] ✅ 268.44 ✅ 841.86
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] ✅ 48681.32 ✅ 82322.40
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] ✅ 28702.54 ✅ 24658.46
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1] ✅ 7706.82 - -
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1] ✅ 2334.16 ✅ 3554.04
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4] ✅ 3802.34 ✅ 7028.10
test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1] ✅ 1508.22 - -
iron/operators/gemv
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128] ✅ 0.20 ✅ 0.11
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048] ✅ 12.58 ✅ 3.55
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024] ✅ 24.19 ✅ 6.51
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512] ✅ 39.47 ✅ 9.16
test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256] ✅ 42.86 - -
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024] ✅ 12.45 ✅ 3.72
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024] ✅ 24.11 ✅ 6.42
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024] ✅ 38.75 ✅ 10.50
test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024] ✅ 42.19 - -
iron/operators/layer_norm
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] ✅ 171.14 ✅ 501.50
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] ✅ 173.74 ✅ 361.02
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] ✅ 163.56 ✅ 470.84
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] ✅ 175.20 ✅ 397.32
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] ✅ 172.88 ✅ 502.94
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] ✅ 195.38 ✅ 556.68
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256] ✅ 175.38 - -
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128] ✅ 237.76 - -
iron/operators/leaky_relu
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01] ✅ 171.64 ✅ 258.00
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1] ✅ 174.90 ✅ 360.26
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25] ✅ 167.50 ✅ 352.96
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01] ✅ 176.88 ✅ 717.84
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01] ✅ 164.58 ✅ 382.52
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01] ✅ 182.96 ✅ 377.78
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01] ✅ 189.22 ✅ 308.04
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01] ✅ 198.46 ✅ 585.78
test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-alpha_0.01] ✅ 212.86 - -
test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-alpha_0.01] ✅ 220.20 - -
iron/operators/mem_copy
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048] ✅ 156.72 ✅ 251.20
test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128] ✅ 290.40 - -
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024] ✅ 175.90 ✅ 772.64
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024] ✅ 205.72 ✅ 361.96
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512] ✅ 196.32 ✅ 434.62
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512] ✅ 193.98 ✅ 338.70
test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256] ✅ 195.00 - -
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256] ✅ 184.66 ✅ 743.72
iron/operators/mha_prefill_df
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0] ✅ 40783.46 - -
iron/operators/mha_prefill_lxl
Test Krackan Status Krackan Latency (mean) Phoenix Status Phoenix Latency (mean)
test_mha_prefill_lxl[GPT2-Small-256seq] ✅ 28117.44 ❌ 28983.98
test_mha_prefill_lxl[H2] ✅ 24903.74 ❌ 35530.60
test_mha_prefill_lxl[Llama3.2-256seq] ✅ 31446.42 ❌ 44873.64
Krackan - Small

IRON

Tested on 2026_07_13_21_24_51 at commit 2896c25.

iron/operators/axpy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]✅ 5/5185.940.07n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]✅ 5/5200.840.06n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]✅ 5/5204.360.06n/a
test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0]✅ 5/5233.760.05n/a
iron/operators/dequant
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]✅ 5/5156.000.03n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]✅ 5/5189.820.03n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]✅ 5/5184.040.03n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]✅ 5/5168.000.03n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]✅ 5/5163.300.04n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]✅ 5/5162.300.03n/a
test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32]✅ 5/5179.880.03n/a
test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32]✅ 5/5216.960.02n/a
iron/operators/elementwise_add
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5158.320.08n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5140.340.09n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5176.880.07n/a
test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256]✅ 5/5214.360.06n/a
iron/operators/elementwise_mul
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5180.420.07n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5170.980.07n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5180.240.07n/a
test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256]✅ 5/5193.060.07n/a
iron/operators/gelu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5134.600.06n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5183.680.05n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5166.020.05n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5205.540.04n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5166.260.05n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5185.580.04n/a
test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5192.140.04n/a
test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5201.040.04n/a
iron/operators/gemm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1]✅ 5/52295.464.131623.88
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5231.281.0042.65
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5268.440.8536.15
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/548681.320.52352.91
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/528702.540.88598.59
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/57706.823.272229.65
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]✅ 5/52334.163.48912.17
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]✅ 5/53802.340.3317.86
test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1]✅ 5/51508.224.481385.23
iron/operators/gemv
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]✅ 5/5n/a0.200.19
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]✅ 5/5n/a12.5812.58
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]✅ 5/5n/a24.1924.17
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]✅ 5/5n/a39.4739.45
test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256]✅ 5/5n/a42.8642.84
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a12.4512.45
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a24.1124.09
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a38.7538.73
test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a42.1942.16
iron/operators/layer_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5171.140.05n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5173.740.05n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5163.560.05n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5175.200.05n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5172.880.05n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5195.380.04n/a
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]✅ 5/5175.380.05n/a
test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]✅ 5/5237.760.04n/a
iron/operators/leaky_relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]✅ 5/5171.640.05n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]✅ 5/5174.900.05n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]✅ 5/5167.500.05n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]✅ 5/5176.880.05n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]✅ 5/5164.580.05n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]✅ 5/5182.960.05n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]✅ 5/5189.220.04n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]✅ 5/5198.460.04n/a
test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-alpha_0.01]✅ 5/5212.860.04n/a
test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-alpha_0.01]✅ 5/5220.200.04n/a
iron/operators/mem_copy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]✅ 5/5156.720.05n/a
test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128]✅ 5/5290.400.03n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]✅ 5/5175.900.05n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]✅ 5/5205.720.04n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]✅ 5/5196.320.04n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]✅ 5/5193.980.04n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256]✅ 5/5195.000.04n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]✅ 5/5184.660.05n/a
iron/operators/mha_prefill_df
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]✅ 5/540783.460.21n/a
iron/operators/mha_prefill_lxl
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mha_prefill_lxl[GPT2-Small-256seq]✅ 5/528117.44n/a63.93
test_mha_prefill_lxl[H2]✅ 5/524903.74n/a16.43
test_mha_prefill_lxl[Llama3.2-256seq]✅ 5/531446.42n/a103.28

Trends:

IRON Trends

iron/operators/axpy

test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.08 (-41.11%)0.07 (-24.11%)0.06 (-18.27%)0.06 (-8.14%)0.01 (-75.03%)208.70 (+8.87%)185.94 (+26.06%)189.20 (+22.38%)163.80 (+69.74%)16.52 (-51.94%)
1f9e5a9 — 2026-07-13 18:35:330.13 (n/a)0.09 (n/a)0.08 (n/a)0.06 (n/a)0.02 (n/a)191.70 (n/a)147.50 (n/a)154.60 (n/a)96.50 (n/a)34.37 (n/a)

test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.07 (-39.36%)0.06 (-26.20%)0.06 (-30.36%)0.06 (+18.12%)0.00 (-84.11%)220.00 (-15.35%)200.84 (+24.73%)195.60 (+43.61%)187.70 (+64.94%)12.83 (-78.29%)
1f9e5a9 — 2026-07-13 18:35:330.11 (n/a)0.08 (n/a)0.09 (n/a)0.05 (n/a)0.02 (n/a)259.90 (n/a)161.02 (n/a)136.20 (n/a)113.80 (n/a)59.11 (n/a)

test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.08 (-17.39%)0.06 (-17.58%)0.07 (-16.33%)0.04 (+13.01%)0.01 (-34.17%)280.80 (-11.50%)204.36 (+14.41%)183.90 (+19.49%)151.70 (+21.07%)52.29 (-33.47%)
1f9e5a9 — 2026-07-13 18:35:330.10 (n/a)0.08 (n/a)0.08 (n/a)0.04 (n/a)0.02 (n/a)317.30 (n/a)178.62 (n/a)153.90 (n/a)125.30 (n/a)78.60 (n/a)

test_axpy[input_length_2048-num_aie_columns_8-tile_size_256-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (-30.21%)0.05 (-31.31%)0.06 (-29.86%)0.04 (-38.10%)0.01 (-5.32%)306.30 (+61.55%)233.76 (+47.48%)223.00 (+42.58%)193.60 (+43.30%)43.79 (+121.27%)
1f9e5a9 — 2026-07-13 18:35:330.09 (n/a)0.08 (n/a)0.08 (n/a)0.06 (n/a)0.01 (n/a)189.60 (n/a)158.50 (n/a)156.40 (n/a)135.10 (n/a)19.79 (n/a)
iron/operators/dequant

test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (-18.54%)0.03 (+2.34%)0.04 (+21.45%)0.03 (+11.04%)0.01 (-31.05%)191.30 (-9.93%)156.00 (-4.36%)139.00 (-17.65%)130.10 (+22.74%)30.70 (-19.19%)
1f9e5a9 — 2026-07-13 18:35:330.05 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)212.40 (n/a)163.12 (n/a)168.80 (n/a)106.00 (n/a)37.99 (n/a)

test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (-8.57%)0.03 (-18.71%)0.03 (-32.60%)0.02 (-20.24%)0.01 (-5.16%)241.30 (+25.35%)189.82 (+23.84%)202.30 (+48.31%)139.50 (+9.41%)40.72 (+28.84%)
1f9e5a9 — 2026-07-13 18:35:330.04 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)192.50 (n/a)153.28 (n/a)136.40 (n/a)127.50 (n/a)31.60 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (-6.00%)0.03 (+11.78%)0.03 (+7.74%)0.02 (+23.07%)0.01 (-30.86%)236.70 (-18.74%)184.04 (-14.90%)191.10 (-7.19%)136.80 (+6.38%)38.56 (-41.90%)
1f9e5a9 — 2026-07-13 18:35:330.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)291.30 (n/a)216.26 (n/a)205.90 (n/a)128.60 (n/a)66.37 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (-2.58%)0.03 (+9.24%)0.04 (+33.30%)0.02 (-19.46%)0.01 (+24.84%)258.00 (+24.16%)168.00 (-5.87%)146.40 (-25.00%)136.80 (+2.63%)51.10 (+63.03%)
1f9e5a9 — 2026-07-13 18:35:330.04 (n/a)0.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)207.80 (n/a)178.48 (n/a)195.20 (n/a)133.30 (n/a)31.34 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (+33.14%)0.04 (+22.25%)0.04 (+35.87%)0.02 (-8.42%)0.01 (+143.93%)253.00 (+9.19%)163.30 (-12.41%)130.90 (-26.38%)118.80 (-24.86%)58.30 (+95.05%)
1f9e5a9 — 2026-07-13 18:35:330.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.00 (n/a)231.70 (n/a)186.44 (n/a)177.80 (n/a)158.10 (n/a)29.89 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (+10.79%)0.03 (+18.00%)0.04 (+36.46%)0.03 (+1.44%)0.01 (+51.64%)207.10 (-1.43%)162.30 (-13.10%)140.10 (-26.73%)125.50 (-9.71%)40.24 (+43.87%)
1f9e5a9 — 2026-07-13 18:35:330.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)210.10 (n/a)186.76 (n/a)191.20 (n/a)139.00 (n/a)27.97 (n/a)

test_dequant[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.03 (-21.34%)0.03 (-3.70%)0.03 (+6.94%)0.03 (+6.41%)0.00 (-60.69%)201.90 (-6.05%)179.88 (+0.94%)174.30 (-6.49%)164.80 (+27.16%)16.41 (-53.39%)
1f9e5a9 — 2026-07-13 18:35:330.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)214.90 (n/a)178.20 (n/a)186.40 (n/a)129.60 (n/a)35.21 (n/a)

test_dequant[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.03 (-2.79%)0.02 (+16.98%)0.02 (+2.66%)0.02 (+54.77%)0.00 (-43.64%)254.00 (-35.39%)216.96 (-20.48%)218.00 (-2.55%)183.30 (+2.86%)33.38 (-64.54%)
1f9e5a9 — 2026-07-13 18:35:330.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)393.10 (n/a)272.84 (n/a)223.70 (n/a)178.20 (n/a)94.13 (n/a)
iron/operators/elementwise_add

test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.10 (n/a)0.08 (n/a)0.07 (n/a)0.07 (n/a)0.01 (n/a)183.10 (n/a)158.32 (n/a)170.40 (n/a)122.40 (n/a)26.24 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.11 (n/a)0.09 (n/a)0.09 (n/a)0.08 (n/a)0.01 (n/a)158.20 (n/a)140.34 (n/a)138.80 (n/a)115.50 (n/a)16.68 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.07 (n/a)0.07 (n/a)0.07 (n/a)0.06 (n/a)0.00 (n/a)195.80 (n/a)176.88 (n/a)173.50 (n/a)166.70 (n/a)11.15 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.09 (n/a)0.06 (n/a)0.06 (n/a)0.04 (n/a)0.02 (n/a)340.70 (n/a)214.36 (n/a)198.10 (n/a)140.60 (n/a)76.28 (n/a)
iron/operators/elementwise_mul

test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.09 (n/a)0.07 (n/a)0.08 (n/a)0.05 (n/a)0.02 (n/a)252.30 (n/a)180.42 (n/a)161.60 (n/a)133.70 (n/a)46.79 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.09 (n/a)0.07 (n/a)0.08 (n/a)0.06 (n/a)0.01 (n/a)213.70 (n/a)170.98 (n/a)157.00 (n/a)139.40 (n/a)32.68 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.08 (n/a)0.07 (n/a)0.08 (n/a)0.05 (n/a)0.01 (n/a)256.20 (n/a)180.24 (n/a)161.40 (n/a)149.10 (n/a)43.35 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_8-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.08 (n/a)0.07 (n/a)0.07 (n/a)0.05 (n/a)0.01 (n/a)266.80 (n/a)193.06 (n/a)179.00 (n/a)158.50 (n/a)44.52 (n/a)
iron/operators/gelu

test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.08 (n/a)0.06 (n/a)0.06 (n/a)0.04 (n/a)0.02 (n/a)183.40 (n/a)134.60 (n/a)133.00 (n/a)97.00 (n/a)33.79 (n/a)

test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)251.50 (n/a)183.68 (n/a)155.90 (n/a)144.30 (n/a)49.39 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.07 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)216.90 (n/a)166.02 (n/a)163.90 (n/a)116.70 (n/a)38.22 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (n/a)0.04 (n/a)0.05 (n/a)0.02 (n/a)0.01 (n/a)361.00 (n/a)205.54 (n/a)162.40 (n/a)139.60 (n/a)89.87 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)196.00 (n/a)166.26 (n/a)162.90 (n/a)130.20 (n/a)24.59 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)203.40 (n/a)185.58 (n/a)194.30 (n/a)162.10 (n/a)18.29 (n/a)

test_gelu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.05 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)245.80 (n/a)192.14 (n/a)187.00 (n/a)168.20 (n/a)31.56 (n/a)

test_gelu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)223.50 (n/a)201.04 (n/a)201.00 (n/a)183.20 (n/a)15.70 (n/a)
iron/operators/gemm

test_gemm[M_1792-K_896-N_1152-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_64-k_32-n_48-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:434.61 (+1.54%)4.13 (+1.22%)4.16 (+2.03%)3.52 (+2.24%)0.39 (-5.67%)2672.80 (-2.20%)2295.46 (-1.32%)2258.80 (-1.99%)2041.40 (-1.52%)230.56 (-8.51%)1812.16 (+1.54%)1623.88 (+1.22%)1637.75 (+2.03%)1384.09 (+2.24%)153.43 (-5.67%)
1f9e5a9 — 2026-07-13 18:35:334.54 (n/a)4.08 (n/a)4.08 (n/a)3.44 (n/a)0.41 (n/a)2732.80 (n/a)2326.16 (n/a)2304.70 (n/a)2072.90 (n/a)252.00 (n/a)1784.65 (n/a)1604.38 (n/a)1605.12 (n/a)1353.71 (n/a)162.66 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:431.25 (-2.81%)1.00 (-7.77%)1.05 (-4.36%)0.67 (-26.80%)0.21 (+27.80%)332.00 (+36.57%)231.28 (+11.15%)211.40 (+4.55%)177.60 (+2.84%)59.54 (+83.95%)53.12 (-2.81%)42.65 (-7.77%)44.64 (-4.36%)28.42 (-26.80%)9.13 (+27.80%)
1f9e5a9 — 2026-07-13 18:35:331.28 (n/a)1.08 (n/a)1.09 (n/a)0.91 (n/a)0.17 (n/a)243.10 (n/a)208.08 (n/a)202.20 (n/a)172.70 (n/a)32.37 (n/a)54.66 (n/a)46.24 (n/a)46.68 (n/a)38.83 (n/a)7.14 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:431.03 (-11.76%)0.85 (-5.60%)0.77 (+4.43%)0.68 (-5.80%)0.16 (-32.52%)327.00 (+6.13%)268.44 (+3.37%)288.10 (-4.25%)215.60 (+13.35%)49.11 (-21.64%)43.78 (-11.76%)36.15 (-5.60%)32.75 (+4.43%)28.86 (-5.80%)6.83 (-32.52%)
1f9e5a9 — 2026-07-13 18:35:331.16 (n/a)0.90 (n/a)0.74 (n/a)0.72 (n/a)0.24 (n/a)308.10 (n/a)259.68 (n/a)300.90 (n/a)190.20 (n/a)62.67 (n/a)49.62 (n/a)38.30 (n/a)31.36 (n/a)30.63 (n/a)10.13 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:430.52 (-0.12%)0.52 (-0.01%)0.52 (+0.05%)0.51 (-0.11%)0.00 (+5.53%)48911.50 (+0.11%)48681.32 (+0.01%)48633.20 (-0.05%)48605.80 (+0.12%)129.86 (+5.82%)353.45 (-0.12%)352.91 (-0.01%)353.25 (+0.05%)351.24 (-0.11%)0.94 (+5.53%)
1f9e5a9 — 2026-07-13 18:35:330.52 (n/a)0.52 (n/a)0.52 (n/a)0.52 (n/a)0.00 (n/a)48855.70 (n/a)48678.68 (n/a)48655.50 (n/a)48546.90 (n/a)122.72 (n/a)353.88 (n/a)352.93 (n/a)353.09 (n/a)351.64 (n/a)0.89 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:430.89 (-0.36%)0.88 (-0.79%)0.87 (-0.93%)0.87 (-0.92%)0.01 (+29.64%)28954.00 (+0.92%)28702.54 (+0.80%)28783.30 (+0.94%)28289.60 (+0.36%)274.81 (+31.32%)607.28 (-0.36%)598.59 (-0.79%)596.87 (-0.93%)593.35 (-0.92%)5.76 (+29.64%)
1f9e5a9 — 2026-07-13 18:35:330.89 (n/a)0.88 (n/a)0.88 (n/a)0.88 (n/a)0.01 (n/a)28688.90 (n/a)28476.06 (n/a)28516.40 (n/a)28188.10 (n/a)209.28 (n/a)609.47 (n/a)603.34 (n/a)602.46 (n/a)598.83 (n/a)4.44 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:433.34 (+2.68%)3.27 (+3.31%)3.28 (+4.18%)3.21 (+3.65%)0.05 (-5.22%)7839.40 (-3.52%)7706.82 (-3.21%)7679.40 (-4.01%)7541.40 (-2.61%)125.03 (-10.45%)2278.08 (+2.68%)2229.65 (+3.31%)2237.12 (+4.18%)2191.49 (+3.65%)36.24 (-5.22%)
1f9e5a9 — 2026-07-13 18:35:333.25 (n/a)3.16 (n/a)3.15 (n/a)3.10 (n/a)0.06 (n/a)8125.50 (n/a)7962.46 (n/a)8000.50 (n/a)7743.40 (n/a)139.62 (n/a)2218.65 (n/a)2158.14 (n/a)2147.35 (n/a)2114.31 (n/a)38.23 (n/a)

test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:433.77 (-4.33%)3.48 (-1.45%)3.52 (-1.48%)2.95 (+2.17%)0.31 (-21.61%)2730.90 (-2.12%)2334.16 (+1.05%)2290.60 (+1.50%)2139.30 (+4.52%)231.71 (-19.69%)988.12 (-4.33%)912.17 (-1.45%)922.86 (-1.48%)774.09 (+2.17%)82.35 (-21.61%)
1f9e5a9 — 2026-07-13 18:35:333.94 (n/a)3.53 (n/a)3.57 (n/a)2.89 (n/a)0.40 (n/a)2790.00 (n/a)2309.92 (n/a)2256.70 (n/a)2046.70 (n/a)288.52 (n/a)1032.84 (n/a)925.60 (n/a)936.73 (n/a)757.68 (n/a)105.05 (n/a)

test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:430.38 (-24.03%)0.33 (-12.54%)0.31 (-16.40%)0.29 (-5.37%)0.04 (-42.56%)4258.80 (+5.67%)3802.34 (+12.84%)4028.70 (+19.62%)3282.70 (+31.63%)456.56 (-18.64%)20.44 (-24.03%)17.86 (-12.54%)16.66 (-16.40%)15.76 (-5.37%)2.22 (-42.56%)
1f9e5a9 — 2026-07-13 18:35:330.50 (n/a)0.38 (n/a)0.37 (n/a)0.31 (n/a)0.07 (n/a)4030.20 (n/a)3369.80 (n/a)3368.00 (n/a)2493.90 (n/a)561.13 (n/a)26.91 (n/a)20.42 (n/a)19.93 (n/a)16.65 (n/a)3.87 (n/a)

test_gemm[M_896-K_1792-N_640-num_aie_columns_8-b_col_maj_False-c_col_maj_True-m_32-k_64-n_80-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:435.19 (-2.95%)4.48 (-10.36%)4.79 (-4.21%)3.80 (-18.46%)0.63 (+129.09%)1749.90 (+22.64%)1508.22 (+13.13%)1388.10 (+4.39%)1282.20 (+3.05%)218.81 (+197.98%)1602.93 (-2.95%)1385.23 (-10.36%)1480.57 (-4.21%)1174.45 (-18.46%)194.64 (+129.09%)
1f9e5a9 — 2026-07-13 18:35:335.35 (n/a)5.00 (n/a)5.00 (n/a)4.66 (n/a)0.27 (n/a)1426.90 (n/a)1333.12 (n/a)1329.70 (n/a)1244.30 (n/a)73.43 (n/a)1651.64 (n/a)1545.37 (n/a)1545.60 (n/a)1440.31 (n/a)84.96 (n/a)
iron/operators/gemv

test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:430.24 (+13.81%)0.20 (+9.89%)0.19 (+10.98%)0.13 (-14.19%)0.05 (+94.33%)0.24 (+13.81%)0.19 (+9.89%)0.19 (+10.98%)0.13 (-14.19%)0.04 (+94.34%)
1f9e5a9 — 2026-07-13 18:35:330.21 (n/a)0.18 (n/a)0.17 (n/a)0.15 (n/a)0.02 (n/a)0.21 (n/a)0.18 (n/a)0.17 (n/a)0.15 (n/a)0.02 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:4313.50 (+1.35%)12.58 (+6.33%)12.16 (+2.13%)11.71 (+15.35%)0.82 (-27.43%)13.49 (+1.35%)12.58 (+6.33%)12.16 (+2.13%)11.71 (+15.35%)0.82 (-27.43%)
1f9e5a9 — 2026-07-13 18:35:3313.32 (n/a)11.83 (n/a)11.91 (n/a)10.16 (n/a)1.13 (n/a)13.31 (n/a)11.83 (n/a)11.90 (n/a)10.15 (n/a)1.13 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:4325.05 (-0.00%)24.19 (+7.07%)24.44 (+5.45%)23.29 (+31.66%)0.79 (-72.24%)25.04 (-0.00%)24.17 (+7.07%)24.43 (+5.45%)23.27 (+31.66%)0.79 (-72.24%)
1f9e5a9 — 2026-07-13 18:35:3325.05 (n/a)22.59 (n/a)23.18 (n/a)17.69 (n/a)2.85 (n/a)25.04 (n/a)22.58 (n/a)23.16 (n/a)17.68 (n/a)2.85 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:4340.53 (-6.02%)39.47 (-0.64%)40.21 (+1.94%)37.20 (-1.50%)1.43 (-32.74%)40.50 (-6.02%)39.45 (-0.64%)40.19 (+1.94%)37.17 (-1.50%)1.42 (-32.74%)
1f9e5a9 — 2026-07-13 18:35:3343.12 (n/a)39.72 (n/a)39.45 (n/a)37.76 (n/a)2.12 (n/a)43.09 (n/a)39.70 (n/a)39.42 (n/a)37.74 (n/a)2.12 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_8-tile_size_input_1-tile_size_output_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:4345.78 (+1.88%)42.86 (+0.12%)43.15 (+0.62%)39.40 (-4.11%)2.28 (+41.54%)45.75 (+1.88%)42.84 (+0.12%)43.12 (+0.62%)39.37 (-4.11%)2.28 (+41.54%)
1f9e5a9 — 2026-07-13 18:35:3344.94 (n/a)42.82 (n/a)42.88 (n/a)41.09 (n/a)1.61 (n/a)44.91 (n/a)42.79 (n/a)42.85 (n/a)41.06 (n/a)1.61 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:4313.52 (+0.30%)12.45 (-2.47%)12.52 (-2.90%)10.94 (-3.88%)1.05 (+23.22%)13.51 (+0.30%)12.45 (-2.47%)12.52 (-2.90%)10.93 (-3.88%)1.05 (+23.22%)
1f9e5a9 — 2026-07-13 18:35:3313.48 (n/a)12.77 (n/a)12.90 (n/a)11.38 (n/a)0.85 (n/a)13.47 (n/a)12.76 (n/a)12.89 (n/a)11.37 (n/a)0.85 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:4324.63 (-1.40%)24.11 (+1.03%)24.14 (-0.93%)23.49 (+11.24%)0.42 (-73.35%)24.61 (-1.40%)24.09 (+1.03%)24.13 (-0.93%)23.48 (+11.24%)0.42 (-73.35%)
1f9e5a9 — 2026-07-13 18:35:3324.98 (n/a)23.86 (n/a)24.37 (n/a)21.12 (n/a)1.58 (n/a)24.96 (n/a)23.85 (n/a)24.35 (n/a)21.10 (n/a)1.58 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:4339.85 (-1.42%)38.75 (-0.82%)38.50 (-1.32%)38.39 (+4.32%)0.62 (-58.18%)39.82 (-1.42%)38.73 (-0.82%)38.48 (-1.32%)38.37 (+4.32%)0.62 (-58.18%)
1f9e5a9 — 2026-07-13 18:35:3340.42 (n/a)39.07 (n/a)39.01 (n/a)36.80 (n/a)1.48 (n/a)40.40 (n/a)39.05 (n/a)38.99 (n/a)36.78 (n/a)1.48 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_8-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:4345.39 (+0.85%)42.19 (-2.54%)43.26 (-1.29%)36.76 (-11.22%)3.44 (+138.20%)45.37 (+0.85%)42.16 (-2.54%)43.23 (-1.29%)36.74 (-11.22%)3.44 (+138.20%)
1f9e5a9 — 2026-07-13 18:35:3345.01 (n/a)43.29 (n/a)43.82 (n/a)41.41 (n/a)1.44 (n/a)44.98 (n/a)43.26 (n/a)43.80 (n/a)41.38 (n/a)1.44 (n/a)
iron/operators/layer_norm

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.07 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)207.70 (n/a)171.14 (n/a)166.80 (n/a)113.50 (n/a)37.86 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.05 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.00 (n/a)203.70 (n/a)173.74 (n/a)167.60 (n/a)157.40 (n/a)18.11 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.05 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.00 (n/a)167.00 (n/a)163.56 (n/a)165.70 (n/a)156.50 (n/a)4.30 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)212.60 (n/a)175.20 (n/a)169.60 (n/a)134.30 (n/a)30.92 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)205.70 (n/a)172.88 (n/a)178.00 (n/a)138.80 (n/a)32.36 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)221.10 (n/a)195.38 (n/a)202.40 (n/a)168.20 (n/a)22.30 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)208.40 (n/a)175.38 (n/a)178.70 (n/a)138.80 (n/a)27.25 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)341.50 (n/a)237.76 (n/a)219.50 (n/a)188.30 (n/a)59.58 (n/a)
iron/operators/leaky_relu

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (+5.80%)0.05 (+10.88%)0.05 (+16.53%)0.04 (+7.34%)0.01 (-7.97%)214.90 (-6.85%)171.64 (-10.49%)167.60 (-14.18%)138.00 (-5.48%)28.96 (-18.96%)
1f9e5a9 — 2026-07-13 18:35:330.06 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)230.70 (n/a)191.76 (n/a)195.30 (n/a)146.00 (n/a)35.73 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (-13.49%)0.05 (-6.82%)0.05 (-4.02%)0.04 (-8.09%)0.01 (-30.28%)212.80 (+8.79%)174.90 (+6.05%)175.50 (+4.22%)144.70 (+15.58%)26.52 (-14.27%)
1f9e5a9 — 2026-07-13 18:35:330.07 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)195.60 (n/a)164.92 (n/a)168.40 (n/a)125.20 (n/a)30.93 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (+26.07%)0.05 (+4.35%)0.05 (+4.37%)0.04 (-9.17%)0.01 (+252.26%)206.10 (+10.10%)167.50 (-2.05%)159.90 (-4.14%)129.90 (-20.70%)28.82 (+205.47%)
1f9e5a9 — 2026-07-13 18:35:330.05 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.00 (n/a)187.20 (n/a)171.00 (n/a)166.80 (n/a)163.80 (n/a)9.43 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (+15.38%)0.05 (+5.26%)0.04 (-0.16%)0.04 (-11.70%)0.01 (+115.57%)219.20 (+13.28%)176.88 (-1.55%)186.60 (+0.16%)127.30 (-13.34%)40.53 (+115.87%)
1f9e5a9 — 2026-07-13 18:35:330.06 (n/a)0.05 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)193.50 (n/a)179.66 (n/a)186.30 (n/a)146.90 (n/a)18.78 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (+12.07%)0.05 (+9.80%)0.06 (+20.00%)0.04 (-7.06%)0.01 (+91.24%)210.70 (+7.61%)164.58 (-7.22%)147.70 (-16.65%)140.50 (-10.79%)31.21 (+78.96%)
1f9e5a9 — 2026-07-13 18:35:330.05 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.00 (n/a)195.80 (n/a)177.38 (n/a)177.20 (n/a)157.50 (n/a)17.44 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (+17.43%)0.05 (+4.10%)0.05 (+9.92%)0.04 (-0.93%)0.01 (+80.33%)221.70 (+0.96%)182.96 (-2.07%)170.70 (-9.01%)139.90 (-14.80%)34.60 (+59.71%)
1f9e5a9 — 2026-07-13 18:35:330.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)219.60 (n/a)186.82 (n/a)187.60 (n/a)164.20 (n/a)21.67 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.05 (-20.09%)0.04 (-16.71%)0.05 (-6.40%)0.04 (-13.70%)0.01 (-25.76%)222.70 (+15.87%)189.22 (+19.67%)172.10 (+6.83%)168.80 (+25.13%)25.44 (+8.80%)
1f9e5a9 — 2026-07-13 18:35:330.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)192.20 (n/a)158.12 (n/a)161.10 (n/a)134.90 (n/a)23.38 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (-5.99%)0.04 (+1.30%)0.04 (+1.18%)0.04 (+6.35%)0.00 (-52.05%)214.90 (-5.95%)198.46 (-2.06%)198.90 (-1.14%)184.20 (+6.41%)11.04 (-52.37%)
1f9e5a9 — 2026-07-13 18:35:330.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)228.50 (n/a)202.64 (n/a)201.20 (n/a)173.10 (n/a)23.17 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (-35.56%)0.04 (-19.56%)0.04 (-17.45%)0.04 (-3.19%)0.00 (-76.04%)230.00 (+3.28%)212.86 (+20.74%)214.60 (+21.17%)198.70 (+55.11%)13.27 (-61.14%)
1f9e5a9 — 2026-07-13 18:35:330.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)222.70 (n/a)176.30 (n/a)177.10 (n/a)128.10 (n/a)34.15 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (-8.44%)0.04 (-6.79%)0.04 (-3.73%)0.03 (-2.05%)0.00 (-35.71%)236.00 (+2.08%)220.20 (+6.51%)222.50 (+3.87%)190.00 (+9.26%)18.34 (-29.54%)
1f9e5a9 — 2026-07-13 18:35:330.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)231.20 (n/a)206.74 (n/a)214.20 (n/a)173.90 (n/a)26.03 (n/a)
iron/operators/mem_copy

test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (-3.90%)0.05 (+4.43%)0.05 (+4.44%)0.05 (+20.79%)0.01 (-30.98%)174.50 (-17.18%)156.72 (-5.58%)160.30 (-4.24%)135.30 (+4.08%)17.44 (-40.72%)
1f9e5a9 — 2026-07-13 18:35:330.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)210.70 (n/a)165.98 (n/a)167.40 (n/a)130.00 (n/a)29.43 (n/a)

test_mem_copy[input_length_2048-num_cores_16-num_channels_2-bypass_False-tile_size_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.04 (-25.95%)0.03 (-28.04%)0.03 (-38.20%)0.02 (-25.51%)0.01 (-31.28%)421.40 (+34.25%)290.40 (+36.96%)301.60 (+61.80%)190.70 (+35.06%)89.02 (+22.35%)
1f9e5a9 — 2026-07-13 18:35:330.06 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)313.90 (n/a)212.04 (n/a)186.40 (n/a)141.20 (n/a)72.76 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (-10.07%)0.05 (+0.61%)0.05 (-0.72%)0.04 (+1.24%)0.01 (-29.72%)227.20 (-1.22%)175.90 (-2.42%)171.70 (+0.70%)141.30 (+11.17%)31.59 (-21.10%)
1f9e5a9 — 2026-07-13 18:35:330.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)230.00 (n/a)180.26 (n/a)170.50 (n/a)127.10 (n/a)40.03 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.05 (-18.04%)0.04 (-9.27%)0.05 (-1.71%)0.02 (-19.65%)0.01 (-10.58%)334.40 (+24.45%)205.72 (+12.05%)181.30 (+1.74%)150.70 (+22.02%)74.05 (+39.99%)
1f9e5a9 — 2026-07-13 18:35:330.07 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)268.70 (n/a)183.60 (n/a)178.20 (n/a)123.50 (n/a)52.89 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (+0.56%)0.04 (-17.17%)0.04 (-25.13%)0.04 (-21.15%)0.01 (+75.49%)230.60 (+26.77%)196.32 (+24.25%)203.50 (+33.53%)135.00 (-0.52%)39.75 (+120.52%)
1f9e5a9 — 2026-07-13 18:35:330.06 (n/a)0.05 (n/a)0.05 (n/a)0.05 (n/a)0.01 (n/a)181.90 (n/a)158.00 (n/a)152.40 (n/a)135.70 (n/a)18.03 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.05 (-21.44%)0.04 (-14.92%)0.04 (-15.76%)0.04 (+7.76%)0.01 (-49.21%)218.40 (-7.18%)193.98 (+13.89%)191.70 (+18.70%)157.90 (+27.24%)25.41 (-39.77%)
1f9e5a9 — 2026-07-13 18:35:330.07 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)235.30 (n/a)170.32 (n/a)161.50 (n/a)124.10 (n/a)42.18 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_1-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.05 (-7.47%)0.04 (-9.92%)0.04 (-17.64%)0.04 (-11.23%)0.00 (+15.32%)216.60 (+12.64%)195.00 (+11.40%)206.40 (+21.41%)169.00 (+8.06%)20.90 (+37.62%)
1f9e5a9 — 2026-07-13 18:35:330.05 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.00 (n/a)192.30 (n/a)175.04 (n/a)170.00 (n/a)156.40 (n/a)15.19 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.06 (+2.45%)0.05 (+2.49%)0.05 (-2.99%)0.04 (+43.15%)0.01 (-31.69%)227.80 (-30.14%)184.66 (-7.34%)175.50 (+3.05%)147.90 (-2.38%)31.79 (-55.70%)
1f9e5a9 — 2026-07-13 18:35:330.05 (n/a)0.04 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)326.10 (n/a)199.28 (n/a)170.30 (n/a)151.50 (n/a)71.76 (n/a)
iron/operators/mha

test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.21 (n/a)0.21 (n/a)0.21 (n/a)0.20 (n/a)0.00 (n/a)41048.30 (n/a)40864.76 (n/a)40818.90 (n/a)40772.70 (n/a)109.15 (n/a)
iron/operators/mha_prefill_df

test_mha[seq_len_16384-dim_64-num_heads_1-num_pipelines_8-num_kv_heads_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:19:430.21 (n/a)0.21 (n/a)0.21 (n/a)0.21 (n/a)0.00 (n/a)40917.10 (n/a)40783.46 (n/a)40849.20 (n/a)40464.90 (n/a)185.02 (n/a)
iron/operators/mha_prefill_lxl

test_mha_prefill_lxl[GPT2-Small-256seq]

Commit/Date Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:43130450.00 (n/a)28117.44 (n/a)2532.20 (n/a)2420.30 (n/a)57205.70 (n/a)83.18 (n/a)63.93 (n/a)79.51 (n/a)1.54 (n/a)34.97 (n/a)

test_mha_prefill_lxl[H2]

Commit/Date Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:43117957.00 (n/a)24903.74 (n/a)1617.60 (n/a)1609.00 (n/a)52018.38 (n/a)20.85 (n/a)16.43 (n/a)20.74 (n/a)0.28 (n/a)9.05 (n/a)

test_mha_prefill_lxl[Llama3.2-256seq]

Commit/Date Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:19:43140449.30 (n/a)31446.42 (n/a)4212.80 (n/a)4046.80 (n/a)60934.67 (n/a)132.67 (n/a)103.28 (n/a)127.44 (n/a)3.82 (n/a)55.80 (n/a)
iron/operators/rms_norm

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.05 (-19.86%)0.04 (-3.86%)0.04 (-1.17%)0.04 (+2.69%)0.01 (-44.21%)215.20 (-2.58%)185.88 (+1.61%)193.80 (+1.20%)156.90 (+24.72%)25.15 (-31.41%)
17769b8 — 2026-07-09 23:01:350.07 (n/a)0.05 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)220.90 (n/a)182.94 (n/a)191.50 (n/a)125.80 (n/a)36.66 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.10 (+12.00%)0.08 (+8.11%)0.08 (+9.86%)0.07 (+16.52%)0.01 (+11.05%)182.50 (-14.16%)160.84 (-7.63%)160.90 (-8.99%)125.20 (-10.76%)21.99 (-16.52%)
17769b8 — 2026-07-09 23:01:350.09 (n/a)0.07 (n/a)0.07 (n/a)0.06 (n/a)0.01 (n/a)212.60 (n/a)174.12 (n/a)176.80 (n/a)140.30 (n/a)26.34 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.06 (-6.78%)0.05 (-1.21%)0.05 (+3.84%)0.04 (+16.24%)0.01 (-35.81%)193.50 (-13.96%)170.80 (-1.81%)181.40 (-3.66%)134.70 (+7.25%)26.54 (-37.78%)
17769b8 — 2026-07-09 23:01:350.07 (n/a)0.05 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)224.90 (n/a)173.94 (n/a)188.30 (n/a)125.60 (n/a)42.65 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.09 (+55.66%)0.07 (+30.57%)0.06 (+19.69%)0.06 (+35.90%)0.01 (+115.38%)178.60 (-26.41%)160.22 (-22.37%)170.30 (-16.48%)116.20 (-35.77%)25.62 (+0.83%)
17769b8 — 2026-07-09 23:01:350.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)242.70 (n/a)206.40 (n/a)203.90 (n/a)180.90 (n/a)25.41 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.07 (+2.17%)0.05 (-10.07%)0.04 (-12.84%)0.04 (-13.44%)0.01 (+30.43%)215.40 (+15.50%)177.66 (+13.90%)185.10 (+14.76%)111.40 (-2.11%)39.31 (+42.61%)
17769b8 — 2026-07-09 23:01:350.07 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)186.50 (n/a)155.98 (n/a)161.30 (n/a)113.80 (n/a)27.57 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.10 (+47.22%)0.07 (+18.93%)0.06 (-5.14%)0.06 (+99.21%)0.02 (+5.22%)178.90 (-49.80%)153.64 (-21.60%)165.80 (+5.47%)102.30 (-32.03%)30.06 (-66.52%)
17769b8 — 2026-07-09 23:01:350.07 (n/a)0.06 (n/a)0.07 (n/a)0.03 (n/a)0.02 (n/a)356.40 (n/a)195.96 (n/a)157.20 (n/a)150.50 (n/a)89.79 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.07 (+19.15%)0.05 (+7.05%)0.04 (-3.91%)0.03 (-6.88%)0.02 (+87.83%)235.50 (+7.39%)175.98 (-1.17%)185.10 (+4.11%)113.70 (-16.03%)53.96 (+68.82%)
17769b8 — 2026-07-09 23:01:350.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)219.30 (n/a)178.06 (n/a)177.80 (n/a)135.40 (n/a)31.96 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.06 (-12.68%)0.05 (-8.51%)0.05 (-2.93%)0.04 (-3.02%)0.01 (-21.62%)236.50 (+3.14%)197.40 (+8.27%)192.20 (+3.06%)144.40 (+14.51%)35.51 (-5.00%)
17769b8 — 2026-07-09 23:01:350.07 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)229.30 (n/a)182.32 (n/a)186.50 (n/a)126.10 (n/a)37.38 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.05 (-12.34%)0.04 (-7.32%)0.04 (-0.52%)0.03 (-5.48%)0.01 (-23.50%)237.50 (+5.79%)204.04 (+6.76%)204.50 (+0.49%)150.20 (+14.13%)34.87 (-7.02%)
17769b8 — 2026-07-09 23:01:350.06 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)224.50 (n/a)191.12 (n/a)203.50 (n/a)131.60 (n/a)37.51 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.05 (-17.82%)0.05 (-13.24%)0.04 (-14.33%)0.04 (-5.09%)0.00 (-46.76%)219.20 (+5.38%)204.50 (+14.12%)208.70 (+16.72%)177.40 (+21.67%)16.58 (-32.33%)
17769b8 — 2026-07-09 23:01:350.06 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)208.00 (n/a)179.20 (n/a)178.80 (n/a)145.80 (n/a)24.50 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.04 (-22.21%)0.04 (-14.56%)0.04 (-13.46%)0.03 (-2.61%)0.00 (-55.94%)239.50 (+2.70%)220.54 (+15.28%)219.00 (+15.51%)200.50 (+28.61%)18.13 (-41.18%)
17769b8 — 2026-07-09 23:01:350.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.01 (n/a)233.20 (n/a)191.30 (n/a)189.60 (n/a)155.90 (n/a)30.82 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.05 (+11.33%)0.04 (-1.87%)0.04 (-3.32%)0.03 (-27.30%)0.01 (+241.43%)304.80 (+37.55%)216.22 (+6.05%)210.00 (+3.45%)170.60 (-10.16%)53.86 (+323.46%)
17769b8 — 2026-07-09 23:01:350.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)221.60 (n/a)203.88 (n/a)203.00 (n/a)189.90 (n/a)12.72 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.04 (-6.80%)0.04 (-10.61%)0.04 (-11.03%)0.03 (-16.18%)0.00 (+86.90%)237.20 (+19.26%)211.90 (+12.87%)211.70 (+12.43%)184.70 (+7.32%)25.15 (+141.01%)
17769b8 — 2026-07-09 23:01:350.05 (n/a)0.04 (n/a)0.04 (n/a)0.04 (n/a)0.00 (n/a)198.90 (n/a)187.74 (n/a)188.30 (n/a)172.10 (n/a)10.43 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_1-tile_size_256-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.04 (-21.26%)0.04 (-16.86%)0.04 (-20.27%)0.03 (-11.50%)0.00 (-52.87%)252.80 (+12.96%)224.00 (+19.14%)219.80 (+25.39%)209.10 (+26.96%)17.23 (-31.82%)
17769b8 — 2026-07-09 23:01:350.05 (n/a)0.05 (n/a)0.05 (n/a)0.04 (n/a)0.01 (n/a)223.80 (n/a)188.02 (n/a)175.30 (n/a)164.70 (n/a)25.27 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_8-num_channels_2-tile_size_128-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.05 (-4.22%)0.04 (-1.57%)0.04 (-1.38%)0.04 (+8.33%)0.00 (-28.97%)228.90 (-7.70%)202.90 (+0.74%)206.50 (+1.42%)179.40 (+4.36%)20.20 (-32.17%)
17769b8 — 2026-07-09 23:01:350.05 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)248.00 (n/a)201.40 (n/a)203.60 (n/a)171.90 (n/a)29.78 (n/a)
iron/operators/rope

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.58 (-20.91%)0.48 (-25.75%)0.47 (-29.89%)0.39 (-29.37%)0.08 (-6.51%)253.10 (+41.63%)211.34 (+35.65%)209.00 (+42.56%)170.60 (+26.37%)34.47 (+64.54%)
17769b8 — 2026-07-09 23:01:350.73 (n/a)0.64 (n/a)0.67 (n/a)0.55 (n/a)0.08 (n/a)178.70 (n/a)155.80 (n/a)146.60 (n/a)135.00 (n/a)20.95 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.75 (+14.19%)0.60 (+1.64%)0.55 (-5.83%)0.53 (+6.16%)0.09 (+38.89%)184.60 (-5.77%)166.62 (-0.99%)179.60 (+6.15%)131.30 (-12.47%)22.70 (+17.57%)
17769b8 — 2026-07-09 23:01:350.66 (n/a)0.59 (n/a)0.58 (n/a)0.50 (n/a)0.07 (n/a)195.90 (n/a)168.28 (n/a)169.20 (n/a)150.00 (n/a)19.31 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.68 (-10.50%)0.61 (+1.69%)0.63 (+13.16%)0.54 (+7.16%)0.06 (-42.45%)180.60 (-6.71%)162.34 (-2.88%)156.00 (-11.61%)145.40 (+11.67%)15.18 (-38.34%)
17769b8 — 2026-07-09 23:01:350.76 (n/a)0.60 (n/a)0.56 (n/a)0.51 (n/a)0.10 (n/a)193.60 (n/a)167.16 (n/a)176.50 (n/a)130.20 (n/a)24.62 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_8-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.65 (+5.25%)0.51 (-4.27%)0.48 (-8.11%)0.46 (-4.39%)0.08 (+44.97%)213.60 (+4.60%)196.86 (+5.32%)206.20 (+8.81%)151.80 (-5.01%)25.42 (+41.10%)
17769b8 — 2026-07-09 23:01:350.62 (n/a)0.53 (n/a)0.52 (n/a)0.48 (n/a)0.05 (n/a)204.20 (n/a)186.92 (n/a)189.50 (n/a)159.80 (n/a)18.02 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.54 (+12.88%)0.48 (+16.27%)0.48 (+16.94%)0.43 (+16.82%)0.05 (+18.14%)171.90 (-14.39%)154.58 (-13.94%)152.90 (-14.49%)136.80 (-11.40%)15.59 (-8.76%)
17769b8 — 2026-07-09 23:01:350.48 (n/a)0.41 (n/a)0.41 (n/a)0.37 (n/a)0.04 (n/a)200.80 (n/a)179.62 (n/a)178.80 (n/a)154.40 (n/a)17.09 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.49 (-5.74%)0.40 (-14.19%)0.39 (-13.42%)0.34 (-21.40%)0.05 (+72.94%)213.70 (+27.20%)188.94 (+17.75%)189.40 (+15.49%)151.70 (+6.08%)23.68 (+130.78%)
17769b8 — 2026-07-09 23:01:350.52 (n/a)0.46 (n/a)0.45 (n/a)0.44 (n/a)0.03 (n/a)168.00 (n/a)160.46 (n/a)164.00 (n/a)143.00 (n/a)10.26 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.56 (+2.64%)0.45 (+0.73%)0.48 (+9.80%)0.29 (-17.68%)0.11 (+56.08%)258.40 (+21.49%)174.66 (+3.25%)152.40 (-8.91%)130.90 (-2.60%)52.15 (+84.43%)
17769b8 — 2026-07-09 23:01:350.55 (n/a)0.45 (n/a)0.44 (n/a)0.35 (n/a)0.07 (n/a)212.70 (n/a)169.16 (n/a)167.30 (n/a)134.40 (n/a)28.28 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_8-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.59 (+11.42%)0.43 (+5.67%)0.41 (+1.59%)0.35 (+4.94%)0.10 (+31.32%)210.10 (-4.72%)176.72 (-4.26%)178.20 (-1.55%)125.80 (-10.27%)34.68 (+14.99%)
17769b8 — 2026-07-09 23:01:350.53 (n/a)0.41 (n/a)0.41 (n/a)0.33 (n/a)0.07 (n/a)220.50 (n/a)184.58 (n/a)181.00 (n/a)140.20 (n/a)30.16 (n/a)
iron/operators/softmax

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.99 (+18.55%)0.75 (+0.18%)0.71 (-5.10%)0.60 (-4.16%)0.15 (+86.79%)217.80 (+4.36%)180.66 (+1.78%)184.60 (+5.37%)131.80 (-15.62%)32.56 (+60.17%)
17769b8 — 2026-07-09 23:01:350.84 (n/a)0.75 (n/a)0.75 (n/a)0.63 (n/a)0.08 (n/a)208.70 (n/a)177.50 (n/a)175.20 (n/a)156.20 (n/a)20.33 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.90 (-4.84%)0.73 (-1.04%)0.74 (+5.03%)0.60 (+9.10%)0.12 (-28.79%)219.20 (-8.36%)183.74 (-1.07%)177.10 (-4.78%)145.60 (+5.05%)29.80 (-29.77%)
17769b8 — 2026-07-09 23:01:350.95 (n/a)0.74 (n/a)0.70 (n/a)0.55 (n/a)0.17 (n/a)239.20 (n/a)185.72 (n/a)186.00 (n/a)138.60 (n/a)42.43 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.79 (+6.65%)0.67 (+4.84%)0.61 (-0.94%)0.57 (+13.01%)0.11 (+11.01%)231.90 (-11.49%)200.90 (-4.57%)216.40 (+0.98%)166.00 (-6.21%)30.36 (-9.19%)
17769b8 — 2026-07-09 23:01:350.74 (n/a)0.63 (n/a)0.61 (n/a)0.50 (n/a)0.09 (n/a)262.00 (n/a)210.52 (n/a)214.30 (n/a)177.00 (n/a)33.43 (n/a)
iron/operators/swiglu_decode

test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.00 (+2.33%)0.00 (+2.86%)0.00 (+0.00%)0.00 (+13.16%)0.00 (-80.00%)957.84 (-11.23%)945.73 (-3.37%)942.84 (-1.04%)941.27 (-0.96%)6.92 (-87.70%)
17769b8 — 2026-07-09 23:01:350.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)1079.03 (n/a)978.73 (n/a)952.71 (n/a)950.43 (n/a)56.23 (n/a)

test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.01 (+2.44%)0.01 (+1.25%)0.01 (+1.23%)0.01 (-1.30%)0.00 (+39.91%)1073.53 (+0.95%)1016.68 (-0.86%)1004.39 (-0.86%)976.21 (-2.12%)36.26 (+33.17%)
17769b8 — 2026-07-09 23:01:350.01 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)1063.48 (n/a)1025.53 (n/a)1013.10 (n/a)997.32 (n/a)27.23 (n/a)
iron/operators/swiglu_prefill

test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:330.96 (+0.67%)0.94 (-0.68%)0.95 (-0.58%)0.92 (-2.53%)0.01 (+455.08%)2274.04 (+2.59%)2222.86 (+0.70%)2218.69 (+0.58%)2188.25 (-0.66%)32.01 (+467.36%)
17769b8 — 2026-07-09 23:01:350.95 (n/a)0.95 (n/a)0.95 (n/a)0.95 (n/a)0.00 (n/a)2216.53 (n/a)2207.36 (n/a)2205.81 (n/a)2202.82 (n/a)5.64 (n/a)
iron/operators/transpose

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:333.34 (+9.18%)3.08 (+11.07%)3.26 (+22.75%)2.52 (-2.26%)0.34 (+57.47%)207.90 (+2.31%)172.12 (-9.40%)161.00 (-18.56%)156.90 (-8.41%)21.40 (+47.75%)
17769b8 — 2026-07-09 23:01:353.06 (n/a)2.77 (n/a)2.65 (n/a)2.58 (n/a)0.22 (n/a)203.20 (n/a)189.98 (n/a)197.70 (n/a)171.30 (n/a)14.48 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:335.97 (-4.30%)5.32 (+18.02%)5.51 (+28.18%)4.73 (+30.16%)0.56 (-45.08%)221.90 (-23.16%)198.90 (-17.35%)190.10 (-21.99%)175.70 (+4.52%)21.12 (-53.08%)
17769b8 — 2026-07-09 23:01:356.24 (n/a)4.51 (n/a)4.30 (n/a)3.63 (n/a)1.01 (n/a)288.80 (n/a)240.64 (n/a)243.70 (n/a)168.10 (n/a)45.00 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-23 23:08:223.72 (+20.32%)3.12 (+16.02%)3.04 (+6.32%)2.73 (+32.06%)0.43 (+8.15%)192.20 (-24.30%)170.32 (-14.26%)172.50 (-5.94%)141.00 (-16.86%)22.57 (-32.31%)
4d4b803 — 2026-06-22 17:54:573.09 (n/a)2.69 (n/a)2.86 (n/a)2.07 (n/a)0.40 (n/a)253.90 (n/a)198.64 (n/a)183.40 (n/a)169.60 (n/a)33.34 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:35:333.73 (+23.90%)3.26 (+27.93%)3.33 (+34.88%)2.81 (+25.21%)0.41 (+38.93%)186.70 (-20.15%)163.10 (-21.60%)157.20 (-25.88%)140.40 (-19.26%)20.81 (-7.93%)
17769b8 — 2026-07-09 23:01:353.01 (n/a)2.55 (n/a)2.47 (n/a)2.24 (n/a)0.29 (n/a)233.80 (n/a)208.04 (n/a)212.10 (n/a)173.90 (n/a)22.60 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-23 23:08:224.36 (+7.95%)3.26 (+13.04%)3.33 (+22.37%)2.38 (+18.97%)0.79 (-14.92%)220.70 (-15.96%)168.40 (-14.68%)157.40 (-18.28%)120.30 (-7.39%)40.78 (-34.38%)
4d4b803 — 2026-06-22 17:54:574.04 (n/a)2.89 (n/a)2.72 (n/a)2.00 (n/a)0.93 (n/a)262.60 (n/a)197.38 (n/a)192.60 (n/a)129.90 (n/a)62.14 (n/a)
Krackan - Examples

IRON

Tested on 2026_07_13_21_07_38 at commit 2896c25.

iron/applications/llama_3.2_1b
TestChecksTTFT (mean)TPS (mean)
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]✅ 5/52.13n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]✅ 5/52.177.43
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]✅ 5/52.08n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]✅ 5/52.117.22

Trends:

IRON Trends

iron/applications/llama_3.2_1b

test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
2896c25 — 2026-07-13 21:02:302.16 (+0.61%)2.13 (+0.48%)2.13 (+0.52%)2.11 (+0.62%)0.02 (-7.55%)
1f9e5a9 — 2026-07-13 18:05:372.14 (n/a)2.12 (n/a)2.12 (n/a)2.10 (n/a)0.02 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
2896c25 — 2026-07-13 21:02:307.48 (+0.05%)7.43 (+0.24%)7.42 (+0.19%)7.40 (+0.33%)0.03 (-10.27%)2.28 (+0.31%)2.17 (+0.22%)2.14 (+0.05%)2.13 (+0.61%)0.06 (+1.55%)
1f9e5a9 — 2026-07-13 18:05:377.47 (n/a)7.42 (n/a)7.41 (n/a)7.37 (n/a)0.04 (n/a)2.27 (n/a)2.16 (n/a)2.14 (n/a)2.12 (n/a)0.06 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
2896c25 — 2026-07-13 21:02:302.11 (+0.48%)2.08 (+0.24%)2.08 (+0.00%)2.06 (+0.10%)0.02 (+20.44%)
1f9e5a9 — 2026-07-13 18:05:372.10 (n/a)2.08 (n/a)2.08 (n/a)2.06 (n/a)0.02 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
2896c25 — 2026-07-13 21:02:307.24 (-0.44%)7.22 (-0.14%)7.23 (-0.07%)7.20 (+0.01%)0.01 (-41.43%)2.12 (+0.14%)2.11 (+1.23%)2.11 (+1.45%)2.09 (+1.70%)0.01 (-52.15%)
1f9e5a9 — 2026-07-13 18:05:377.27 (n/a)7.24 (n/a)7.24 (n/a)7.20 (n/a)0.02 (n/a)2.12 (n/a)2.08 (n/a)2.08 (n/a)2.06 (n/a)0.02 (n/a)
Phoenix - Small

IRON

Tested on 2026_07_13_21_10_45 at commit 2896c25.

iron/operators/axpy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]✅ 5/5427.100.03n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]✅ 5/5438.980.03n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]✅ 5/5497.620.03n/a
iron/operators/dequant
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]✅ 5/5344.280.02n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]✅ 5/5554.860.01n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]✅ 5/5364.240.02n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]✅ 5/5501.620.01n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]✅ 5/5649.280.01n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]✅ 5/5461.560.01n/a
iron/operators/elementwise_add
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5390.620.03n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5635.020.03n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5483.240.03n/a
iron/operators/elementwise_mul
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5287.520.05n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5316.600.04n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5291.820.04n/a
iron/operators/gelu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5301.140.03n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5381.060.03n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5297.340.03n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5536.480.02n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5355.400.03n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5411.540.02n/a
iron/operators/gemm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5476.440.4720.26
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5841.860.3615.23
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/582322.400.31208.75
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/524658.461.02696.72
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]✅ 5/53554.042.59678.91
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]✅ 5/57028.100.189.91
iron/operators/gemv
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]✅ 5/5n/a0.110.11
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]✅ 5/5n/a3.553.55
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]✅ 5/5n/a6.516.50
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]✅ 5/5n/a9.169.15
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a3.723.72
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a6.426.41
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a10.5010.49
iron/operators/layer_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5501.500.02n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5361.020.03n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5470.840.02n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5397.320.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5502.940.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5556.680.01n/a
iron/operators/leaky_relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]✅ 5/5258.000.03n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]✅ 5/5360.260.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]✅ 5/5352.960.03n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]✅ 5/5717.840.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]✅ 5/5382.520.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]✅ 5/5377.780.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]✅ 5/5308.040.03n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]✅ 5/5585.780.02n/a
iron/operators/mem_copy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]✅ 5/5251.200.03n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]✅ 5/5772.640.02n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]✅ 5/5361.960.03n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]✅ 5/5434.620.02n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]✅ 5/5338.700.03n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]✅ 5/5743.720.02n/a
iron/operators/mha_prefill_lxl
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mha_prefill_lxl[GPT2-Small-256seq]❌ 0/528983.98n/a8.72
test_mha_prefill_lxl[H2]❌ 0/535530.60n/a2.64
test_mha_prefill_lxl[Llama3.2-256seq]❌ 0/544873.64n/a13.13

Trends:

IRON Trends

iron/operators/axpy

test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (-11.78%)0.03 (-12.32%)0.03 (-33.54%)0.02 (-16.53%)0.01 (-0.35%)593.80 (+19.81%)427.10 (+15.48%)471.50 (+50.45%)295.50 (+13.39%)127.42 (+21.39%)
1f9e5a9 — 2026-07-13 18:23:110.05 (n/a)0.04 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)495.60 (n/a)369.86 (n/a)313.40 (n/a)260.60 (n/a)104.97 (n/a)

test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.05 (+24.32%)0.03 (-3.85%)0.02 (-34.34%)0.02 (-13.02%)0.01 (+66.82%)578.10 (+14.98%)438.98 (+11.03%)519.40 (+52.32%)246.30 (-19.56%)147.26 (+50.41%)
1f9e5a9 — 2026-07-13 18:23:110.04 (n/a)0.03 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)502.80 (n/a)395.36 (n/a)341.00 (n/a)306.20 (n/a)97.90 (n/a)

test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (+7.04%)0.03 (-5.98%)0.02 (-6.75%)0.02 (-1.69%)0.01 (+9.67%)604.30 (+1.72%)497.62 (+7.22%)568.60 (+7.24%)278.80 (-6.60%)134.16 (+2.40%)
1f9e5a9 — 2026-07-13 18:23:110.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)594.10 (n/a)464.10 (n/a)530.20 (n/a)298.50 (n/a)131.02 (n/a)
iron/operators/dequant

test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.02 (+26.64%)0.02 (+33.87%)0.02 (+56.52%)0.01 (-3.26%)0.00 (+89.32%)569.80 (+3.37%)344.28 (-20.76%)272.50 (-36.11%)252.80 (-21.05%)132.13 (+59.32%)
1f9e5a9 — 2026-07-13 18:23:110.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)551.20 (n/a)434.46 (n/a)426.50 (n/a)320.20 (n/a)82.94 (n/a)

test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.02 (-5.24%)0.01 (-38.67%)0.01 (-42.78%)0.00 (-60.42%)0.01 (+32.12%)1092.00 (+152.66%)554.86 (+96.44%)458.60 (+74.77%)225.30 (+5.53%)324.37 (+260.73%)
1f9e5a9 — 2026-07-13 18:23:110.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)432.20 (n/a)282.46 (n/a)262.40 (n/a)213.50 (n/a)89.92 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (+14.54%)0.02 (-9.48%)0.01 (-47.09%)0.01 (+13.09%)0.01 (+15.27%)464.50 (-11.57%)364.24 (+12.23%)448.70 (+89.01%)202.10 (-12.70%)130.89 (-1.11%)
1f9e5a9 — 2026-07-13 18:23:110.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)525.30 (n/a)324.56 (n/a)237.40 (n/a)231.50 (n/a)132.36 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.02 (-5.30%)0.01 (+16.61%)0.02 (+69.76%)0.00 (+91.91%)0.01 (-16.47%)1112.30 (-47.89%)501.62 (-34.63%)318.40 (-41.10%)286.00 (+5.61%)351.91 (-54.76%)
1f9e5a9 — 2026-07-13 18:23:110.02 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)0.01 (n/a)2134.60 (n/a)767.30 (n/a)540.60 (n/a)270.80 (n/a)777.94 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.01 (-40.19%)0.01 (-31.77%)0.01 (-12.03%)0.00 (-44.15%)0.00 (-42.30%)1083.90 (+79.07%)649.28 (+45.89%)548.20 (+13.69%)400.90 (+67.18%)277.25 (+69.43%)
1f9e5a9 — 2026-07-13 18:23:110.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)605.30 (n/a)445.04 (n/a)482.20 (n/a)239.80 (n/a)163.64 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.02 (+24.47%)0.01 (+9.52%)0.01 (+2.56%)0.01 (-9.17%)0.00 (+67.35%)632.10 (+10.08%)461.56 (-5.18%)473.30 (-2.49%)284.60 (-19.65%)124.52 (+42.97%)
1f9e5a9 — 2026-07-13 18:23:110.01 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)574.20 (n/a)486.80 (n/a)485.40 (n/a)354.20 (n/a)87.09 (n/a)
iron/operators/elementwise_add

test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)497.40 (n/a)390.62 (n/a)405.90 (n/a)282.70 (n/a)97.93 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.05 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)1347.60 (n/a)635.02 (n/a)519.70 (n/a)251.30 (n/a)415.86 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.05 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)643.20 (n/a)483.24 (n/a)476.90 (n/a)270.40 (n/a)146.04 (n/a)
iron/operators/elementwise_mul

test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.06 (n/a)0.05 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)407.20 (n/a)287.52 (n/a)236.60 (n/a)216.50 (n/a)84.44 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.05 (n/a)0.04 (n/a)0.05 (n/a)0.02 (n/a)0.01 (n/a)561.70 (n/a)316.60 (n/a)267.30 (n/a)235.00 (n/a)137.83 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.05 (n/a)0.04 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)457.10 (n/a)291.82 (n/a)250.30 (n/a)243.00 (n/a)92.56 (n/a)
iron/operators/gelu

test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.00 (n/a)402.50 (n/a)301.14 (n/a)283.90 (n/a)253.20 (n/a)58.82 (n/a)

test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.05 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)0.02 (n/a)587.70 (n/a)381.06 (n/a)393.10 (n/a)162.00 (n/a)180.47 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)454.60 (n/a)297.34 (n/a)260.50 (n/a)211.80 (n/a)93.79 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)1008.10 (n/a)536.48 (n/a)436.00 (n/a)285.50 (n/a)284.56 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)578.30 (n/a)355.40 (n/a)323.80 (n/a)243.20 (n/a)134.57 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)544.60 (n/a)411.54 (n/a)457.40 (n/a)286.50 (n/a)114.05 (n/a)
iron/operators/gemm

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:360.58 (+8.50%)0.47 (+21.23%)0.48 (+8.77%)0.38 (+325.43%)0.08 (-57.03%)581.80 (-76.49%)476.44 (-46.16%)456.10 (-8.08%)378.60 (-7.84%)79.85 (-91.06%)24.93 (+8.50%)20.26 (+21.23%)20.69 (+8.77%)16.22 (+325.43%)3.39 (-57.03%)
1f9e5a9 — 2026-07-13 18:23:110.54 (n/a)0.39 (n/a)0.45 (n/a)0.09 (n/a)0.19 (n/a)2475.00 (n/a)884.98 (n/a)496.20 (n/a)410.80 (n/a)893.13 (n/a)22.97 (n/a)16.71 (n/a)19.02 (n/a)3.81 (n/a)7.90 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:360.55 (+2.54%)0.36 (-22.57%)0.40 (-15.29%)0.11 (-71.81%)0.16 (+177.11%)2024.40 (+254.72%)841.86 (+73.21%)558.20 (+18.04%)403.60 (-2.46%)667.99 (+978.61%)23.38 (+2.54%)15.23 (-22.57%)16.90 (-15.29%)4.66 (-71.81%)6.82 (+177.11%)
1f9e5a9 — 2026-07-13 18:23:110.53 (n/a)0.46 (n/a)0.47 (n/a)0.39 (n/a)0.06 (n/a)570.70 (n/a)486.04 (n/a)472.90 (n/a)413.80 (n/a)61.93 (n/a)22.80 (n/a)19.67 (n/a)19.96 (n/a)16.54 (n/a)2.46 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:360.31 (-0.02%)0.31 (-0.61%)0.31 (-0.28%)0.30 (-2.63%)0.01 (+104.50%)84956.20 (+2.70%)82322.40 (+0.63%)81886.00 (+0.28%)80959.20 (+0.02%)1529.25 (+110.93%)212.20 (-0.02%)208.75 (-0.61%)209.80 (-0.28%)202.22 (-2.63%)3.80 (+104.49%)
1f9e5a9 — 2026-07-13 18:23:110.31 (n/a)0.31 (n/a)0.31 (n/a)0.30 (n/a)0.00 (n/a)82720.30 (n/a)81805.12 (n/a)81655.50 (n/a)80946.20 (n/a)724.99 (n/a)212.24 (n/a)210.02 (n/a)210.39 (n/a)207.69 (n/a)1.86 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:361.02 (+0.27%)1.02 (+1.82%)1.02 (+2.12%)1.02 (+2.45%)0.00 (-70.10%)24789.00 (-2.39%)24658.46 (-1.80%)24624.50 (-2.08%)24555.70 (-0.27%)89.87 (-70.88%)699.63 (+0.27%)696.72 (+1.82%)697.68 (+2.12%)693.04 (+2.45%)2.54 (-70.10%)
1f9e5a9 — 2026-07-13 18:23:111.02 (n/a)1.00 (n/a)1.00 (n/a)0.99 (n/a)0.01 (n/a)25396.10 (n/a)25110.42 (n/a)25146.30 (n/a)24622.10 (n/a)308.60 (n/a)697.74 (n/a)684.26 (n/a)683.20 (n/a)676.48 (n/a)8.48 (n/a)

test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:363.73 (-1.75%)2.59 (+18.97%)2.57 (+54.83%)1.36 (-15.66%)0.97 (+3.62%)5933.50 (+18.57%)3554.04 (-13.82%)3134.30 (-35.41%)2163.60 (+1.78%)1531.04 (+23.12%)977.04 (-1.75%)678.91 (+18.97%)674.45 (+54.83%)356.27 (-15.66%)254.13 (+3.62%)
1f9e5a9 — 2026-07-13 18:23:113.79 (n/a)2.18 (n/a)1.66 (n/a)1.61 (n/a)0.94 (n/a)5004.40 (n/a)4124.02 (n/a)4852.70 (n/a)2125.70 (n/a)1243.53 (n/a)994.47 (n/a)570.65 (n/a)435.62 (n/a)422.42 (n/a)245.25 (n/a)

test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:360.25 (-6.09%)0.18 (-12.29%)0.18 (-12.03%)0.14 (-17.02%)0.04 (+13.68%)8986.20 (+20.51%)7028.10 (+15.65%)6752.70 (+13.68%)5048.20 (+6.48%)1469.65 (+45.14%)13.29 (-6.09%)9.91 (-12.29%)9.94 (-12.03%)7.47 (-17.02%)2.19 (+13.68%)
1f9e5a9 — 2026-07-13 18:23:110.26 (n/a)0.21 (n/a)0.21 (n/a)0.17 (n/a)0.04 (n/a)7456.50 (n/a)6076.94 (n/a)5940.20 (n/a)4741.00 (n/a)1012.61 (n/a)14.15 (n/a)11.30 (n/a)11.30 (n/a)9.00 (n/a)1.93 (n/a)
iron/operators/gemv

test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:360.16 (+23.09%)0.11 (+49.37%)0.12 (+98.87%)0.07 (+37.33%)0.03 (+0.43%)0.16 (+23.09%)0.11 (+49.37%)0.11 (+98.87%)0.07 (+37.33%)0.03 (+0.43%)
1f9e5a9 — 2026-07-13 18:23:110.13 (n/a)0.08 (n/a)0.06 (n/a)0.05 (n/a)0.03 (n/a)0.13 (n/a)0.07 (n/a)0.06 (n/a)0.05 (n/a)0.03 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:363.75 (-1.26%)3.55 (-2.59%)3.54 (-2.61%)3.38 (-1.11%)0.17 (+7.02%)3.75 (-1.26%)3.55 (-2.59%)3.53 (-2.61%)3.37 (-1.11%)0.17 (+7.02%)
1f9e5a9 — 2026-07-13 18:23:113.80 (n/a)3.64 (n/a)3.63 (n/a)3.41 (n/a)0.16 (n/a)3.80 (n/a)3.64 (n/a)3.63 (n/a)3.41 (n/a)0.16 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:367.27 (-3.86%)6.51 (-1.58%)6.65 (+0.86%)5.72 (+0.84%)0.72 (+7.80%)7.26 (-3.86%)6.50 (-1.58%)6.64 (+0.86%)5.72 (+0.84%)0.72 (+7.80%)
1f9e5a9 — 2026-07-13 18:23:117.56 (n/a)6.61 (n/a)6.59 (n/a)5.68 (n/a)0.67 (n/a)7.56 (n/a)6.61 (n/a)6.59 (n/a)5.67 (n/a)0.67 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:3611.56 (-16.79%)9.16 (+5.64%)8.44 (+6.40%)7.53 (+34.73%)1.59 (-49.09%)11.56 (-16.79%)9.15 (+5.64%)8.43 (+6.40%)7.53 (+34.73%)1.59 (-49.09%)
1f9e5a9 — 2026-07-13 18:23:1113.90 (n/a)8.67 (n/a)7.93 (n/a)5.59 (n/a)3.13 (n/a)13.89 (n/a)8.66 (n/a)7.93 (n/a)5.59 (n/a)3.12 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:363.94 (+0.21%)3.72 (+1.66%)3.67 (+0.67%)3.54 (+4.13%)0.17 (-8.81%)3.94 (+0.21%)3.72 (+1.66%)3.67 (+0.67%)3.54 (+4.13%)0.17 (-8.81%)
1f9e5a9 — 2026-07-13 18:23:113.93 (n/a)3.66 (n/a)3.65 (n/a)3.40 (n/a)0.19 (n/a)3.93 (n/a)3.66 (n/a)3.65 (n/a)3.40 (n/a)0.19 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:367.41 (-1.74%)6.42 (-1.47%)6.59 (-3.37%)5.29 (-6.77%)0.93 (+14.99%)7.41 (-1.74%)6.41 (-1.47%)6.58 (-3.37%)5.29 (-6.77%)0.93 (+14.99%)
1f9e5a9 — 2026-07-13 18:23:117.54 (n/a)6.51 (n/a)6.82 (n/a)5.68 (n/a)0.81 (n/a)7.54 (n/a)6.51 (n/a)6.81 (n/a)5.67 (n/a)0.81 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:3613.70 (+1.87%)10.50 (+1.81%)8.57 (-15.99%)8.13 (-1.95%)2.91 (+49.86%)13.69 (+1.87%)10.49 (+1.81%)8.57 (-15.99%)8.13 (-1.95%)2.91 (+49.86%)
1f9e5a9 — 2026-07-13 18:23:1113.45 (n/a)10.31 (n/a)10.20 (n/a)8.29 (n/a)1.94 (n/a)13.44 (n/a)10.30 (n/a)10.20 (n/a)8.29 (n/a)1.94 (n/a)
iron/operators/layer_norm

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (n/a)0.02 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)1048.40 (n/a)501.50 (n/a)274.20 (n/a)233.90 (n/a)360.55 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)596.10 (n/a)361.02 (n/a)319.90 (n/a)233.70 (n/a)143.24 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (n/a)0.02 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)1080.70 (n/a)470.84 (n/a)267.60 (n/a)239.10 (n/a)360.99 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)571.90 (n/a)397.32 (n/a)455.30 (n/a)227.20 (n/a)143.76 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)709.90 (n/a)502.94 (n/a)510.50 (n/a)342.20 (n/a)136.00 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.02 (n/a)0.01 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)675.00 (n/a)556.68 (n/a)533.50 (n/a)508.90 (n/a)67.43 (n/a)
iron/operators/leaky_relu

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (+12.78%)0.03 (+25.76%)0.03 (+34.73%)0.03 (+57.97%)0.00 (-41.60%)299.20 (-36.69%)258.00 (-25.35%)250.20 (-25.78%)209.10 (-11.36%)35.93 (-66.43%)
1f9e5a9 — 2026-07-13 18:23:110.03 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)472.60 (n/a)345.62 (n/a)337.10 (n/a)235.90 (n/a)107.02 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (-29.58%)0.02 (-11.16%)0.02 (-19.92%)0.02 (-7.72%)0.01 (-40.76%)508.90 (+8.35%)360.26 (+6.02%)365.90 (+24.88%)255.80 (+41.95%)103.65 (-16.90%)
1f9e5a9 — 2026-07-13 18:23:110.05 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)469.70 (n/a)339.80 (n/a)293.00 (n/a)180.20 (n/a)124.74 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (-2.56%)0.03 (-7.11%)0.03 (+13.03%)0.01 (-37.01%)0.01 (+69.01%)601.10 (+58.77%)352.96 (+18.47%)263.70 (-11.51%)232.50 (+2.60%)153.95 (+177.42%)
1f9e5a9 — 2026-07-13 18:23:110.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)378.60 (n/a)297.92 (n/a)298.00 (n/a)226.60 (n/a)55.49 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (+0.74%)0.02 (-18.62%)0.02 (-41.49%)0.00 (-74.51%)0.01 (+42.59%)2067.40 (+292.22%)717.84 (+98.78%)483.10 (+70.89%)235.40 (-0.72%)768.79 (+418.24%)
1f9e5a9 — 2026-07-13 18:23:110.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)527.10 (n/a)361.12 (n/a)282.70 (n/a)237.10 (n/a)148.35 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (-1.20%)0.02 (-14.57%)0.02 (-16.38%)0.01 (-23.92%)0.01 (+54.58%)556.40 (+31.44%)382.52 (+28.70%)354.70 (+19.59%)229.70 (+1.23%)157.39 (+103.51%)
1f9e5a9 — 2026-07-13 18:23:110.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)423.30 (n/a)297.22 (n/a)296.60 (n/a)226.90 (n/a)77.34 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (-0.38%)0.02 (-0.80%)0.03 (+22.02%)0.01 (-9.04%)0.01 (+3.40%)596.60 (+9.93%)377.78 (+3.34%)287.00 (-18.05%)241.60 (+0.37%)157.96 (+20.75%)
1f9e5a9 — 2026-07-13 18:23:110.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)542.70 (n/a)365.56 (n/a)350.20 (n/a)240.70 (n/a)130.81 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (-4.57%)0.03 (+29.00%)0.03 (+90.90%)0.02 (+10.63%)0.01 (-20.13%)470.90 (-9.62%)308.04 (-25.63%)254.00 (-47.62%)246.70 (+4.80%)95.39 (-27.58%)
1f9e5a9 — 2026-07-13 18:23:110.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)521.00 (n/a)414.22 (n/a)484.90 (n/a)235.40 (n/a)131.71 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (+2.61%)0.02 (+3.27%)0.02 (+17.40%)0.01 (-57.18%)0.01 (+66.40%)1373.60 (+133.57%)585.78 (+21.58%)417.20 (-14.82%)314.50 (-2.54%)442.84 (+343.55%)
1f9e5a9 — 2026-07-13 18:23:110.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)588.10 (n/a)481.80 (n/a)489.80 (n/a)322.70 (n/a)99.84 (n/a)
iron/operators/mem_copy

test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (-6.64%)0.03 (+8.47%)0.03 (+6.60%)0.03 (+34.82%)0.00 (-66.47%)271.10 (-25.83%)251.20 (-10.01%)250.10 (-6.19%)235.90 (+7.08%)13.79 (-74.02%)
1f9e5a9 — 2026-07-13 18:23:110.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)365.50 (n/a)279.14 (n/a)266.60 (n/a)220.30 (n/a)53.08 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (+11.78%)0.02 (-2.59%)0.02 (+5.67%)0.00 (-75.61%)0.01 (+77.91%)2454.10 (+310.04%)772.64 (+82.67%)389.20 (-5.37%)238.60 (-10.57%)942.46 (+692.36%)
1f9e5a9 — 2026-07-13 18:23:110.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)598.50 (n/a)422.98 (n/a)411.30 (n/a)266.80 (n/a)118.94 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.04 (+14.25%)0.03 (-0.69%)0.02 (-24.17%)0.01 (-18.15%)0.01 (+32.93%)612.80 (+22.17%)361.96 (+7.32%)354.70 (+31.91%)198.10 (-12.46%)163.64 (+36.68%)
1f9e5a9 — 2026-07-13 18:23:110.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)501.60 (n/a)337.26 (n/a)268.90 (n/a)226.30 (n/a)119.73 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (-7.43%)0.02 (-17.89%)0.02 (-48.25%)0.01 (+1.80%)0.01 (+19.03%)599.30 (-1.77%)434.62 (+27.78%)530.80 (+93.23%)234.20 (+8.03%)179.93 (+14.85%)
1f9e5a9 — 2026-07-13 18:23:110.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)610.10 (n/a)340.12 (n/a)274.70 (n/a)216.80 (n/a)156.67 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.03 (-16.54%)0.03 (-8.73%)0.02 (-21.49%)0.02 (+23.65%)0.01 (-33.16%)431.10 (-19.12%)338.70 (+2.74%)328.90 (+27.38%)245.90 (+19.78%)87.07 (-35.26%)
1f9e5a9 — 2026-07-13 18:23:110.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)533.00 (n/a)329.68 (n/a)258.20 (n/a)205.30 (n/a)134.50 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
2896c25 — 2026-07-13 21:07:360.02 (-38.51%)0.02 (-20.24%)0.02 (-1.35%)0.00 (-65.00%)0.01 (-26.10%)1968.10 (+185.69%)743.72 (+59.33%)469.10 (+1.36%)368.50 (+62.62%)685.80 (+317.50%)
1f9e5a9 — 2026-07-13 18:23:110.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)688.90 (n/a)466.78 (n/a)462.80 (n/a)226.60 (n/a)164.26 (n/a)
iron/operators/mha_prefill_lxl

test_mha_prefill_lxl[GPT2-Small-256seq]

Commit/Date Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:3665242.00 (n/a)28983.98 (n/a)20374.50 (n/a)18406.20 (n/a)20293.50 (n/a)10.94 (n/a)8.72 (n/a)9.88 (n/a)3.09 (n/a)3.19 (n/a)

test_mha_prefill_lxl[H2]

Commit/Date Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:36135992.10 (n/a)35530.60 (n/a)10681.40 (n/a)9550.20 (n/a)56166.76 (n/a)3.51 (n/a)2.64 (n/a)3.14 (n/a)0.25 (n/a)1.37 (n/a)

test_mha_prefill_lxl[Llama3.2-256seq]

Commit/Date Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
2896c25 — 2026-07-13 21:07:3677248.90 (n/a)44873.64 (n/a)35319.50 (n/a)35087.40 (n/a)18312.03 (n/a)15.30 (n/a)13.13 (n/a)15.20 (n/a)6.95 (n/a)3.60 (n/a)
iron/operators/rms_norm

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.03 (-7.02%)0.02 (-15.15%)0.02 (-31.90%)0.01 (-16.38%)0.01 (+1.69%)588.00 (+19.59%)425.86 (+20.64%)428.00 (+46.83%)260.20 (+7.57%)153.56 (+26.18%)
17769b8 — 2026-07-09 22:38:300.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)491.70 (n/a)353.00 (n/a)291.50 (n/a)241.90 (n/a)121.70 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.05 (-0.35%)0.03 (-14.90%)0.03 (-42.93%)0.02 (-8.24%)0.01 (-2.50%)612.40 (+8.97%)422.60 (+17.58%)473.00 (+75.25%)242.30 (+0.37%)151.48 (+4.63%)
17769b8 — 2026-07-09 22:38:300.05 (n/a)0.04 (n/a)0.05 (n/a)0.02 (n/a)0.01 (n/a)562.00 (n/a)359.42 (n/a)269.90 (n/a)241.40 (n/a)144.78 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.04 (+15.34%)0.02 (-5.60%)0.02 (-34.96%)0.01 (-13.82%)0.01 (+18.69%)672.10 (+16.04%)420.52 (+9.41%)452.70 (+53.72%)202.50 (-13.31%)183.53 (+10.06%)
17769b8 — 2026-07-09 22:38:300.04 (n/a)0.02 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)579.20 (n/a)384.34 (n/a)294.50 (n/a)233.60 (n/a)166.75 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.05 (+26.29%)0.02 (-10.33%)0.02 (-26.09%)0.02 (-5.44%)0.01 (+45.91%)595.40 (+5.75%)498.28 (+17.91%)566.40 (+35.31%)225.30 (-20.84%)155.09 (+17.81%)
17769b8 — 2026-07-09 22:38:300.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)563.00 (n/a)422.60 (n/a)418.60 (n/a)284.60 (n/a)131.65 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.03 (-3.01%)0.02 (+33.51%)0.03 (+61.36%)0.02 (+98.26%)0.01 (-22.70%)529.20 (-49.56%)361.94 (-34.10%)300.30 (-38.02%)272.20 (+3.11%)112.39 (-62.16%)
17769b8 — 2026-07-09 22:38:300.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)1049.10 (n/a)549.26 (n/a)484.50 (n/a)264.00 (n/a)297.02 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.04 (+9.19%)0.02 (-14.34%)0.02 (-38.94%)0.01 (+86.76%)0.01 (-9.46%)1078.80 (-46.46%)583.30 (-14.68%)540.50 (+63.74%)249.70 (-8.40%)306.47 (-59.07%)
17769b8 — 2026-07-09 22:38:300.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)2014.80 (n/a)683.68 (n/a)330.10 (n/a)272.60 (n/a)748.79 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.03 (-36.12%)0.02 (-35.92%)0.02 (-13.94%)0.00 (-71.09%)0.01 (-26.86%)1984.80 (+245.90%)771.40 (+99.18%)493.50 (+16.20%)315.40 (+56.53%)685.01 (+365.63%)
17769b8 — 2026-07-09 22:38:300.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)573.80 (n/a)387.28 (n/a)424.70 (n/a)201.50 (n/a)147.11 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.03 (+29.75%)0.02 (-7.58%)0.02 (-24.38%)0.01 (-24.25%)0.01 (+232.56%)654.30 (+32.00%)513.26 (+16.69%)574.30 (+32.27%)290.70 (-22.93%)143.06 (+229.73%)
17769b8 — 2026-07-09 22:38:300.02 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)495.70 (n/a)439.86 (n/a)434.20 (n/a)377.20 (n/a)43.39 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.04 (+94.03%)0.02 (+54.11%)0.02 (+7.01%)0.02 (+99.15%)0.01 (+128.93%)515.50 (-49.79%)410.42 (-33.17%)494.70 (-6.55%)212.00 (-48.46%)137.43 (-42.47%)
17769b8 — 2026-07-09 22:38:300.02 (n/a)0.01 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)1026.60 (n/a)614.14 (n/a)529.40 (n/a)411.30 (n/a)238.90 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.02 (-29.99%)0.02 (-29.71%)0.02 (-41.03%)0.02 (+4.05%)0.00 (-62.67%)553.30 (-3.91%)488.32 (+32.32%)516.50 (+69.62%)376.40 (+42.85%)68.33 (-48.91%)
17769b8 — 2026-07-09 22:38:300.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)575.80 (n/a)369.04 (n/a)304.50 (n/a)263.50 (n/a)133.75 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.03 (+18.27%)0.02 (+27.77%)0.02 (+26.23%)0.01 (+67.91%)0.01 (+7.21%)648.00 (-40.45%)460.76 (-27.10%)449.80 (-20.78%)282.20 (-15.46%)176.25 (-44.20%)
17769b8 — 2026-07-09 22:38:300.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)1088.10 (n/a)632.02 (n/a)567.80 (n/a)333.80 (n/a)315.84 (n/a)
iron/operators/rope

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.38 (+1.31%)0.25 (+2.33%)0.20 (-3.75%)0.15 (+20.28%)0.10 (-13.11%)649.10 (-16.86%)453.38 (-8.49%)500.80 (+3.90%)258.30 (-1.30%)166.99 (-27.64%)
17769b8 — 2026-07-09 22:38:300.38 (n/a)0.24 (n/a)0.20 (n/a)0.13 (n/a)0.12 (n/a)780.70 (n/a)495.44 (n/a)482.00 (n/a)261.70 (n/a)230.78 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.42 (+44.13%)0.28 (+18.19%)0.22 (-5.39%)0.19 (-1.42%)0.10 (+145.73%)519.30 (+1.45%)388.98 (-9.80%)437.60 (+5.70%)235.50 (-30.59%)116.94 (+68.98%)
17769b8 — 2026-07-09 22:38:300.29 (n/a)0.23 (n/a)0.24 (n/a)0.19 (n/a)0.04 (n/a)511.90 (n/a)431.24 (n/a)414.00 (n/a)339.30 (n/a)69.21 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.24 (-35.47%)0.18 (-36.41%)0.18 (-47.14%)0.09 (-45.35%)0.06 (-45.65%)1103.60 (+82.99%)613.44 (+53.98%)551.50 (+89.19%)402.00 (+54.97%)282.69 (+60.99%)
17769b8 — 2026-07-09 22:38:300.38 (n/a)0.29 (n/a)0.34 (n/a)0.16 (n/a)0.11 (n/a)603.10 (n/a)398.38 (n/a)291.50 (n/a)259.40 (n/a)175.59 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.28 (+2.88%)0.21 (+4.65%)0.21 (+27.10%)0.15 (+3.53%)0.06 (+0.85%)501.00 (-3.41%)381.96 (-4.31%)356.60 (-21.32%)262.20 (-2.82%)113.61 (+1.82%)
17769b8 — 2026-07-09 22:38:300.27 (n/a)0.20 (n/a)0.16 (n/a)0.14 (n/a)0.06 (n/a)518.70 (n/a)399.16 (n/a)453.20 (n/a)269.80 (n/a)111.58 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.29 (+17.18%)0.24 (+34.98%)0.27 (+75.56%)0.15 (+34.37%)0.06 (+8.00%)482.90 (-25.57%)332.30 (-27.19%)277.30 (-43.04%)255.60 (-14.66%)96.61 (-30.22%)
17769b8 — 2026-07-09 22:38:300.25 (n/a)0.17 (n/a)0.15 (n/a)0.11 (n/a)0.05 (n/a)648.80 (n/a)456.40 (n/a)486.80 (n/a)299.50 (n/a)138.44 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.26 (-11.69%)0.18 (-18.40%)0.18 (-36.03%)0.13 (+34.68%)0.05 (-40.16%)576.30 (-25.74%)433.38 (+8.26%)409.60 (+56.34%)288.00 (+13.25%)124.22 (-45.13%)
17769b8 — 2026-07-09 22:38:300.29 (n/a)0.22 (n/a)0.28 (n/a)0.10 (n/a)0.09 (n/a)776.10 (n/a)400.32 (n/a)262.00 (n/a)254.30 (n/a)226.37 (n/a)
iron/operators/softmax

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.48 (+83.82%)0.33 (+41.39%)0.26 (+18.21%)0.22 (+6.84%)0.11 (+368.76%)594.80 (-6.40%)440.90 (-23.27%)497.00 (-15.42%)272.80 (-45.59%)138.01 (+134.10%)
17769b8 — 2026-07-09 22:38:300.26 (n/a)0.23 (n/a)0.22 (n/a)0.21 (n/a)0.02 (n/a)635.50 (n/a)574.58 (n/a)587.60 (n/a)501.40 (n/a)58.95 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.44 (+8.72%)0.30 (+13.26%)0.30 (+11.21%)0.20 (+27.70%)0.09 (+4.81%)647.90 (-21.68%)469.64 (-13.07%)441.90 (-10.07%)301.10 (-8.00%)140.16 (-24.08%)
17769b8 — 2026-07-09 22:38:300.40 (n/a)0.27 (n/a)0.27 (n/a)0.16 (n/a)0.09 (n/a)827.30 (n/a)540.28 (n/a)491.40 (n/a)327.30 (n/a)184.62 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.49 (+12.88%)0.34 (+3.15%)0.40 (+3.42%)0.13 (+3.90%)0.15 (+20.89%)1028.90 (-3.76%)493.84 (-0.01%)326.10 (-3.29%)266.60 (-11.43%)319.61 (-1.86%)
17769b8 — 2026-07-09 22:38:300.44 (n/a)0.33 (n/a)0.39 (n/a)0.12 (n/a)0.13 (n/a)1069.10 (n/a)493.90 (n/a)337.20 (n/a)301.00 (n/a)325.66 (n/a)
iron/operators/swiglu_decode

test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.00 (+16.67%)0.00 (+42.86%)0.00 (+50.00%)0.00 (+0.00%)0.00 (+31.10%)21443.72 (-0.72%)13189.90 (-16.59%)15663.38 (-8.39%)5681.54 (-15.15%)6854.89 (+24.83%)
17769b8 — 2026-07-09 22:38:300.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)21598.80 (n/a)15813.49 (n/a)17097.23 (n/a)6695.96 (n/a)5491.52 (n/a)

test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.00 (-7.69%)0.00 (-2.44%)0.00 (+50.00%)0.00 (+0.00%)0.00 (-5.39%)19265.52 (-10.23%)12265.28 (+0.02%)8914.40 (-32.94%)7112.82 (+13.27%)6157.45 (-0.80%)
17769b8 — 2026-07-09 22:38:300.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)21462.12 (n/a)12263.19 (n/a)13292.95 (n/a)6279.46 (n/a)6206.94 (n/a)
iron/operators/swiglu_prefill

test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:110.16 (+18.96%)0.11 (+15.00%)0.08 (-8.39%)0.07 (-4.08%)0.04 (+78.71%)28796.81 (+4.20%)22064.41 (-6.61%)26678.69 (+9.11%)13318.89 (-15.94%)7552.47 (+57.19%)
17769b8 — 2026-07-09 22:38:300.13 (n/a)0.09 (n/a)0.09 (n/a)0.08 (n/a)0.02 (n/a)27636.97 (n/a)23625.81 (n/a)24451.70 (n/a)15844.81 (n/a)4804.73 (n/a)
iron/operators/transpose

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:111.00 (-43.62%)0.55 (-62.33%)0.46 (-69.91%)0.16 (-83.84%)0.42 (+42.14%)3266.60 (+518.79%)1746.34 (+369.65%)1142.10 (+232.30%)522.70 (+77.37%)1409.58 (+1445.91%)
17769b8 — 2026-07-09 22:38:301.78 (n/a)1.47 (n/a)1.53 (n/a)0.99 (n/a)0.29 (n/a)527.90 (n/a)371.84 (n/a)343.70 (n/a)294.70 (n/a)91.18 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:112.76 (+6.55%)2.25 (+6.13%)2.45 (+25.47%)1.56 (-12.13%)0.52 (+52.04%)672.90 (+13.80%)489.40 (-3.02%)427.20 (-20.30%)380.60 (-6.14%)126.75 (+62.66%)
17769b8 — 2026-07-09 22:38:302.59 (n/a)2.12 (n/a)1.96 (n/a)1.77 (n/a)0.34 (n/a)591.30 (n/a)504.64 (n/a)536.00 (n/a)405.50 (n/a)77.93 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.58 (-2.67%)1.21 (+5.65%)1.29 (+41.89%)0.66 (-20.05%)0.38 (-2.53%)792.90 (+25.08%)479.12 (-3.80%)406.80 (-29.52%)331.10 (+2.76%)189.18 (+26.62%)
4bb8427 — 2026-06-23 22:46:491.63 (n/a)1.14 (n/a)0.91 (n/a)0.83 (n/a)0.38 (n/a)633.90 (n/a)498.02 (n/a)577.20 (n/a)322.20 (n/a)149.40 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
1f9e5a9 — 2026-07-13 18:23:111.71 (-13.93%)1.13 (-15.25%)1.10 (-11.79%)0.57 (-37.94%)0.45 (+5.37%)912.50 (+61.13%)536.64 (+26.77%)474.60 (+13.38%)305.70 (+16.19%)242.26 (+96.01%)
17769b8 — 2026-07-09 22:38:301.99 (n/a)1.34 (n/a)1.25 (n/a)0.93 (n/a)0.43 (n/a)566.30 (n/a)423.32 (n/a)418.60 (n/a)263.10 (n/a)123.59 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.75 (-15.44%)1.32 (-0.50%)1.36 (+22.47%)0.77 (-19.20%)0.36 (-21.75%)678.10 (+23.76%)430.02 (-0.09%)385.10 (-18.34%)300.10 (+18.29%)146.96 (+19.46%)
4bb8427 — 2026-06-23 22:46:492.07 (n/a)1.32 (n/a)1.11 (n/a)0.96 (n/a)0.46 (n/a)547.90 (n/a)430.40 (n/a)471.60 (n/a)253.70 (n/a)123.02 (n/a)
Phoenix - Examples

IRON

Tested on 2026_07_13_21_04_16 at commit 2896c25.

Trends:

IRON Trends

andrej
andrej commented Jul 9, 2026
View reviewed changes
Comment thread aie_kernels/aie2/softmax.cc Outdated
Comment on lines +68 to +74
// stats[0] = running max
// stats[1] = running sum (of exp(x - max))
// ---------------------------------------------------------------------------

void softmax_partial_stats_impl(bfloat16 *restrict input,
bfloat16 *stats,
const int32_t vector_size)

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For readability, can we make stats a struct rather than an array with hard-coded indices? Also pointer should never alias with the input so we can annotate with *restrict.

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll have struct softmax_stats *stats as the function argument but if that's hard to do from the design.py it's fine if it doesn't match from there and is just a void * there in the function signature

Comment thread aie_kernels/aie2/softmax.cc Outdated
return;
}

// ---------------------------------------------------------------------------

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure code comment style aligns with rest of the file, not sure these // --- are used elsewhere

Comment thread aie_kernels/aie2p/softmax.cc Outdated
Comment on lines +163 to +172
// ---------------------------------------------------------------------------
// Online (partial / tiled) softmax helpers
//
// These three kernels implement a two-pass online softmax that processes a row
// in sub-tile chunks, keeping running max and sum statistics in a small local
// buffer (`stats`). Layout of the stats buffer (bfloat16[16], only [0..1]
// used):
// stats[0] = running max (scaled by log2e)
// stats[1] = running sum (of exp2(x*log2e - max))
// ---------------------------------------------------------------------------

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apply same comments as in aie_kernels/aie2/softmax.cc: comment style, struct instead of array for stats

Comment thread aie_kernels/aie2p/softmax.cc Outdated
Comment on lines +189 to +226
// --- Phase 1: find local max (scaled by log2e) -------------------------
float local_max = -INFINITY;
auto it_in1 = aie::cbegin_restrict_vector<SM_VEC_LEN>((bfloat16 *)input);
for (int i = 0; i < elem_iters; i++) {
input_bf16 = *it_in1++;
scaled_accum = aie::mul(input_bf16, log2e_vec);
float chunk_max = aie::reduce_max(scaled_accum.to_vector<bfloat16>());
if (chunk_max > local_max) {
local_max = chunk_max;
}
}

// --- Phase 2: update running max, rescale running sum ------------------
float old_max = (float)stats[0];
float old_sum = (float)stats[1];

if (local_max > old_max) {
// New max is larger — rescale the old sum by exp2(old_max - new_max)
aie::vector<float, SM_VEC_LEN> diff_vec =
aie::broadcast<float, SM_VEC_LEN>(old_max - local_max);
aie::vector<bfloat16, SM_VEC_LEN> corr = aie::exp2<bfloat16>(diff_vec);
old_sum = old_sum * (float)corr[0];
old_max = local_max;
}

// --- Phase 3: accumulate exp2(input * log2e - max) for this chunk ------
aie::vector<bfloat16, SM_VEC_LEN> max_val_vec =
aie::broadcast<bfloat16, SM_VEC_LEN>((bfloat16)old_max);

auto it_in2 = aie::cbegin_restrict_vector<SM_VEC_LEN>((bfloat16 *)input);
for (int i = 0; i < elem_iters; i++) {
input_bf16 = *it_in2++;
scaled_accum = aie::mul(input_bf16, log2e_vec);
exp_in_accum = aie::sub(scaled_accum, max_val_vec);
aie::vector<bfloat16, SM_VEC_LEN> exp_val =
aie::exp2<bfloat16>(exp_in_accum.to_vector<float>());
exp_val_accum = add(exp_val_accum, exp_val);
}

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this use the same on-line approach as in aie_kernels/aie2/softmax.cc

Comment thread aie_kernels/aie2p/softmax.cc
Comment on lines 84 to 90
void partial_softmax_alias_bf16(bfloat16 *restrict input_vector,
bfloat16 *restrict output_vector,
bfloat16 *restrict scale_buffer,
const int32_t vector_size,
const int32_t row_idx,
const int32_t num_rows,
const bfloat16 scale)

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There appears to be a lot of code duplicated in the new partial softmax functions below. Factor out shared functionality into a separate function used across both, forcing inlining so it does not affect performance -- or, if sharing code is not easily possible, explain what the differences between the two implementations are.

Comment thread aie_kernels/generic/axpy.cc Outdated
Comment on lines +82 to +87
// The tile is interpreted as a `vector_size`-wide horizontal strip of the
// per-head (S, S) attention-score block; idx[0] is the strip's starting
// column within that block, idx[1] is the strip's row within the block.
// The kernel implements the causal mask in-place by writing `a` to elements
// strictly above the diagonal and copying y -> z everywhere else. This
// avoids materialising an H*S*S mask buffer entirely.

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is a general 'triangular fill', not specific to attention. Make the comments reflect this. Consider factoring this out into a separate operator since it has less to do with AXPY, or making it more AXPY-like by allowing a scale or constant addition be applied within the triangular masked region (again using C++ templates).

Comment thread iron/common/fusion.py Outdated
Comment on lines +26 to +34
"""Operator that fuses multiple MLIROperators into one.

Args:
dispatch: Dispatch strategy for the fused operator.
``"auto"`` (default) selects ``"fused"`` on NPU2 and
``"separate"`` on NPU1. ``"fused"`` uses a single-ELF
dispatch (requires NPU2). ``"separate"`` compiles each
sub-operator to its own xclbin and invokes them sequentially.
"""

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fusion's a bit of a big term for what this does. It still reconfigures the array between each operator. Let's say it fuses multiple operators into a "single dispatch." (except for NPU1 where full ELF is not yet supported where this still degenerates into multiple dispatches). The sequential feature would be used on NPU2 for debugging and benchmarking the impact of single-dispatch.

Comment thread iron/common/fusion.py Outdated
Comment on lines -55 to -57
for obj in objs:
obj.filename = f"op{idx}_{obj.filename}"
obj.prefix_symbols = f"op{idx}_"

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed? Disambiguating function names between the sub-operators seems desirable. An explanation / reasoning for removing it is a valid response to this if there are good reasons to remove this.

Comment thread iron/common/fusion.py Outdated
Comment on lines -85 to -86
if len(op.get_kernel_artifacts()) > 0:
mlir_artifact.generator.kwargs["func_prefix"] = f"op{idx}_"

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed? Disambiguating function names between the sub-operators seems desirable. An explanation / reasoning for removing it is a valid response to this if there are good reasons to remove this.

Comment thread iron/common/fusion.py Outdated
Comment on lines +275 to +277
Mirrors the pattern from ``chain_swiglu_artifacts`` in
``iron/operators/swiglu_base.py``: each unique operator gets its own
xclbin + insts compiled separately, linked via ``--xclbin-input``.

@andrej andrej Apr 20, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

irrelevant remove

andrej added a commit to andrej/IRON that referenced this pull request Jul 13, 2026
@andrej
Fix CI benchmark marker + address PR amd#91 review comments
937328d 
CI: register 'benchmark' pytest marker and fold benchmark tests into the
'extensive' suite (conftest adds extensive marker before -m deselection), so
they no longer run in the small suite where large-S benchmarks OOM'd the
process and left tests_latest.csv unwritten.

Kernels:
- softmax (aie2/aie2p): stats buffer -> softmax_stats struct with *restrict;
  drop '// ---' comment banners to match file style.
- softmax (aie2p): rewrite partial-stats to the single-pass online algorithm
  used in aie2; document how partial_softmax_alias differs from the online pair.
- axpy: collapse saxpy/scale/scalar_add into one templated saxpy_family with
  extern-C specializations; generalize the causal mask into a templated
  triangular_fill_impl and update comments (general triangular fill).

sequence.py: clarify 'single dispatch' terminology in dispatch docstrings.
andrej added 3 commits July 13, 2026 14:16
@andrej
Add single-dispatch layer-by-layer multi-head attention (mha_prefill_…
d028a7c 
…lxl)

Add a layer-by-layer (LxL) prefill MHA operator built as an
OperatorSequence, and rename the existing data-flow (DF) MHA operator
for clarity:

- Rename iron/operators/mha -> iron/operators/mha_prefill_df (DF =
  data-flow: single fused kernel).
- Add iron/operators/mha_prefill_lxl (LxL = layer-by-layer: chained
  sub-operators dispatched back-to-back).
- Extend axpy with scale / scalar_add / causal-mask modes and softmax
  with partial/online softmax, used by the LxL design.
- Register the benchmark marker in an operator-local conftest.py.

Update README and operator registrations to the new names.
@andrej
Fix CI benchmark marker + address PR amd#91 review comments
3a476a4 
CI: register 'benchmark' pytest marker and fold benchmark tests into the
'extensive' suite (conftest adds extensive marker before -m deselection), so
they no longer run in the small suite where large-S benchmarks OOM'd the
process and left tests_latest.csv unwritten.

Kernels:
- softmax (aie2/aie2p): stats buffer -> softmax_stats struct with *restrict;
  drop '// ---' comment banners to match file style.
- softmax (aie2p): rewrite partial-stats to the single-pass online algorithm
  used in aie2; document how partial_softmax_alias differs from the online pair.
- axpy: collapse saxpy/scale/scalar_add into one templated saxpy_family with
  extern-C specializations; generalize the causal mask into a templated
  triangular_fill_impl and update comments (general triangular fill).

sequence.py: clarify 'single dispatch' terminology in dispatch docstrings.
@andrej
fix sync regression
55b50d5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jgmelber jgmelber Awaiting requested review from jgmelber jgmelber will be requested when the pull request is marked ready for review jgmelber is a code owner
@hunhoffe hunhoffe Awaiting requested review from hunhoffe hunhoffe will be requested when the pull request is marked ready for review hunhoffe is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@andrej