Skip to content

Add hoist_invariants() directive for semiring reductions - #9190

Open
alexreinking wants to merge 7 commits into
alexreinking/change-typefrom
alexreinking/rfactor-hoisting
Open

Add hoist_invariants() directive for semiring reductions#9190
alexreinking wants to merge 7 commits into
alexreinking/change-typefrom
alexreinking/rfactor-hoisting

Conversation

@alexreinking

@alexreinking alexreinking commented Jul 4, 2026

Copy link
Copy Markdown
Member

This PR adds a scheduling directive similar to rfactor that hoists invariant factors according to a detected distributive (semiring) law into the write-back part of the accumulation step. This enables writing straightforward quantized kernels in the algorithm language and factoring them into efficient kernels.

This is meant to work towards optimizing quantized mat-mul and mat-vec kernels (especially those from GGML).

Checklist

  • Tests added or updated (not required for docs, CI config, or typo fixes)
  • Documentation updated (if public API changed)
  • Python bindings updated (if public API changed)
  • Benchmarks are included here if the change is intended to affect performance.
  • Commits include AI attribution where applicable (see Code of Conduct)

@alexreinking alexreinking added the release_notes For changes that may warrant a note in README for official releases. label Jul 4, 2026
@alexreinking alexreinking changed the title Alexreinking/rfactor hoisting Allow hoisting invariant factors in rfactor Jul 4, 2026
@codecov

codecov Bot commented Jul 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.76821% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.37%. Comparing base (db7edbe) to head (c8dc9c6).

Files with missing lines Patch % Lines
src/Func.cpp 84.76% 8 Missing and 15 partials ⚠️
Additional details and impacted files
@@                     Coverage Diff                      @@
##           alexreinking/change-type    #9190      +/-   ##
============================================================
+ Coverage                     70.25%   70.37%   +0.11%     
============================================================
  Files                           258      258              
  Lines                         79605    79714     +109     
  Branches                      19111    19141      +30     
============================================================
+ Hits                          55926    56097     +171     
+ Misses                        17939    17914      -25     
+ Partials                       5740     5703      -37     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alexreinking
alexreinking marked this pull request as draft July 6, 2026 14:22
@alexreinking alexreinking added the dev_meeting Topic to be discussed at the next dev meeting label Jul 6, 2026
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 2 times, most recently from 9e244ff to 328fd0a Compare July 20, 2026 15:28
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 3 times, most recently from f2cbdc3 to f271649 Compare July 29, 2026 19:07
@alexreinking
alexreinking changed the base branch from main to alexreinking/change-type July 29, 2026 19:07
@alexreinking alexreinking changed the title Allow hoisting invariant factors in rfactor Add hoist_invariants() directive Jul 29, 2026
@alexreinking alexreinking changed the title Add hoist_invariants() directive Add hoist_invariants() directive for semiring reductions Jul 29, 2026
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from f271649 to 9660010 Compare July 29, 2026 19:33
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 9660010 to 12834ac Compare July 29, 2026 19:59
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 12834ac to d049c0c Compare July 30, 2026 12:45
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from d049c0c to 460d780 Compare July 30, 2026 13:35
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 2 times, most recently from 8f48f71 to 25be6cb Compare July 30, 2026 15:20
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 3 times, most recently from a26996d to 9787945 Compare July 30, 2026 16:57
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 9787945 to 33e3729 Compare July 30, 2026 18:03
@abadams abadams removed the dev_meeting Topic to be discussed at the next dev meeting label Jul 30, 2026
Comment thread src/Func.cpp
@abadams

abadams commented Jul 30, 2026

Copy link
Copy Markdown
Member

Please add a test that turns a gaussian blur implemented as this:

f(x, y) += kernel(r.x) * kernel(r.y) * input(x + r.x, y + r.y)

into a separable implementation

@abadams

abadams commented Jul 30, 2026

Copy link
Copy Markdown
Member

Also it would be good to demo turning a one-step downsample into two-stage separable downsample (basically the same as the blur example but there's a stride) - it can be a lot faster to separate a downsample into a first stage that just downsamples in y, followed by a second that just downsamples in x.

@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 2 times, most recently from 1c5f111 to 97c1c65 Compare July 30, 2026 20:11
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 97c1c65 to b237059 Compare July 31, 2026 15:49
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch 2 times, most recently from 7f7fd7c to cf8298e Compare July 31, 2026 21:30
@alexreinking
alexreinking marked this pull request as ready for review August 1, 2026 03:18
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from cf8298e to c6aed7f Compare August 4, 2026 19:39
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from c6aed7f to 7d6e34b Compare August 4, 2026 21:37
alexreinking and others added 7 commits August 4, 2026 18:13
Introduce Stage::hoist_invariants(), which applies a semiring distributive law to hoist loop-invariant factors out of associative reductions. The returned intermediate accumulates the factor-free reduction over the original RDom, while the write-back applies the factor once.

Support +/*, min/+, max/+, or/&&, and and/|| laws, including factors nested within associative chains. Add reusable binary-operator helpers, the Python binding, and correctness tests.
Add a quantized (int8 x int8 -> f32) mat-vec performance test targeting ARM's
SDOT instruction, scheduled by composing eager_inline(), hoist_invariants(),
rfactor(), and change_type(). eager_inline() folds the Wt and VecDq producers
into Acc's update so their scale factors -- WtScale(i) and VecScale -- surface
as explicit leaves that hoist_invariants() can then lift out of the reduction as
the invariant product WtScale(i) * VecScale. rfactor() splits the now scale-free
reduction by block, and change_type(Int(32)) retypes the innermost per-block dot
product so CodeGen_ARM matches it to SDOT.

The generated assembly contains sdot instructions, the Hoisted/PlainRfactor
numerical cross-check passes, and the composed schedule is several times faster
than the non-hoisting variant.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: GPT 5.6 Sol <noreply@openai.com>
@alexreinking
alexreinking force-pushed the alexreinking/rfactor-hoisting branch from 7d6e34b to c8dc9c6 Compare August 4, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release_notes For changes that may warrant a note in README for official releases.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants