Epic: Row-oriented scalar functions

[connortsui20]

Right now, functions whose natural implementation operates on one row still have to own much of the machinery for executing that operation over columnar Vortex arrays.

This Epic tracks a RowFn definition that separates those concerns. A scalar function implementor can choose typed row elements and define the computation/operation, while the framework handles batch decoding, dtype validation, constants, null rows, output construction, and more.

Status

In progress.

The work is now an eight-PR stack on top of the merged lane-source prerequisite in #9358. #9353 contains the public framework and executor. #9345 and #9346 migrate primitive arithmetic and comparisons. #9347 through #9350 migrate tensor and spatial functions. #9351 adds benchmark tooling.

RowFn adoption is automatic through a blanket ScalarFnVTable implementation. A type that needs custom vtable hooks can keep them on a separate public type and delegate to a private RowFn kernel through row_fn_return_dtype and execute_rows. Every sealed element-tuple arity supports indexed traversal, while unary and binary kernels keep specialized sources.

InputElement and OutputSink are unsafe to implement. InitializedElement::write is unsafe because its token is not lifetime-branded to one callback row. OutputSink<Options>::sink_dtype receives function options and input dtypes.

Rust 1.97.1 and LLVM 22.1.6 measurements use one CGU, fat LTO, and target-cpu=native. Large per-row arithmetic is within -2.7% to +4.8% of develop, and tensor product paths usually improve. Mixed-constant arithmetic and RowFn comparisons remain a documented LLVM 22 limitation at 4.6–8.5x slower because those loops no longer vectorize.

Subissues

• Tracking Issue: Define the RowFn API #9129
• Tracking Issue: Execute RowFn over Vortex arrays #9130

Goal

A scalar function whose natural implementation operates on one typed row should be able to define that operation without also implementing the surrounding array machinery.

The goals are:

• Define a stable RowFn API for choosing typed inputs, optionally preparing batch state, computing rows, and building outputs.
• Share validation, batch decoding, constant handling, null propagation, output allocation, and validity handling across row functions.
• Allow crates such as vortex-tensor and vortex-spatial to add row representations without changing vortex-array.
• Preserve encoding-aware shortcuts for functions that have a better answer for a specific encoding.
• Avoid a performance penalty for already-efficient row kernels, while making commonly missed optimizations reusable.
• Support optional row outputs for strict functions that may return null from otherwise valid inputs.

Note that we will focus solely on strict functions. as the semantics around non-strict functions are complicated enough that it's probably not worth extending this already-somewhat-complicated API further.

RowFn is also not intended for columnar or zero-copy kernels (not, list_length), kernels with state shared across rows (like), or heterogeneous variadic kernels.

Motivation

We have a good number of scalar function implementations in the Vortex codebase, and because it is simply a trait implementation of ScalarFnVTable, anyone can add their own scalar function.

However, writing a good implementation is not exactly trivial. Suppose we want to add a scalar function Hypot to Vortex that is similar to hypot. This simply gets distance from the origin to the point (x, y) via sqrt(x^2 + y^2).

Even though this is arguably a "basic" scalar function, there are MANY things that the implementor needs to worry about w.r.t. correctness and performance.

• Depending on what the implementor wants, they might need to implement this for when x and y are arbitrary floating-point types such as f16, f32, and f64. Maybe they want to accept integers without an explicit cast!
• How should they deal with null values / validity? What should the result of hypot(null, 5.0) be?
• Depending on how they want their null semantics, they might want to do a bunch of stuff with validity up front before moving onto the compute (perhaps intersect the validity of the 2 inputs).
• This sqrt(x^2 + y^2) operation is small enough that the implementor would want to ensure auto-vectorization can happen, so they need to make sure there is no branching in hot loops.
• How do they pre-allocate memory correctly?
• If one or both columns of x and y are ConstantArray, then they definitely want to precompute x^2 and y^2 rather than doing it n times.

Note that in practice we would probably want to decompose hypot() into an expression that does sqrt(x^2 + y^2) as a tree of numeric expressions for better optimizations, but hopefully you get the idea that there are several things that many scalar function implementations have to worry about, even if the function is "simple".

More often than not, the scalar function implementor is only going to worry about a subset of these and leave potential performance optimizations on the table.

So this begs the question: Can scalar functions be defined in terms of their natural operation on a single typed row, without giving up Vortex’s array semantics, extension types, encoding-aware shortcuts, or performance?

RowFn rationale

Every row-oriented scalar function execution follows roughly the same execution pipeline:

1. Validate the input types.
2. Handle degenerate inputs, such as all null or entirely constant arguments.
3. Decode each input column once.
4. Compute anything that is constant/static for the batch.
5. Read and compute one row at a time (hopefully in a vectorized manner).
6. Build the output and apply its validity.

Without a shared definition, every scalar function has to assemble this pipeline itself.

RowFn makes the owner of each step explicit:

1. RowFn::ARG_NAMES declares the arity, and InputElement validates each element dtype.
2. Null constants and entirely constant inputs are handled automatically.
3. InputElement::decode prepares each column once and stores each batch-constant argument as one decoded row.
4. The prepare closure on the visit_prepared* methods computes state derived from constant arguments once per batch.
5. The sealed tuple adapter reads each row and the function's closure performs the actual computation.
6. An OutputElement or OutputSink builds the result, after which the framework applies the input validity.

Every RowFn receives the standard ScalarFnVTable implementation automatically. A function that needs custom vtable hooks can keep them on a separate public type and delegate row planning and execution to a private RowFn kernel.

Preliminary performance

The current x86 gate uses Rust 1.97.1, LLVM 22.1.6, one codegen unit, fat LTO, and target-cpu=native, with two warm runs and seven alternating measured pairs. Large per-row arithmetic is within -2.7% to +4.8% of develop. Tensor product paths improve by up to 92%, and ordinary spatial distance stays within 2.7%.

LLVM 22 does not vectorize several mixed-constant RowFn loops. Mixed-constant arithmetic and primitive comparisons are 4.6–8.5x slower. This is a documented compiler limitation rather than executor plumbing added solely to influence code generation.

Unresolved questions

• Are nullable outputs from otherwise valid inputs required before the first version is complete? This is needed for strict but non-total functions such as list_sum and variant_get.
• InputElement, OutputElement, and OutputSink are supported downstream extension points. InputElement is unsafe to implement, and OutputSink::finish is unsafe to call. ElementTuple and SinkResult remain sealed executor mechanics.

[connortsui20]

@mhk197 gave me the idea that this could potentially work for gpu execution too 👀