Return an empty array from fill_null on an empty input by ump45nose · Pull Request #10627 · vortex-data/vortex

ump45nose · 2026-10-20T01:19:55Z

Summary

The array_ops fuzzer crashed with fill_null fill_value must be a constant/scalar. The crash input is a nullable u16 chunked array whose first two chunks are empty, filled with nulls more than once.

When a fill_null lands on a chunked input, the constant fill is sliced to each chunk. ArrayRef::slice returns Canonical::empty for an empty range, so an empty chunk ends up as fill_null(empty chunk, empty PrimitiveArray). Executing that node tries to turn the fill into a constant, gets the empty primitive array back, and fails. An empty input has no nulls to fill, so the fill value doesn't matter there.

Changes

FillNull::execute now returns an empty canonical array of the result dtype (the input dtype with the fill value's nullability) when the input is empty, before it asks for a scalar fill. Compare and the boolean operators already return early for empty inputs in the same way.
New test empty_input_ignores_non_constant_fill runs FillNull on an empty input with an empty, non-constant fill and checks the result is empty and non-nullable.
New test repeated_fill_over_empty_chunk builds a chunked array with an empty chunk, fills nulls twice and checks the values. This is the shape the fuzzer found.

Both new tests fail on develop with the error from the issue and pass with this change. I also replayed the crash file from the issue artifact (crash-63a1effbe22f12235e8f9728f3d22005069f43e6) through run_fuzz_action in a temporary test. It errored on develop and returns Ok with this change. cargo test -p vortex-array --lib and cargo clippy -p vortex-array --all-targets --all-features -- -D warnings pass locally.

AI assistance

This PR was written by an AI coding agent working under my account (the code, the tests and this description), and I'm the responsible human for it. One thing for reviewers to check: I put the guard in FillNull::execute rather than changing slice to keep empty constants constant, since other code may rely on empty slices being canonical.

Filling nulls on a chunked array slices the constant fill to each chunk, and slicing to an empty range returns an empty canonical array rather than a constant. Executing fill_null on an empty chunk then failed with "fill_null fill_value must be a constant/scalar", which the array_ops fuzzer hit by filling a chunked array with empty chunks more than once. An empty input has no nulls to fill, so return an empty array of the result dtype before asking for a scalar fill. Closes vortex-data#10626 Signed-off-by: Vincent <1729065730@qq.com>

robert3005 · 2026-10-20T10:23:23Z

        let fill_value = args.get(1)?;
+        // An empty input has no nulls to fill. Slicing a constant fill to an empty range yields
+        // an empty canonical array rather than a constant, so return before requiring a scalar.
+        if input.is_empty() {


I think you still need to run the checks on run 143-150 - we should move them to the top

Moved the constant and non-null fill checks to the top in 25d87fe. The empty-input return now only runs when the fill isn't a constant, which is the empty slice of a constant fill that chunked fill_null hands to an empty chunk, so an empty input with a null constant fill errors again. (AI-assisted reply.)

robert3005

I think we can simplify this

robert3005 · 2026-10-20T10:23:48Z

+    #[test]
+    fn empty_input_ignores_non_constant_fill() -> VortexResult<()> {
+        let mut ctx = array_session().create_execution_ctx();
+        let input = PrimitiveArray::from_option_iter(Vec::<Option<i32>>::new()).into_array();
+        let fill = PrimitiveArray::from_iter(Vec::<i32>::new()).into_array();
+
+        let result = FillNull::try_new(input, fill)?
+            .into_array()
+            .execute::<ArrayRef>(&mut ctx)?;
+
+        assert_eq!(
+            result.dtype(),
+            &DType::Primitive(PType::I32, Nullability::NonNullable)
+        );
+        assert!(result.is_empty());
+        Ok(())
+    }
+
+    /// Filling a chunked array slices the constant fill per chunk, and an empty slice is a
+    /// canonical array rather than a constant. Stacked fills over an empty chunk must still run.
+    #[test]
+    fn repeated_fill_over_empty_chunk() -> VortexResult<()> {
+        let mut ctx = array_session().create_execution_ctx();
+        let input = ChunkedArray::try_new(
+            vec![
+                PrimitiveArray::from_option_iter(Vec::<Option<u16>>::new()).into_array(),
+                PrimitiveArray::from_option_iter([Some(7u16), None]).into_array(),
+            ],
+            DType::Primitive(PType::U16, Nullability::Nullable),
+        )?
+        .into_array();
+
+        let result = input
+            .fill_null(Scalar::primitive(1u16, Nullability::Nullable))?
+            .fill_null(Scalar::primitive(2u16, Nullability::NonNullable))?;
+
+        assert_arrays_eq!(result, PrimitiveArray::from_iter([7u16, 1]), &mut ctx);
+        Ok(())
+    }
+


do we need both of the tests?

No, I dropped empty_input_ignores_non_constant_fill and kept repeated_fill_over_empty_chunk, which is the case the fuzzer hit. (AI-assisted reply.)

Run the constant and non-null fill checks first, and only return an empty array for an empty input when the fill is not a constant (an empty slice of a constant fill). Drop empty_input_ignores_non_constant_fill, since repeated_fill_over_empty_chunk covers the case the fuzzer hit. Signed-off-by: Vincent <1729065730@qq.com>

codspeed · 2026-10-20T14:15:30Z

Merging this PR will degrade performance by 9.61%

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️

15 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

test_scan_filter[int_float]

random_i128[0.01]

decode_primitives[i64, (1000, 8)]

decompress[u32, (1000, 256)]

decompress[u16, (4000, 256)]

test_scan_filter[int_str_float]

test_scan_filter[int_str]

take_varbin

zstd_compress_string

filter_run_end[len=4096_run=16_density=0.1]

…and 5 more

ℹ️ 40 benchmarks have no flame graph

The flamegraph could not be generated because we failed to resolve the symbols in the profiling trace. This happens when we fail to harvest information about language frames.

test_scan_filter_to_arrow_array[int_float]

test_repeated_scan_filter[int_float]

test_repeated_scan_filter[int]

test_scan_scalar_at[int]

test_scan_filter_to_arrow_array[int]

test_scan_filter[int]

test_scan[int]

test_scan_read_all_to_arrow_array[int]

test_scan_filter[int_float]

test_repeated_scan_scalar_at[int]

…and 30 more

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 6 regressed benchmarks
✅ 2400 untouched benchmarks
⏩ 12 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
❌	WallTime	`filter_neon[8]`	918 ns	1,304 ns	-29.6%
❌	WallTime	`filter_neon[64]`	1.8 µs	2.2 µs	-18.26%
❌	WallTime	`filter_neon[80]`	2.5 µs	3 µs	-15.29%
❌ ℹ️	Simulation	`test_scan_filter_to_arrow_array[int_float]`	3.5 ms	4.1 ms	-13.75%
❌ ℹ️	Simulation	`test_repeated_scan_filter[int_float]`	3.4 ms	3.9 ms	-11.03%
❌ ℹ️	Simulation	`test_repeated_scan_filter[int]`	2.4 ms	2.7 ms	-10.35%
⚡	WallTime	`starts_with_avx512[8, VarBin]`	46.1 µs	39.2 µs	+17.4%
⚡ ℹ️	Simulation	`test_scan_scalar_at[int]`	888.6 µs	784.9 µs	+13.22%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing ump45nose:fix/fill-null-empty-input (25d87fe) with develop (c48457b)}

12 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

robert3005 reviewed Oct 20, 2026

View reviewed changes

robert3005 added the changelog/fix A bug fix label Oct 21, 2026

robert3005 approved these changes Oct 21, 2026

View reviewed changes

robert3005 enabled auto-merge (squash) October 21, 2026 16:35

robert3005 merged commit 633c52d into vortex-data:develop Oct 21, 2026
119 of 121 checks passed

Navigation Menu

Repository navigation

Return an empty array from fill_null on an empty input - #10627

Return an empty array from fill_null on an empty input#10627
robert3005 merged 2 commits into
vortex-data:developfrom
ump45nose:fix/fill-null-empty-input

ump45nose commented Oct 20, 2026

Uh oh!

robert3005 Oct 20, 2026

Uh oh!

ump45nose Oct 20, 2026

Uh oh!

robert3005 left a comment

Uh oh!

robert3005 Oct 20, 2026

Uh oh!

ump45nose Oct 20, 2026

Uh oh!

codspeed Bot commented Oct 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ump45nose commented Oct 20, 2026

Summary

Changes

AI assistance

Uh oh!

robert3005 Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

ump45nose Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

robert3005 left a comment

Choose a reason for hiding this comment

Uh oh!

robert3005 Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

ump45nose Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

codspeed Bot commented Oct 20, 2026

Merging this PR will degrade performance by 9.61%

Performance Changes

Footnotes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants