Skip to content

perf(compute): vectorize mixed 64-bit filters - #1311

Open
fallintoplace wants to merge 1 commit into
apache:mainfrom
fallintoplace:perf/compute-filter-uint64-compress
Open

perf(compute): vectorize mixed 64-bit filters#1311
fallintoplace wants to merge 1 commit into
apache:mainfrom
fallintoplace:perf/compute-filter-uint64-compress

Conversation

@fallintoplace

Copy link
Copy Markdown
Contributor

What changed

  • Add AVX2 and ARM64 NEON paths for null-free 64-bit fixed-width filters.
  • Compact two groups of four uint64 values from each filter byte with table-driven permutations and exact-size stores.
  • Keep the existing path for nulls, unsupported CPUs, short or unaligned inputs, dense masks, and noasm builds.
  • Add exhaustive mask tests, guard tests, and int64/float64 benchmarks.

Benchmark

Apple M1 Pro, macOS arm64, Go 1.26.3. Median of five runs. The baseline uses GODEBUG=cpu.all=off.

Case Baseline NEON Speedup
int64, 1M, alternating 3.97 ms 0.45 ms 8.7x
int64, 1M, random50 2.86 ms 1.61 ms 1.8x
float64, 1M, alternating 4.07 ms 0.47 ms 8.6x
float64, 1M, random50 2.92 ms 1.65 ms 1.8x

The mixed-mask cases also go from 33 to 32 allocations per operation. The output allocation is still the main B/op cost.

Tests

  • Full go test suite with Parquet test data
  • go test ./arrow/compute/...
  • go test -tags noasm ./arrow/compute/...
  • go test -race ./arrow/compute/internal/kernels ./arrow/compute
  • amd64 cross-build of the kernel package

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant