perf(arrow-ipc): Add writer benchmarks for dictionaries

[JakeDern]

Which issue does this PR close?

• Closes arrow-ipc: Extend writer benchmarks to include dictionaries #10119

Rationale for this change

This PR adds writer benchmarks for dictionaries so that we can measure the performance impact of code changes on those code paths.

What changes are included in this PR?

Three new benchmarks:

• StreamWriter benchmark for dictionaries
• StreamWriter benchmark for delta dictionaries
• FileWriter benchmark for delta dictionaries

Are these changes tested?

Yes, just benchmarks included which I ran locally.

Are there any user-facing changes?

No.

[JakeDern]

CC: @alamb @Rich-T-kid

[Rich-T-kid]

taking a look 👀

[Rich-T-kid]

@JakeDern the PR looks good to me. I have some comments but they arent blocking. 🚀 nice work

[Rich-T-kid]

For an 8k batch size there will be 4k unique values, and for 64k there will be 32k unique values. I don't think this alone is the right way to benchmark dictionaries. Dictionaries are focused on low cardinality, so it makes more sense to parameterize benchmarks by target cardinality. For example, (5%, 10%, 25%, 50%) unique values relative to batch size.
The implementation should also grow with the number of unique values, since that's the point of the IPC format in delta mode:

A dictionary batch with isDelta set indicates that its vector should be concatenated with those of any previous batches with the same id.

Varying cardinality helps detect whether the encoder is doing O(N) work (proportional to total rows) when it should be doing O(K) work (proportional to unique values).

[JakeDern]

Thank you for the thoughtful review!

Varying cardinality helps detect whether the encoder is doing O(N) work (proportional to total rows) when it should be doing O(K) work (proportional to unique values).

I would clarify that we are doing O(K + N) work as there is still per row overhead in encoding the dictionary keys as well as per unique value cost in encoding the dictionary values.

I agree that benchmarks varying the amount of unique values could yield useful information, but these simple benchmarks can still answer whether we've meaningfully changed the amount of work required to emit dictionary batches.

I think until those additional parameters are needed to demonstrate something the current set cannot, I favor the simplicity of omitting them.

Just my .02 of course, happy to take more feedback on this.

[Rich-T-kid]

makes sense to me.

[Rich-T-kid]

Similar comment to before.

[Rich-T-kid]

a possible minor/medium optimization for the uncompressed case came to mind & I remembered the current benchmarks dont cover this.
@JakeDern what do you think about adding an identical case to

group.bench_function("StreamWriter/write_10/zstd", |b| {
        let batch = create_batch(8192, true);
        let mut buffer = Vec::with_capacity(2 * 1024 * 1024);
        b.iter(move || {
            buffer.clear();
            let options = IpcWriteOptions::default()
                .try_with_compression(Some(CompressionType::ZSTD))
                .unwrap();
            let mut writer =
                StreamWriter::try_new_with_options(&mut buffer, batch.schema().as_ref(), options)
                    .unwrap();
            for _ in 0..10 {
                writer.write(&batch).unwrap();
            }
            writer.finish().unwrap();
        })
    });

but with try_with_compression set to None. It may make sense to actually refactor the StreamWriter benchmarks to iterate over the possible compression types.
I would add it but that would come with possible merge conflicts with this change. If not thats fine I can submit a PR.

[JakeDern]

a possible minor/medium optimization for the uncompressed case came to mind & I remembered the current benchmarks dont cover this. @JakeDern what do you think about adding an identical case to

group.bench_function("StreamWriter/write_10/zstd", |b| {
        let batch = create_batch(8192, true);
        let mut buffer = Vec::with_capacity(2 * 1024 * 1024);
        b.iter(move || {
            buffer.clear();
            let options = IpcWriteOptions::default()
                .try_with_compression(Some(CompressionType::ZSTD))
                .unwrap();
            let mut writer =
                StreamWriter::try_new_with_options(&mut buffer, batch.schema().as_ref(), options)
                    .unwrap();
            for _ in 0..10 {
                writer.write(&batch).unwrap();
            }
            writer.finish().unwrap();
        })
    });

but with try_with_compression set to None. It may make sense to actually refactor the StreamWriter benchmarks to iterate over the possible compression types. I would add it but that would come with possible merge conflicts with this change. If not thats fine I can submit a PR.

I think this benchmark is covering that case:

arrow-rs/arrow-ipc/benches/ipc_writer.rs

Lines 29 to 41 in c4a831a

	group.bench_function("StreamWriter/write_10", |b| {
	let batch = create_batch(8192, true);
	let mut buffer = Vec::with_capacity(2 * 1024 * 1024);
	b.iter(move || {
	buffer.clear();
	let mut writer = StreamWriter::try_new(&mut buffer, batch.schema().as_ref()).unwrap();
	for _ in 0..10 {
	writer.write(&batch).unwrap();
	}
	writer.finish().unwrap();
	})
	});

The default StreamWriter::try_new uses the default IpcWriteOptions which sets batch_compression_type: None. Let me know if I'm misunderstanding though.

[Rich-T-kid]

yea your right, source.