[lohmag]

With MYSQLI_CLIENT_COMPRESS, mysqlnd runs compress() on every packet it sends, however small. Each call sets up a new deflate stream, and short packets do not shrink, so the client pays the full setup cost for nothing.

This sends packets smaller than 170 bytes uncompressed inside the compressed envelope (uncompressed length 0), which the protocol already allows and which mysqlnd already does for packets that do not shrink. libmysqlclient uses the same mechanism with a 50 byte threshold.

compress() at level 6 on single statements of the kinds PHP applications send, 200 of each with random values, packet header included, zlib 1.3, one core:

Statement	Packet	compress()	Saved
COMMIT, SET NAMES utf8mb4	22 B	1.8 µs	nothing
COM_STMT_EXECUTE with 1 parameter	26 B	2.0 µs	nothing
SELECT * FROM users WHERE id = ...	43 B	2.4 µs	nothing
WordPress wp_options autoload	104 B	3.1 µs	15 B (14%)
COM_STMT_EXECUTE with 11 parameters	172 B	7.0 µs	36 B (21%)
UPDATE of 4 columns by id	172 B	5.5 µs	39 B (23%)
Eloquent select ... order by ... limit	177 B	5.1 µs	53 B (30%)
Hibernate select by id	232 B	5.7 µs	89 B (38%)
INSERT of one row with 11 columns	348 B	10.1 µs	103 B (29%)
WordPress WP_Query	387 B	7.8 µs	155 B (40%)

Nothing under 50 bytes shrinks. From about 170 bytes typical queries save 20% or more, which is where the cost starts to pay off.

The test uses the fake server to read what the client sends and checks the uncompressed length in the compressed header: 0 for a 94 byte query and the real length for a 1014 byte one. Without the change the 94 byte query is sent compressed and the test fails.

[Sjord]

Looks good to me.

MySQL 8.0.46 on the same host

I think it would be interesting to get performance information when the connection goes over the internet. Compression would be more benificial on slow links, but I guess it would not make much of a difference as long as the query fits in one packet.

[kamil-tekiela]

I am in favor of this change and it makes sense to me. But I would like to see some justification for 512. Why is it 512 and not 50? Or maybe something in the middle? How much compression can we get by compressing a 512-byte-long string? Why is it even a power of 2? I would rather keep it at 50, just like libmysqlclient does or at most increase it to 100 bytes.

Regarding the new test: it just looks like AI slop. I don't understand what exactly it tests or its relation to the change in this PR. It doesn't seem to verify that compression is happening. I think you should write the test yourself, or, if there is one, adjust an existing test to cover the branch change.

[lohmag]

There is no good reason for 512, and it being a power of 2 means nothing. I measured it on single statements of the kinds PHP applications send. I used 200 of each with random values and compressed them with compress() at level 6. Nothing under 50 bytes shrinks, so mysqlnd already sends these packets uncompressed and only wastes the compress() call. I think 170 is the limit where it starts paying off.

Statement	Packet	compress()	Saved
COMMIT, SET NAMES utf8mb4	22 B	1.8 µs	nothing
COM_STMT_EXECUTE with 1 parameter	26 B	2.0 µs	nothing
SELECT * FROM users WHERE id = ...	43 B	2.4 µs	nothing
WordPress wp_options autoload	104 B	3.1 µs	15 B (14%)
COM_STMT_EXECUTE with 11 parameters	172 B	7.0 µs	36 B (21%)
UPDATE of 4 columns by id	172 B	5.5 µs	39 B (23%)
Eloquent select ... order by ... limit	177 B	5.1 µs	53 B (30%)
Hibernate select by id	232 B	5.7 µs	89 B (38%)
INSERT of one row with 11 columns	348 B	10.1 µs	103 B (29%)
WordPress WP_Query	387 B	7.8 µs	155 B (40%)

[kamil-tekiela]

There is no good reason for 512, and it being a power of 2 means nothing. I measured it on single statements of the kinds PHP applications send. I used 200 of each with random values and compressed them with compress() at level 6. Nothing under 50 bytes shrinks, so mysqlnd already sends these packets uncompressed and only wastes the compress() call. I think 170 is the limit where it starts paying off.

Statement Packet compress() Saved
COMMIT, SET NAMES utf8mb4 22 B 1.8 µs nothing
COM_STMT_EXECUTE with 1 parameter 26 B 2.0 µs nothing
SELECT * FROM users WHERE id = ... 43 B 2.4 µs nothing
WordPress wp_options autoload 104 B 3.1 µs 15 B (14%)
COM_STMT_EXECUTE with 11 parameters 172 B 7.0 µs 36 B (21%)
UPDATE of 4 columns by id 172 B 5.5 µs 39 B (23%)
Eloquent select ... order by ... limit 177 B 5.1 µs 53 B (30%)
Hibernate select by id 232 B 5.7 µs 89 B (38%)
INSERT of one row with 11 columns 348 B 10.1 µs 103 B (29%)
WordPress WP_Query 387 B 7.8 µs 155 B (40%)

Yes, Claude gave me an almost identical table. From it, I read that 100 is the most that we can apply because after 100 bytes we already start to see some significant compression. Claude does not seem to understand that we are not trying to optimise the performance of the local CPU but the network instead. I think that keeping it at 50 bytes is the safest option.

[lohmag]

There is no good reason for 512, and it being a power of 2 means nothing. I measured it on single statements of the kinds PHP applications send. I used 200 of each with random values and compressed them with compress() at level 6. Nothing under 50 bytes shrinks, so mysqlnd already sends these packets uncompressed and only wastes the compress() call. I think 170 is the limit where it starts paying off.
Statement Packet compress() Saved
COMMIT, SET NAMES utf8mb4 22 B 1.8 µs nothing
COM_STMT_EXECUTE with 1 parameter 26 B 2.0 µs nothing
SELECT * FROM users WHERE id = ... 43 B 2.4 µs nothing
WordPress wp_options autoload 104 B 3.1 µs 15 B (14%)
COM_STMT_EXECUTE with 11 parameters 172 B 7.0 µs 36 B (21%)
UPDATE of 4 columns by id 172 B 5.5 µs 39 B (23%)
Eloquent select ... order by ... limit 177 B 5.1 µs 53 B (30%)
Hibernate select by id 232 B 5.7 µs 89 B (38%)
INSERT of one row with 11 columns 348 B 10.1 µs 103 B (29%)
WordPress WP_Query 387 B 7.8 µs 155 B (40%)

Yes, Claude gave me an almost identical table. From it, I read that 100 is the most that we can apply because after 100 bytes we already start to see some significant compression. Claude does not seem to understand that we are not trying to optimise the performance of the local CPU but the network instead. I think that keeping it at 50 bytes is the safest option.

A new test of 8,665 packets from 0 to 700B shows packets of 50–99 B saving 1.7 B on average (median 0). From 100–149 B they save 9% and from 150 B up to 18-28%. So any threshold less 100 makes little sense.

[Sjord]

More bytes do not necessarily mean less performance. My understanding is that it makes little difference as long as it fits in a single network packet.

Is performance the only consideration here? I can imagine users that pay per byte for their bandwidth and want to enable compression to reduce bandwidth costs?

[lohmag]

We had a case: a high-throughput service whose link to the database was at 1 Gbit, and we had to turn compression on to fit the traffic. But at thousands of requests per second the client CPU matters too, so every compress() call that saves nothing on the wire is pure cost.

[kamil-tekiela]

In that case, let's settle on 100 bytes.