borg2: support zstd negative (--fast) compression levels

[ThomasWaldmann]

borg currently accepts zstd,L with 1 <= L <= 22 (parser: src/borg/helpers/parseformat.py:228, docs: src/borg/archiver/help_cmd.py:481). That covers zstd's positive range exactly, so nothing is missing at the high-ratio end.

What is missing are zstd's negative levels (--fast=N in the zstd CLI). The binding we use reports its bounds as (-131072, 22):

>>> zstd.CompressionParameter.compression_level.bounds()
(-131072, 22)

so roughly half of the library's usable range is unreachable from borg.

Why this is interesting

This came out of #9947 (lz4 levels). The conclusion there was: don't add lz4 levels, point people at zstd instead. But the part of zstd that actually competes with lz4 — borg's default compressor — is exactly the negative range we don't expose.

Benchmark, 256 MiB slice of a dev working tree (.py/.pyc/.json/.so mix), single-threaded, Apple M3 Pro, best of 3, via the CLI so numbers include pipe overhead:

codec	ratio	comp MB/s	decomp MB/s
lz4 -1 (borg default)	1.85	2207	3234
zstd --fast=10	1.72	1569	2843
zstd --fast=5	1.84	1385	2608
zstd --fast=3	1.93	1326	2619
zstd --fast=1	2.03	1233	2478
zstd -1	2.14	926	2134

--fast=1 stores ~10% less than lz4 -1 while still compressing at >1.2 GB/s, and --fast=5 matches lz4's ratio exactly. For users who picked lz4 because "compression must be nearly free" but are not actually saturating their storage or network, the --fast range is the useful knob — and today they have to jump all the way to zstd,1 (926 MB/s) or stay on lz4.

What would need to change

Three places, the third being the only real design question:

1. src/borg/helpers/parseformat.py:228 — the 1 <= level <= 22 range check.
2. src/borg/compress.pyx:60 — assert 0 <= level <= 255 in CompressorBase.__init__.
3. src/borg/repoobj.py:158 and src/borg/compress.pyx:604 — both build bytes((ctype, clevel)), i.e. clevel is packed into a single unsigned byte. A negative clevel raises ValueError there.

(3) looks worse than it is. meta["clevel"] itself is msgpack'd and handles negative ints fine; only the reconstruction into that 2-byte header breaks. And that header goes to Compressor.detect(), which matches solely on hdr[0] (the ctype ID) — the level byte is never used to select a decompressor. Decompression does not need it either: zstd frames are self-describing and ZSTD.decompress() ignores self.level. So clevel is purely informational.

That gives us at least two options:

• a) clamp negative levels to 255 ("unknown") in the byte representation, keeping the true value only in meta["clevel"];
• b) bias the stored byte (e.g. level + 128) so the full range round-trips through the header.

(a) is less invasive but loses the level for borg repo-compress / recreate recompression decisions; (b) keeps the information but changes the meaning of that byte, so it needs care w.r.t. existing repos and src/borg/archiver/transfer_cmd.py:104, which passes clevel through when copying objects between repos.

CLI syntax

zstd's own CLI spells these --fast=N rather than as negative levels. For borg, zstd,-1 … zstd,-10 seems most natural given the existing algo,L grammar, but note int(values[1]) already parses a leading minus, so only the range check rejects it today. Alternatively cap the accepted range at something sane like -10 rather than allowing the full -131072, since anything beyond roughly --fast=10 is not useful in practice.

Opinions on (a) vs (b), and on whether to expose this at all?