extract: avoid refetching/reparsing repeated chunks, fixes #1678

[ThomasWaldmann]

Extracting a sparse file repeatedly fetched and decrypted the same all-zero chunk, once per occurrence in the item's chunk list (see #1678, from 2016). A 258 MiB file with a 256 MiB hole fetched and parsed (decrypt + authenticate + decompress) the same chunk 63 times.

Two changes to DownloadPipeline.fetch_many:

1. Small LRU cache of recently parsed chunks (parsed_cache, 4 entries, keyed (id, ro_type))

Repeated chunks are parsed only once; repetitions are served from the cache. The repository is still asked for every occurrence — for local repos that is a cheap slice from the cached pack, and it keeps the request stream to legacy remote repositories (pipelined get_many during borg transfer --from-borg1) unchanged. This helps any kind of repeated chunk, not just zeros.

2. Serve all-zero chunks directly from the zeros constant, without repository access

A chunk is recognized as all-zero by comparing its id against the (memoized) id of an all-zero chunk of the same size. The memo is the already existing zero_chunk_ids mapping, now shared with the create side via the new zero_chunk_id() helper (refactored out of cached_hash). To bound the memoized id computations to a few chunk sizes, they are only done for ids occurring repeatedly within the requested stream — a repeated id means repeating plaintext, which usually is a run of zeros. Unique ids are only compared against already-memoized sizes (cheap dict lookup).

Effect (instrumented extract --sparse of the 256 MiB-hole file, aes256-ocb repo): the all-zero chunk went from 63 repository requests + 63 parses to 0 requests + 0 parses; total get_many requests for the extract dropped from 68 to 5. Extracted content verified byte-identical, holes still punched.

Notes:

• The missing-chunk / replacement-chunk error path is unchanged and uncached, so problems are still reported per occurrence.
• The zeros shortcut only applies where chunk sizes are known (ChunkListEntry lists), so the archive metadata stream path (unpack_many) is unaffected.
• New unit tests cover both behaviors, including that all-zero chunks are served without any repository request.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.85%. Comparing base (28de45c) to head (c76f2ce).
⚠️ Report is 4 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9975      +/-   ##
==========================================
+ Coverage   85.83%   85.85%   +0.02%     
==========================================
  Files          95       95              
  Lines       17034    17059      +25     
  Branches     2607     2611       +4     
==========================================
+ Hits        14621    14646      +25     
  Misses       1673     1673              
  Partials      740      740              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[ThomasWaldmann]

Benchmark: extract --sparse of a 50 GiB all-hole sparse file (aes256-ocb, local repo, Apple Silicon mac, two runs each):

	wall time	user CPU	sys CPU
master (before)	29.38 s / 29.24 s	~25.0 s	~4.2 s
this PR (after)	4.09 s / 4.01 s	~2.5 s	~1.4 s

~7.3x faster wall clock, ~10x less user CPU. The 50 GiB file is one giant hole, archived with create --sparse (repo: 36 kB, one ~4 MiB zero chunk referenced ~12,800 times). Both versions extract byte-identical, fully sparse output (0 bytes allocated).

Before, the ~25 s of user CPU is spent decrypting and lz4-decompressing the same zero chunk ~12,800 times (about 50 GiB of decompression output). After, those chunks involve no repository access and no parsing; the remaining ~4 s is mostly interpreter/repo-open startup plus extract's per-chunk zeros.startswith(data) check and hole seeking.