Add random-access decryption to Cobblestone - #15322
Open
reaperhulk wants to merge 6 commits into
Open
Conversation
Extend the existing Cobblestone128/256 decryptors with a `decrypt_range` method for authenticated random access into a message, without a new class. The C2SP chunked-encryption wire format is already random-access by construction (each 16 KiB chunk is independently AEAD-sealed with the chunk index bound into the nonce), so this adds only decryption capability; the encryptors and wire format are unchanged. `decrypt_range(source, offset, length)` reads only the chunks covering the requested plaintext range. A `source` may be a buffer-protocol object (e.g. an mmap of a file, for local zero-RAM access) or a binary file-like object exposing just `seek` and `read` (e.g. a small adapter issuing ranged requests to remote/object storage) — one method covers both use cases. To never return unauthenticated bytes, the requested range is silently widened to whole chunk boundaries, every covering chunk is decrypted and authenticated, and only then is the requested sub-range sliced out. A range that cannot be fully authenticated (tampering, or truncation within the range) raises InvalidTag. The commitment is verified once, before any plaintext is released. Streaming (`update`/`finalize`) and random-access decryption are mutually exclusive on a single instance. A range read authenticates the bytes it returns but not the message as a whole, so callers must obtain the total length from a trusted source; this caveat is documented. Rust changes: a stateless `ChunkNonces::nonce_for(index)` (shared with the streaming `next`), a `ChunkCipher::decrypt_chunk_at`, and the `decrypt_range` implementation with buffer/file-like reads. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
The docs spell-check (sphinx-build -b spelling, warnings-as-errors) only knew the capitalized class-name form "Decryptor"; the new prose in CHANGELOG.rst and docs/cobblestone.rst uses "decryptor" as a common noun. Add the lowercase form to the wordlist. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
The coverage job flagged five uncovered lines in the new random-access code. Address them: - Remove a now-dead bounds check: the `checked_add(...).filter(...)` guard on the requested range already ensures every chunk index is below MAX_CHUNK_COUNT, so the subsequent `last_chunk >= MAX_CHUNK_COUNT` branch was unreachable. - Add tests exercising the remaining edge branches: a file-like source that signals EOF with None, `update_into` after seeking (mixing guard), a source too short to hold the header, and a final wire chunk shorter than the tag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
Fixes an E501 (line too long) flagged by ruff in a test added in the previous commit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
alex
reviewed
Jul 25, 2026
Reworks the random-access API in response to review. decrypt_range is no longer a method on the streaming decryptors. It is the only instance method on new Cobblestone128RangeDecryptor and Cobblestone256RangeDecryptor classes, so streaming and random access are separate objects that cannot be interleaved. The streaming ChunkedDecryptor is left exactly as it was. Ciphertext is now read through a single RangeReader protocol — read_at(offset, length) — rather than accepting either a buffer or a seek()/read() file-like. Reads are positional, so a reader holds no cursor and can serve concurrent ranges. Two implementations are provided: BufferReader, for any bytes-like (including an mmap of a file), and FileReader, which uses os.pread where the platform has it and otherwise falls back to a locked seek/read. Implementing the protocol for remote storage is documented with an object-storage example. Because a reader is supplied per call, the message header is re-read on each call and the derived keys are cached against it, so one decryptor can safely be used with readers for different messages. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
Two problems, both caught only on 3.9 and older targets: - tests/test_cobblestone.py used PEP 604 union syntax (int | None) in an annotation without `from __future__ import annotations`. On 3.9, annotations are evaluated at runtime, so importing the module raised TypeError and the whole file failed to collect. Add the future import. - BufferReader's parameter was annotated as Buffer, which names only bytes, bytearray and memoryview. mmap.mmap supports the buffer protocol and is documented as usable here, but is not one of those types, so passing one was a type error. Widen the annotation to include it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
decrypt_range()method for Cobblestone128Decryptor and Cobblestone256Decryptor, enabling efficient decryption of arbitrary byte ranges from large messages without processing the entire ciphertext.Summary
This change adds random-access decryption capabilities to the Cobblestone chunked encryption implementation. The new
decrypt_range()method allows decrypting a specific range of plaintext bytes by reading and authenticating only the 16 KiB chunks that cover that range, making it practical to work with large encrypted files without loading them entirely into memory.Key Changes
New
decrypt_range()method: Added to bothCobblestone128DecryptorandCobblestone256Decryptorclasses. Takes a source (buffer or file-like object), offset, and length; returns authenticated plaintext for the requested range.Helper functions and utilities:
read_at(): Reads bytes from either buffer-protocol objects (direct indexing) or file-like objects (via seek/read)mixing_error()andtruncated_error(): Error helpers for invalid usage patterns and truncation detectionChunkCipher::decrypt_chunk_at(): Decrypts a single chunk at an explicit index without advancing the sequential counterChunkNonces::nonce_for(): Stateless nonce derivation for arbitrary chunk indicesState tracking: Added fields to
ChunkedDecryptorto:decrypt_range()callupdate/finalize) or seeking (decrypt_range) has been used to enforce mutual exclusivityMutual exclusivity enforcement: Streaming and random-access decryption cannot be mixed on a single instance; attempting to do so raises a
ValueError.Comprehensive test coverage: Added 20+ test cases covering:
Documentation: Added detailed docstrings, type hints in
.pyifile, and comprehensive user documentation with examples showing both buffer and file-like usage patterns, including a remote range-fetch adapter example.Implementation Details
decrypt_range()call and cached for subsequent calls.seek()andread()for remote or streaming sources.https://claude.ai/code/session_0172rPrpx6h454zpXTGdzez6