Repository navigation
feat(setagg)!: ASAPv1 payloads for SetAggregator and DeltaResult; drop the portable format - #177
Closed
GordonYuanyc wants to merge 2 commits into
Closed
GordonYuanyc wants to merge 2 commits into
GordonYuanyc wants to merge 2 commits into
Conversation
…ult; drop the portable format
SetAggregator (0x08 0x00) and DeltaResult (0x09 0x00) get ASAPv1 payloads
(spec §3.21, §3.22): metadata is {metadata_version: 1}; the payloads are
[values] and [added, removed], each an array of str in strictly ascending
UTF-8 byte order. DeltaResult refuses a key in both sets on both sides.
Both types move to sketches::set_aggregator with serialize_to_bytes /
deserialize_from_bytes; the crate-root re-exports are kept. The portable
modules, their MessagePackCodec impls and DeltaResult's serde derives are
removed. Adds four golden fixtures and bumps asapv1_golden.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…8, widen the string fixture Adds SetAggregator and DeltaResult to the Q-NOHASH and Q-ORDER decisions, requires every member to be valid UTF-8, states the encoder's one encoding per set, and numbers §3.22's rules in full. The DeltaResult doc states the disjointness serializing enforces, and read_canonical checks neighbours before collecting. set_aggregator_strings gains a 31-byte fixstr and a proper-prefix chain; asapv1_golden is bumped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the ASAPv1-only series: one PR per group, each off
main. SetAggregator (0x08 0x00) and DeltaResult (0x09 0x00) only had the portable msgpack format and their ASAPv1 payloads were "TBD". This PR designs them (spec §3.21, §3.22), moves both types onto ASAPv1, and removes the portable format.Draft for a decision on the payload design below.
Design (spec §3.21 / §3.22)
{metadata_version: 1}only: no hash-spec group (nothing hashed reaches the wire) and nokey_type(the kind_id fixes the element type asstr, as with Coco).[values], DeltaResult[added, removed]. Empty:[[]]/[[], []].strand valid UTF-8;binis rejected.addedandremovedmust be disjoint, enforced on encode and decode.Code
sketches::set_aggregatorwith awire.rs(crate-rootasap_sketchlib::{SetAggregator, DeltaResult}still resolve). In-memory API unchanged; newserialize_to_bytes/deserialize_from_bytes.message_pack_format::portable::{set_aggregator, delta_set_aggregator}, theirMessagePackCodecimpls, andDeltaResult's serde derives (the serde form was the removed wire format).set_aggregator_strings(empty string, proper-prefix chain, 31/32-byte fixstr/str8 boundary, 2/3/4-byte UTF-8),set_aggregator_empty,delta_result_strings,delta_result_empty. Hand-decoded by the reviewer; checked in both directions.Decisions for you (reviewer recommended keeping each as is)
key_type; reject vs canonicalize out-of-order input; enforce disjointness; dropDeltaResult's serde derives (public API break); two kind_ids; module location undersketches/.Downstream
ASAPQuery's
delta_set_aggregator_accumulator.rscallsDeltaResult::to_msgpack(); it must switch toserialize_to_bytes(), its sink bytes change format, and a DeltaResult with a key in both sets now fails to encode. sketchlib-go has no implementation yet.Verification (rustc 1.99)
cargo fmt --check; clippy-D warningsfor all four CI feature sets;RUSTDOCFLAGS="-D warnings" cargo doc; gen-proto drift clean;--lib -- set_aggregator,asapv1_golden,pbt -- set_aggregator::,e2e -- set_aggregator,msgpack_compat. Wire code mutation-checked. Implemented and reviewed by separate agents; review findings are addressed in the second commit.🤖 Generated with Claude Code