You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Move summary kernel serialization formats out of Planner #471
While splitting #462, we set the boundary that ASAPPlanner should not contain SDS or deployment-specific storage/wire formats. Each deployment picks its own formats. #462 already moves these back to ASAPQuery-backend:
types/post_asap/semantic_definition.rs (v1 and v2)
stored_state/{delta_apply,decoders,native}.rs
the encode/decode version envelopes of CompiledPhysicalDag and PhysicalCandidate
Serialization inside the summary kernels was deliberately left out of #462 to keep the split scope bounded. This issue tracks it.
Scope
In crates/asap-physical-operators on the #462 branch:
SummaryKernel in summary_kernels/traits.rs requires serialize_to_json and serialize_to_bytes.
Every sketch kernel has its own msgpack/proto encode/decode, e.g. count_min_sketch.rs, count_sketch.rs, dd_sketch.rs, hll_sketch.rs, datasketches_kll.rs, *_with_heap.rs.
summary_kernels/sketch_envelope.rs and the asap_sketch_codec crate.
serialize_to_bytes / deserialize_from_bytes on KeyByLabelValues and Measurement.
This code was moved from ASAPQuery-backend/data_plane/src/precompute_engine/operators/*_accumulator.rs with only minor changes.
Observation
On the #462 branch, apart from summary_kernels/ itself and stored_state/ (which is moving back to Backend), no Planner code calls these serializers. Build/merge/readout work on in-memory state. So once stored_state moves back, the byte formats are probably used only by Backend.
Questions
Can the serialization methods be dropped from SummaryKernel, keeping only the in-memory state API (build/merge/readout)?
Should msgpack/proto encoding and asap_sketch_codec move to Backend, next to the SDS wire decoding in sketch_db?
Does Backend need an accessor to read or rebuild kernel state (for example, exposing the underlying asap_sketchlib structure)? The goal is to avoid adding a new abstraction layer.
Non-goals
Changing sketch algorithms, which stay in asap_sketchlib.
Activity