Skip to content

RDDT: Avoid materializing string views on group by - #29

Open
hicder-rddt wants to merge 1 commit into
reddit/v2.6.18from
hieu/string-group-by-view-optimization
Open

RDDT: Avoid materializing string views on group by#29
hicder-rddt wants to merge 1 commit into
reddit/v2.6.18from
hieu/string-group-by-view-optimization

Conversation

@hicder-rddt

@hicder-rddt hicder-rddt commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

3x reduction in both CPU and p99 latency for feeds-posts-3

One of the inefficiency I just found in Milvus: during group-by, let's say a segment has 1 million docs, and we need to get document 5, 10, 20. Currently, Milvus will:

  1. Construct a vector of 1 million elements, each with a "view" (offset, len) into the segment's data. Let's call this vector "string_views".
    a. On a high level, the way is does is: for 0 to 1M, string_views.push_back({offsets[i], len[i]})
  2. For each element (5, 10, 20), Milvus will read string_views[5], string_views[10], string_views[20] respectively, and return the corresponding documents.

From our flamegraph, step 1 takes 66% of the CPU! However, it's a waste! All we need to do is: read ({offsets[5], len[5]}, {offsets[10], len[10]}, {offsets[20], len[20]}), then return the corresponding documents.

Result: we managed to reduce CPU usage by 3x, and latency also reduced by 3x!
Screenshot 2026-07-28 at 3 40 04 PM

Screenshot 2026-07-28 at 3 41 04 PM Screenshot 2026-07-28 at 3 40 53 PM

@hicder-rddt
hicder-rddt force-pushed the hieu/string-group-by-view-optimization branch from d76bbcd to d65731b Compare August 1, 2026 14:24
@hicder-rddt
hicder-rddt changed the base branch from reddit/v2.6.18 to hieu/fix-cpp-tests August 1, 2026 14:25
@hicder-rddt
hicder-rddt force-pushed the hieu/string-group-by-view-optimization branch 2 times, most recently from 36d5d88 to 2a377e3 Compare August 1, 2026 15:52
@hicder-rddt
hicder-rddt marked this pull request as ready for review August 1, 2026 15:52
@hicder-rddt
hicder-rddt force-pushed the hieu/string-group-by-view-optimization branch from 2a377e3 to c17c326 Compare August 1, 2026 15:52
@hicder-rddt
hicder-rddt force-pushed the hieu/fix-cpp-tests branch 4 times, most recently from 948f770 to f02f254 Compare August 1, 2026 15:54
@hicder-rddt
hicder-rddt force-pushed the hieu/string-group-by-view-optimization branch from c17c326 to b0cbde7 Compare August 1, 2026 15:54
@hicder-rddt
hicder-rddt force-pushed the hieu/string-group-by-view-optimization branch from b0cbde7 to 9f1ea7c Compare August 1, 2026 15:56
@hicder-rddt
hicder-rddt changed the base branch from hieu/fix-cpp-tests to reddit/v2.6.18 August 4, 2026 12:31
@hicder-rddt
hicder-rddt force-pushed the hieu/string-group-by-view-optimization branch from 9f1ea7c to 9ede262 Compare August 4, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant