Skip to content

[core] Push down bitmap indexes for data evolution merged reads - #10289

Open
lilei1128 wants to merge 1 commit into
apache:masterfrom
lilei1128:pushdow-bitmap-for-de
Open

lilei1128 wants to merge 1 commit into
apache:masterfrom
lilei1128:pushdow-bitmap-for-de

Conversation

@lilei1128

Copy link
Copy Markdown
Contributor

Purpose

Enable row-level bitmap file-index pushdown for data evolution merged
file groups.

Previously, bitmap indexes could only skip an entire merged group. A
non-empty bitmap could not filter individual rows because applying it to
only one field reader would break positional alignment.

This change builds a shared bitmap selection from eligible file-index
results and applies it to every aligned reader in the merged group. The
optimization is limited to normal data files with identical row-id
ranges. Blob files, vector-store files, and non-bitmap index results
continue to use the conservative fallback path.

Tests

  • Added a test verifying row-level selection with a bitmap index in a
    data evolution merged group.
  • Added a test verifying intersection of bitmap selections from multiple
    field files.
  • Added a test verifying OR predicate semantics for bitmap selection.
  • Updated the existing deletion-vector test to verify that the shared
    bitmap selection returns live matching rows while deleted matching
    groups are skipped.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant