feat: external IVF-PQ vector index over parquet (Phase 1) — RFC draft - #1

Draft

sezruby wants to merge 23 commits into

mainfrom

external-index-rfc-draft

Commits on May 19, 2026

fix(mem_wal): allow append-only tables without primary keys (#6848)
touch-of-grey
authored
93fdd5d
feat: add manifest version hint for fast latest-version lookup (#6752)

touch-of-grey
and
jackye1995
authored
dd887ec
chore: release beta version 7.0.0-beta.16
Lance Release Bot
committed
98e2b8a
fix: make HNSW graph build deterministic to stabilize test_ann_prefilter (#6818)

authored
64d7b78
fix: set _row_created_at_version to new version for MERGE INTO INSERT rows (#6774)

jerryjch
and
Jing chen He
authored
29a8f92

Commits on May 20, 2026

fix: branch_identfier unstable for legacy branches (#6390)
majin1102
authored
742e6a3
docs: document PR publishing requirements (#6870)
Xuanwo
authored
79b363a
feat(index): serializable cache for the BTree scalar index (#6793)

wjones127
and
claude
authored
1607010
feat: implement vector index details (#6099)

wjones127
and
claude
authored
6ddd7e2
feat(index): serializable cache for Bitmap and LabelList scalar indices (#6874)
wjones127
authored
4de5ce6
feat(mem_wal): cache opened L0 flushed-generation datasets (#6816)

hamersaw
and
claude
authored
e808eb1
feat: add MemWAL sharding evaluator (#6854)
jackye1995
authored
bf68171
chore: release beta version 7.0.0-beta.17
Lance Release Bot
committed
c7c5626

Commits on May 21, 2026

feat: expose granular trace event targets (#6853)

beinan
and
Beinan Wang
authored
dbeeefb
fix(mem_wal): exact PK dedup for LSM vector search (#6881)

touch-of-grey
and
jackye1995
authored
65a0e41
fix(mem_wal): dedupe duplicate primary keys in LSM point lookup (#6880)

touch-of-grey
and
jackye1995
authored
bd58ad0
feat: expose multi-base config to Python and Java write_fragments API (#6855)
zhangyue19921010
authored
6e0e5a6
fix: stop double-counting child CPU in node-with-children Exec plans (#6799)
brendanclement
authored
c39f31e
feat: create materialized view API (#6891)

rpgreen
and
claude
authored
95613a7
chore: bump main to 7.1.0-beta.0
Lance Release Bot
committed
0f13993
perf: use roaring's range iter to speedup mask_to_offset_ranges (#6871)

westonpace
and
claude
authored
3d85fc7
chore: release beta version 7.1.0-beta.1
Lance Release Bot
committed
cffa8cb

Commits on May 22, 2026

feat: external IVF-PQ vector index over caller-supplied parquet files
sezruby
committed
d6e2915