feat(lakebase): add Lakebase Vector backend

[junyulabs]

Summary

This PR adds Lakebase Vector backend to VectorDBBench.

Usage

To run the Performance768D100M workload on Neon:

1. Create a Neon instance with 40 CUs and wait until it is ready.
2. Enable the Lakebase Vector extension by https://neon.com/docs/extensions/lakebase-vector
3. Install and run VectorDBBench

export POSTGRES_HOST="<POSTGRES_HOST>"
export POSTGRES_PASSWORD="<POSTGRES_PASSWORD>"
export NUM_PER_BATCH=10000

vectordbbench lakebaseann \
  --host $POSTGRES_HOST \
  --user-name neondb_owner \
  --db-name neondb \
  --table-name vdbbench_laion100m \
  --db-label lakebase-search-laion100m \
  --task-label lakebase-search-laion100m \
  --case-type Performance768D100M \
  --k 100 \
  --load-concurrency 1 \
  --num-concurrency 1,5,10,20,30,40,60,80,100,120 \
  --concurrency-duration 30 \
  --concurrency-timeout 3600 \
  --max-parallel-workers 95 \
  --drop-old \
  --load \
  --search-concurrent \
  --search-serial \
  --probes 54,380 \
  --epsilon 3.2

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: junyulabs
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[junyulabs]

/assign @XuanYang-cn

[yhmo]

vectordb_bench/backend/clients/lakebase_vector/config.py line:89
Medium ---- The --probes value is forwarded verbatim (e.g. "54,380" in the PR usage example, or "10,20" in test_session_guc) into a single SET "lakebase_ann.probes" = "54,380";. The linked lakebase_vector documentation (neon.com/docs/extensions/lakebase-vector) declares this GUC as an integer ("Number of IVF partitions to scan at query time", example SET lakebase_ann.probes TO '10'). If the server parses it as a plain integer, every init()/session would fail before any search runs. The CLI/frontend help advertises "Comma-separated lakebase_ann probe counts", yet nothing in the code splits or iterates that list - only one SET is emitted. Could you confirm whether the server actually accepts a comma-separated value and, if so, how the single value is meant to apply across the concurrent search runs? If multiple probe values were intended, the framework has no expansion for them; otherwise an int with client-side validation would match the documented contract. It would also help to cover this in a test that asserts what the server accepts, not just the rendered SET SQL.

[yhmo]

vectordb_bench/backend/clients/lakebase_vector/lakebase_vector.py line:279
Medium ---- On a COPY failure this returns (0, e) without rolling back the connection. Since _create_connection sets autocommit=False, a failed COPY statement marks the whole transaction aborted; the load runner then retries the batch up to MAX_INSERT_RETRY (5) times on that same connection (ConcurrentInsertRunner._insert_batch_with_retry), and every retry fails with "current transaction is aborted, commands ignored until end of transaction block", so one bad batch fails the entire load and masks the original error. A single self.conn.rollback() in this except handler would keep the connection retryable, and a test asserting insert_embeddings returns (0, e) then a subsequent call succeeds would lock the behavior in.

[yhmo]

vectordb_bench/backend/clients/lakebase_vector/lakebase_vector.py line:269
Low ---- copy.set_types(...) is re-invoked for every row inside the loop. psycopg Copy.set_types() re-registers the per-column dumpers via transformer.set_dumper_types() on each call, so for the 100M-row Performance768D100M load this is avoidable per-row work in the hottest copy path. The type set only depends on with_scalar_labels, so a single set_types(...) call before the loop would be equivalent. Note test_insert currently asserts set_types.call_count == 2 for two rows, which locks in the per-row behavior and would need updating if the call is hoisted.

[yhmo]

vectordb_bench/backend/clients/lakebase_vector/config.py line:27
Low ---- The earlier thread on cli.py --db-name was resolved by making the flag required, but this config-class default db_name: str = "databricks_postgres" was left in place. It is unrelated to Lakebase/Neon, and the non-CLI path still uses it: dbConfigSetting builds the frontend form from the model schema default, so a frontend user who leaves db_name untouched will silently target a nonexistent database and the run fails at connect time. Suggest a neutral default (e.g. "postgres") or no default, consistent with vectorchord ("vectordb") and alloydb (required).