Repository navigation
Introduce kind: text requests for BM25 over a text index - #185
Merged
Merged
Conversation
Qdrant 1.19.3-dev ranks points by BM25 over a payload field's text index:
`Query.text` carries `TextQuery { query, k, b }`, `using` names the field,
and the field's text index needs `TextIndexParams.scoring`. This builds
bfb against qdrant/rust-client#299, which regenerates the client from
that proto (pinned to its commit until it merges).
Search config:
- `kind: text` with `using` (the payload field), optional `k` and `b`,
`filters`, and a query `source`: `random` words drawn from the same
`word_<id>` vocabulary text payloads are generated from, or `file`
with one query per line (`from-start` or `random-sample`).
- `--search-quality` and `--prefetch` are refused with a text request:
BM25 over a text index is exact, and it has no vector to rescore.
Upload config, `type: text` fields:
- `scoring: bm25`, plus the tokenizer options `lowercase`, `stopwords`
and `stemmer` (Snowball), all refused on other field types.
The request builder takes the query as one `MainQuery` (a vector, or a
text query) instead of a vector alone. New examples `upload-bm25-text`
and `search-bm25-text`, and a README section.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Member
Author
|
End-to-end check against Qdrant
The 🤖 Generated with Claude Code |
generall
approved these changes
Oct 7, 2026
qdrant/rust-client#299 merged as 71143d8, whose tree is identical to the d960517 this branch was built and tested against. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
agourlay
marked this pull request as ready for review
October 7, 2026 12:15
agourlay
added a commit
to agourlay/strawmann
that referenced
this pull request
Oct 7, 2026
W15 needs #185's `kind: text` requests and text index scoring options. Read for what it changes in a request §4's other rows send: the dense request is built from the same VectorInput through the new MainQuery, text index options are sent only when a field sets them, and the client moves to rust-client 71143d8, which adds proto fields and changes none a dense request sets. bfb_pin is a stamp key, so pairs measured from here are STALE against earlier ones, as the next pairs already are for the client change. The pinned worktree is checked out at caaeeef and built. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds BM25 over a payload field's text index to the config path, the
Query.textvariant Qdrantdevadded for 1.19.3 (qdrant/qdrant#10786 and the series before it).Builds against qdrant/rust-client's
devbranch, which carries the new API since qdrant/rust-client#299 merged (71143d8).Search config:
kind: textrandomdrawsword_<id>words, the same vocabulary text payloads are generated from, so the queries hit the corpus.filereads one query per line (path,strategy: from-start | random-sample), loaded once at startup. Blank lines are skipped, and an empty file fails at startup.usingnames the payload field and is required.kmust be >= 0 andbwithin [0, 1], the same bounds the server validates.--search-qualityis refused with a text request, because BM25 over a text index is exact and there's no exact reference to compare against.--prefetchis refused too, since there's no vector to rescore.--acornworks on the request's filters.Upload config: text fields
scoring,lowercase,stopwordsandstemmerapply only totype: textfields withindex: true, and are refused elsewhere so they can't be silently dropped.Other changes
create_request_buildertakes the query as a singleMainQuery(a vector or a text query) instead of a vector plus a text argument, which keeps it under clippy's argument limit.upload-bm25-textandsearch-bm25-text, plus a README section.Verification
cargo test --release: 207 passed, 7 of them new (config parsing and validation, random and file query sources, request building, flag guards, text index options).cargo clippy --release --all-targets -- -D warnings: clean.cargo fmt --check: clean.dev(850859ec9): upload, random-source and file-source searches all pass, with the server reporting the BM25-scoring index. See the comment below.devbranch: 207 tests, clippy and fmt pass.71143d8's tree is identical to thed960517the change was built against.🤖 Generated with Claude Code