From f0b1d84fe625f1c41108af69fb55bae4af72426a Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Tue, 11 Aug 2026 09:50:39 +0000 Subject: [PATCH 01/12] [hotfix][vector-store][java] Update Elasticsearch filters documentation Document the implemented equality-only metadata filter translation for get, delete, and queryEmbedding, including metadata field mapping, AND semantics, raw filter_query composition, and ID-path behavior. Generated-by: OpenAI Codex Desktop 26.803.41515 (GPT-5.6 Sol) --- .../ElasticsearchVectorStore.java | 83 ++++++++++--------- 1 file changed, 45 insertions(+), 38 deletions(-) diff --git a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java index 424c9d398..dc931a9f2 100644 --- a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java +++ b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java @@ -336,23 +336,25 @@ public Map getStoreKwargs() { /** * Retrieve documents from the vector store. * - *

If ids is not provided, this method will retrieve documents according to {@code limit}, - * {@code offset}, and {@code filter_query} in additional arguments. If {@code limit} is null, - * up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} documents are returned (an - * Elasticsearch ceiling). + *

When {@code ids} is non-empty, documents are retrieved directly by ID and the filter, + * limit, offset, and {@code filter_query} arguments are not applied. * - *

The unified {@code filters} DSL parameter is not yet translated to Elasticsearch's native - * query DSL — callers needing structured filtering should pass a raw {@code filter_query} via - * {@code extraArgs}. TODO: implement equality-DSL translation parallel to the Python Chroma - * implementation. + *

Otherwise, {@code filters} provides equality-only matching against document metadata. Each + * entry is translated to an Elasticsearch {@code term} query on {@code + * ..keyword}, and multiple entries are combined with AND semantics. A raw + * Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code extraArgs}; + * when both forms are present, they are combined with AND semantics. * - * @param ids The ids of the documents. - * @param collection The name of the collection to be retrieved. If is null, retrieve the - * default collection. - * @param filters Unified filter DSL. Currently ignored — see method Javadoc. - * @param limit Maximum number of documents to return; falls back to {@link - * ElasticsearchVectorStore#MAX_RESULT_WINDOW} when null. - * @param extraArgs Additional arguments. (offset, filter_query, etc.) + *

The {@code limit} parameter takes precedence over a {@code limit} value in {@code + * extraArgs}. If neither is provided, up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} + * documents are returned. {@code extraArgs} may also contain an {@code offset}. + * + * @param ids The IDs of documents to retrieve directly. + * @param collection The collection name, or null to use the default collection. + * @param filters Equality-only metadata filters combined with AND semantics. + * @param limit Maximum number of documents to return for filtered or unfiltered searches. + * @param extraArgs Additional arguments, including {@code offset} and raw JSON {@code + * filter_query}. * @return List of documents retrieved. */ @Override @@ -379,22 +381,22 @@ public List get( } /** - * Delete documents in the vector store. + * Delete documents from the vector store. * - *

If ids is not provided, this method will delete documents matched the {@code filter_query} - * in additional arguments. If neither {@code filter_query} nor {@code filters} is provided, - * this method will delete all the documents. + *

When {@code ids} is non-empty, documents are deleted directly by ID and {@code filters} + * and {@code filter_query} are not applied. * - *

The unified {@code filters} DSL parameter is not yet translated to Elasticsearch's native - * query DSL — callers needing structured filtering should pass a raw {@code filter_query} via - * {@code extraArgs}. TODO: implement equality-DSL translation parallel to the Python Chroma - * implementation. + *

Otherwise, {@code filters} provides equality-only matching against document metadata. Each + * entry is translated to an Elasticsearch {@code term} query on {@code + * ..keyword}, and multiple entries are combined with AND semantics. A raw + * Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code extraArgs}; + * when both forms are present, they are combined with AND semantics. If neither form is + * supplied, all documents in the collection are deleted. * - * @param ids The ids of the documents. - * @param collection The name of the collection the documents belong to. If is null, use the - * default collection. - * @param filters Unified filter DSL. Currently ignored — see method Javadoc. - * @param extraArgs Additional arguments. (filter_query, etc.) + * @param ids The IDs of documents to delete directly. + * @param collection The collection name, or null to use the default collection. + * @param filters Equality-only metadata filters combined with AND semantics. + * @param extraArgs Additional arguments, including raw JSON {@code filter_query}. */ @Override public void delete( @@ -571,17 +573,22 @@ private void deleteDocuments( * Executes a KNN vector search using a pre-computed embedding. * *

The method prepares a KNN search request using the supplied {@code embedding} and merges - * default arguments from the store with the provided {@code args}. Optional filter queries - * (JSON DSL) are applied as a post filter. + * default arguments from the store with the provided {@code args}. {@code filters} provides + * equality-only matching against metadata fields. Each entry targets {@code + * ..keyword}; multiple entries are combined with AND semantics and applied + * as a post-filter. + * + *

A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code + * args}. When both filter forms are present, they are combined with AND semantics. * - * @param embedding The embedding vector to search with - * @param limit Maximum number of items the caller is interested in; used as a fallback for - * {@code k} if not explicitly provided - * @param collection The index to query search. If is null, search the default index. - * @param args Additional arguments. Supported keys: {@code k}, {@code num_candidates}, {@code - * filter_query} - * @return A list of matching documents, possibly empty - * @throws RuntimeException if the search request fails + * @param embedding The embedding vector to search with. + * @param limit Maximum number of items requested; used as a fallback for {@code k}. + * @param collection The collection name, or null to use the default collection. + * @param filters Equality-only metadata filters combined with AND semantics. + * @param args Additional arguments. Supported keys are {@code k}, {@code num_candidates}, and + * raw JSON {@code filter_query}. + * @return A list of matching documents, possibly empty. + * @throws RuntimeException if the search request fails. */ @SuppressWarnings({"rawtypes", "unchecked"}) @Override From 41c78db8a73827d6f594afc4a5b103af0f92713b Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Wed, 12 Aug 2026 06:51:36 +0000 Subject: [PATCH 02/12] [hotfix][vector-store][java] Clarify Elasticsearch filter limits Document that non-string metadata filters do not match keyword sub-fields and clarify the MAX_RESULT_WINDOW result-window ceiling. Generated-by: OpenAI Codex Desktop 26.803.41515 (GPT-5.6 Sol) --- .../ElasticsearchVectorStore.java | 21 +++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java index dc931a9f2..af2e95457 100644 --- a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java +++ b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java @@ -341,13 +341,17 @@ public Map getStoreKwargs() { * *

Otherwise, {@code filters} provides equality-only matching against document metadata. Each * entry is translated to an Elasticsearch {@code term} query on {@code - * ..keyword}, and multiple entries are combined with AND semantics. A raw + * ..keyword}, and multiple entries are combined with AND semantics. Because + * Elasticsearch dynamic mapping creates `.keyword` sub-fields only for strings, filters on + * non-string metadata values do not match; use a raw `filter_query` for those values. A raw * Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code extraArgs}; * when both forms are present, they are combined with AND semantics. * *

The {@code limit} parameter takes precedence over a {@code limit} value in {@code * extraArgs}. If neither is provided, up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} - * documents are returned. {@code extraArgs} may also contain an {@code offset}. + * documents are returned. This is the Elasticsearch result-window ceiling: the combined + * {@code offset} and {@code limit} must not exceed it; an explicit limit above it is rejected + * by Elasticsearch rather than truncated. {@code extraArgs} may also contain an {@code offset}. * * @param ids The IDs of documents to retrieve directly. * @param collection The collection name, or null to use the default collection. @@ -388,7 +392,9 @@ public List get( * *

Otherwise, {@code filters} provides equality-only matching against document metadata. Each * entry is translated to an Elasticsearch {@code term} query on {@code - * ..keyword}, and multiple entries are combined with AND semantics. A raw + * ..keyword}, and multiple entries are combined with AND semantics. Because + * Elasticsearch dynamic mapping creates `.keyword` sub-fields only for strings, filters on + * non-string metadata values do not match; use a raw `filter_query` for those values. A raw * Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code extraArgs}; * when both forms are present, they are combined with AND semantics. If neither form is * supplied, all documents in the collection are deleted. @@ -548,7 +554,7 @@ private void deleteDocuments( if (combined != null) { builder.query(q -> q.withJson(new StringReader(combined))); } else { - // No filter at all → delete every document (match_all). + // No filter at all 闂?delete every document (match_all). builder.query(q -> q.matchAll(ma -> ma)); } @@ -575,7 +581,10 @@ private void deleteDocuments( *

The method prepares a KNN search request using the supplied {@code embedding} and merges * default arguments from the store with the provided {@code args}. {@code filters} provides * equality-only matching against metadata fields. Each entry targets {@code - * ..keyword}; multiple entries are combined with AND semantics and applied + * ..keyword}; multiple entries are combined with AND semantics. Because + * Elasticsearch dynamic mapping creates `.keyword` sub-fields only for strings, filters on + * non-string metadata values do not match; use a raw {@code filter_query} for those values. The + * filters are applied as a post-filter. * as a post-filter. * *

A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code @@ -725,7 +734,7 @@ private List getDocuments( throws JsonProcessingException { final List documents = new ArrayList<>(total); for (Hit> hit : searchResponse.hits().hits()) { - // hit.score() is a Double — null for plain get-all responses, populated for + // hit.score() is a Double 闂?null for plain get-all responses, populated for // KNN / scored search; mirror that null-ness on Document.score. Double score = hit.score(); documents.add( From e423a8aecfcd0910076781a29dcdfaddd9ac1b10 Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Wed, 12 Aug 2026 07:18:43 +0000 Subject: [PATCH 03/12] [hotfix] Retrigger CI after runner certificate failure Retrigger checks after Code Style Check and macOS Python setup failed on a self-signed certificate before project code ran. Generated-by: OpenAI Codex Desktop 26.803.41515 (GPT-5.6 Sol) From 620801bed54b7f96125fca4b84cb9806871cab2a Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Wed, 12 Aug 2026 07:58:50 +0000 Subject: [PATCH 04/12] [hotfix] Retrigger license check after runner certificate failure Retrigger CI after the Code Style Check failed during the runner certificate setup before check-license.sh executed. Generated-by: OpenAI Codex Desktop 26.803.41515 (GPT-5.6 Sol) From bd7f71028e29d762a38cec4ac2a9b7ea1a485eef Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Wed, 12 Aug 2026 09:01:35 +0000 Subject: [PATCH 05/12] [hotfix][vector-store][java] Fix Elasticsearch Javadoc formatting Align the result-window and post-filter documentation with google-java-format and remove an accidental duplicate sentence. Generated-by: OpenAI Codex Desktop 26.803.41515 (GPT-5.6 Sol) --- .../elasticsearch/ElasticsearchVectorStore.java | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java index af2e95457..f3877af21 100644 --- a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java +++ b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java @@ -349,9 +349,9 @@ public Map getStoreKwargs() { * *

The {@code limit} parameter takes precedence over a {@code limit} value in {@code * extraArgs}. If neither is provided, up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} - * documents are returned. This is the Elasticsearch result-window ceiling: the combined - * {@code offset} and {@code limit} must not exceed it; an explicit limit above it is rejected - * by Elasticsearch rather than truncated. {@code extraArgs} may also contain an {@code offset}. + * documents are returned. This is the Elasticsearch result-window ceiling: the combined {@code + * offset} and {@code limit} must not exceed it; an explicit limit above it is rejected by + * Elasticsearch rather than truncated. {@code extraArgs} may also contain an {@code offset}. * * @param ids The IDs of documents to retrieve directly. * @param collection The collection name, or null to use the default collection. @@ -585,7 +585,6 @@ private void deleteDocuments( * Elasticsearch dynamic mapping creates `.keyword` sub-fields only for strings, filters on * non-string metadata values do not match; use a raw {@code filter_query} for those values. The * filters are applied as a post-filter. - * as a post-filter. * *

A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code * args}. When both filter forms are present, they are combined with AND semantics. From 6f944fadc05189515c5101cf6816d9df95eedeb6 Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Wed, 12 Aug 2026 09:39:55 +0000 Subject: [PATCH 06/12] [hotfix] Retrigger pull request checks From 0bf1106288f9f894e5da52633060aa93578a56a9 Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Thu, 13 Aug 2026 01:32:22 +0000 Subject: [PATCH 07/12] [hotfix] Retrigger pull request checks Generated-by: Codex (GPT-5) From f824d8a53cd039e338175c10568fa1f67f8b7757 Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Thu, 13 Aug 2026 02:22:44 +0000 Subject: [PATCH 08/12] [hotfix][vector-store][java] Fix Elasticsearch filtered KNN search Fixes #999 --- .../ElasticsearchVectorStore.java | 29 +++++++++---------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java index f3877af21..9e136c6c4 100644 --- a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java +++ b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java @@ -83,7 +83,7 @@ *

  • {@code num_candidates} (optional): Candidate set size for ANN search; can be overridden per * query. *
  • {@code filter_query} (optional): A raw JSON Elasticsearch filter query (DSL) that is - * applied as a post-filter; can be overridden per query. + * applied during KNN search; can be overridden per query. *
  • {@code host} or {@code hosts} (optional): Elasticsearch endpoint(s). If omitted, defaults * to {@code localhost:9200}. *
  • Authentication (optional): Either basic auth via {@code username}/{@code password}, or API @@ -618,20 +618,19 @@ public List queryEmbedding( List queryVector = new ArrayList<>(embedding.length); for (float v : embedding) queryVector.add(v); - SearchRequest.Builder builder = - new SearchRequest.Builder() - .index(index) - .knn( - kb -> - kb.field(this.vectorField) - .queryVector(queryVector) - .k(k) - .numCandidates(numCandidates)); - - if (combined != null) { - final String finalCombined = combined; - builder = builder.postFilter(f -> f.withJson(new StringReader(finalCombined))); - } + SearchRequest.Builder builder = new SearchRequest.Builder().index(index); + final String finalCombined = combined; + builder.knn( + kb -> { + kb.field(this.vectorField) + .queryVector(queryVector) + .k(k) + .numCandidates(numCandidates); + if (finalCombined != null) { + kb.filter(f -> f.withJson(new StringReader(finalCombined))); + } + return kb; + }); final SearchResponse> searchResponse = (SearchResponse) this.client.search(builder.build(), Map.class); From e24eaca66e03afc53d7998097b09846c26cf146d Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Thu, 13 Aug 2026 07:07:29 +0000 Subject: [PATCH 09/12] [hotfix][vector-store][java] Complete Elasticsearch filtered KNN fix Fixes #999 --- .github/workflows/ci.yml | 14 ++++ .../content/docs/development/vector_stores.md | 2 +- .../ElasticsearchVectorStore.java | 2 +- .../ElasticsearchVectorStoreTest.java | 82 +++++++++++++++++-- 4 files changed, 92 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1bfdbae05..b7850dbc8 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -238,12 +238,26 @@ jobs: distribution: 'temurin' - name: Install flink-agents Java run: bash tools/build.sh -j + - name: Start Elasticsearch + run: | + docker compose -f tools/docker/elasticsearch/docker-compose.yml down -v + docker compose -f tools/docker/elasticsearch/docker-compose.yml up -d + timeout 180 bash -c 'until curl -fsS http://localhost:9200/_cluster/health; do sleep 5; done' + - name: Run Elasticsearch integration tests + env: + ES_HOST: http://localhost:9200 + run: | + mvn -B --no-transfer-progress -pl integrations/vector-stores/elasticsearch -am -Dspotless.skip=true -Drat.skip=true -Dtest=ElasticsearchVectorStoreTest -Dsurefire.failIfNoSpecifiedTests=false test - name: Install ollama run: bash tools/start_ollama_server.sh - name: Run Java IT env: LOG_LEVEL: INFO + ES_HOST: http://localhost:9200 run: tools/ut.sh -j -e -f ${{ matrix.flink-version }} + - name: Stop Elasticsearch + if: always() + run: docker compose -f tools/docker/elasticsearch/docker-compose.yml down -v cross_language_tests: name: cross-language [${{ matrix.os }}] [python-${{ matrix.python-version}}] [java-${{ matrix.java-version}}] diff --git a/docs/content/docs/development/vector_stores.md b/docs/content/docs/development/vector_stores.md index 07d5eafa7..66ebea495 100644 --- a/docs/content/docs/development/vector_stores.md +++ b/docs/content/docs/development/vector_stores.md @@ -800,7 +800,7 @@ Elasticsearch is currently supported in the Java API only. To use Elasticsearch | `dims` | int | `768` | Vector dimensionality | | `k` | int | None | Number of nearest neighbors to return; can be overridden per query | | `num_candidates` | int | None | Candidate set size for ANN search; can be overridden per query | -| `filter_query` | str | None | Raw JSON Elasticsearch filter query (DSL) applied as a post-filter | +| `filter_query` | str | None | Raw JSON Elasticsearch filter query (DSL) applied during KNN candidate selection | | `host` | str | `"http://localhost:9200"` | Elasticsearch endpoint | | `hosts` | str | None | Comma-separated list of Elasticsearch endpoints | | `username` | str | None | Username for basic authentication | diff --git a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java index 9e136c6c4..6b9d248a5 100644 --- a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java +++ b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java @@ -584,7 +584,7 @@ private void deleteDocuments( * ..keyword}; multiple entries are combined with AND semantics. Because * Elasticsearch dynamic mapping creates `.keyword` sub-fields only for strings, filters on * non-string metadata values do not match; use a raw {@code filter_query} for those values. The - * filters are applied as a post-filter. + * filters are applied during KNN candidate selection. * *

    A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code * args}. When both filter forms are present, they are combined with AND semantics. diff --git a/integrations/vector-stores/elasticsearch/src/test/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStoreTest.java b/integrations/vector-stores/elasticsearch/src/test/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStoreTest.java index e89365e1f..e358e36eb 100644 --- a/integrations/vector-stores/elasticsearch/src/test/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStoreTest.java +++ b/integrations/vector-stores/elasticsearch/src/test/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStoreTest.java @@ -29,8 +29,8 @@ import org.apache.flink.agents.api.vectorstores.VectorStoreQuery; import org.junit.jupiter.api.Assertions; import org.junit.jupiter.api.BeforeAll; -import org.junit.jupiter.api.Disabled; import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable; import org.mockito.Mockito; import java.util.ArrayList; @@ -48,7 +48,7 @@ *

    For {@link ElasticsearchVectorStore} doesn't support security check yet, when start the * container, should add "-e xpack.security.enabled=false" option. */ -@Disabled("Should setup Elasticsearch server.") +@EnabledIfEnvironmentVariable(named = "ES_HOST", matches = ".+") public class ElasticsearchVectorStoreTest { public static BaseVectorStore store; @@ -64,7 +64,7 @@ public static Resource getResource(String name, ResourceType type) { } @BeforeAll - public static void initialize() { + public static void initialize() throws Exception { final ResourceDescriptor.Builder builder = ResourceDescriptor.Builder.newBuilder(ElasticsearchVectorStore.class.getName()) .addInitialArgument("embedding_model", "embeddingModel") @@ -76,6 +76,7 @@ public static void initialize() { new ElasticsearchVectorStore( builder.build(), ResourceContext.fromGetResource(ElasticsearchVectorStoreTest::getResource)); + store.open(); } @Test @@ -121,7 +122,13 @@ public void testDocumentManagement() throws Exception { // test get all documents List all = store.get(null, name, null, null, Collections.emptyMap()); - Assertions.assertEquals(documents, all); + Assertions.assertEquals(2, all.size()); + Assertions.assertEquals(documents.get(0).getId(), all.get(0).getId()); + Assertions.assertEquals(documents.get(0).getContent(), all.get(0).getContent()); + Assertions.assertEquals(documents.get(0).getMetadata(), all.get(0).getMetadata()); + Assertions.assertEquals(documents.get(1).getId(), all.get(1).getId()); + Assertions.assertEquals(documents.get(1).getContent(), all.get(1).getContent()); + Assertions.assertEquals(documents.get(1).getMetadata(), all.get(1).getMetadata()); // test get specific document List specific = @@ -132,14 +139,18 @@ public void testDocumentManagement() throws Exception { null, Collections.emptyMap()); Assertions.assertEquals(1, specific.size()); - Assertions.assertEquals(documents.get(0), specific.get(0)); + Assertions.assertEquals(documents.get(0).getId(), specific.get(0).getId()); + Assertions.assertEquals(documents.get(0).getContent(), specific.get(0).getContent()); + Assertions.assertEquals(documents.get(0).getMetadata(), specific.get(0).getMetadata()); // test delete specific document store.delete(Collections.singletonList("doc1"), name, null, Collections.emptyMap()); Thread.sleep(1000); List remain = store.get(null, name, null, null, Collections.emptyMap()); Assertions.assertEquals(1, remain.size()); - Assertions.assertEquals(documents.get(1), remain.get(0)); + Assertions.assertEquals(documents.get(1).getId(), remain.get(0).getId()); + Assertions.assertEquals(documents.get(1).getContent(), remain.get(0).getContent()); + Assertions.assertEquals(documents.get(1).getMetadata(), remain.get(0).getMetadata()); // test delete all documents store.delete(null, name, null, Collections.emptyMap()); @@ -191,6 +202,65 @@ public void testFiltersDsl() throws Exception { ((CollectionManageableVectorStore) store).deleteCollection(name); } + @Test + public void testFilteredKnnReturnsMatchingDocumentsOutsideUnfilteredTopK() throws Exception { + String name = "filtered_knn_top_k"; + ((CollectionManageableVectorStore) store).createCollectionIfNotExists(name, Map.of()); + + float[] queryVector = new float[] {1.0f, 0.0f, 0.0f, 0.0f, 0.0f}; + store.add( + List.of( + new Document( + "near bob", + Map.of("user_id", "bob"), + "doc_bob", + new float[] {1.0f, 0.0f, 0.0f, 0.0f, 0.0f}), + new Document( + "near alice", + Map.of("user_id", "alice", "tenant", "allowed"), + "doc_alice", + new float[] {0.9f, 0.1f, 0.0f, 0.0f, 0.0f}), + new Document( + "far alice", + Map.of("user_id", "alice", "tenant", "blocked"), + "doc_alice_far", + new float[] {0.0f, 0.0f, 1.0f, 0.0f, 0.0f})), + name, + Collections.emptyMap()); + + List filteredByDsl = + store.queryEmbedding( + queryVector, 1, name, Map.of("user_id", "alice"), Collections.emptyMap()); + Assertions.assertEquals(1, filteredByDsl.size()); + Assertions.assertEquals("doc_alice", filteredByDsl.get(0).getId()); + + List filteredByRawQuery = + store.queryEmbedding( + queryVector, + 1, + name, + null, + Map.of( + "filter_query", + "{\"term\":{\"_metadata.user_id.keyword\":\"alice\"}}")); + Assertions.assertEquals(1, filteredByRawQuery.size()); + Assertions.assertEquals("doc_alice", filteredByRawQuery.get(0).getId()); + + List filteredByBoth = + store.queryEmbedding( + queryVector, + 1, + name, + Map.of("user_id", "alice"), + Map.of( + "filter_query", + "{\"term\":{\"_metadata.tenant.keyword\":\"allowed\"}}")); + Assertions.assertEquals(1, filteredByBoth.size()); + Assertions.assertEquals("doc_alice", filteredByBoth.get(0).getId()); + + ((CollectionManageableVectorStore) store).deleteCollection(name); + } + @Test public void testUpdateOverwritesExistingDocument() throws Exception { // ES bulk index is upsert by id — update should rewrite the doc in place. From 5507da411bc2bb9fa9c0dcaf5473a81b53cd0edf Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Thu, 13 Aug 2026 07:26:51 +0000 Subject: [PATCH 10/12] [hotfix][vector-store][java] Restore PR 997 documentation scope --- .github/workflows/ci.yml | 14 ---- .../content/docs/development/vector_stores.md | 2 +- .../ElasticsearchVectorStore.java | 31 +++---- .../ElasticsearchVectorStoreTest.java | 82 ++----------------- 4 files changed, 23 insertions(+), 106 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b7850dbc8..1bfdbae05 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -238,26 +238,12 @@ jobs: distribution: 'temurin' - name: Install flink-agents Java run: bash tools/build.sh -j - - name: Start Elasticsearch - run: | - docker compose -f tools/docker/elasticsearch/docker-compose.yml down -v - docker compose -f tools/docker/elasticsearch/docker-compose.yml up -d - timeout 180 bash -c 'until curl -fsS http://localhost:9200/_cluster/health; do sleep 5; done' - - name: Run Elasticsearch integration tests - env: - ES_HOST: http://localhost:9200 - run: | - mvn -B --no-transfer-progress -pl integrations/vector-stores/elasticsearch -am -Dspotless.skip=true -Drat.skip=true -Dtest=ElasticsearchVectorStoreTest -Dsurefire.failIfNoSpecifiedTests=false test - name: Install ollama run: bash tools/start_ollama_server.sh - name: Run Java IT env: LOG_LEVEL: INFO - ES_HOST: http://localhost:9200 run: tools/ut.sh -j -e -f ${{ matrix.flink-version }} - - name: Stop Elasticsearch - if: always() - run: docker compose -f tools/docker/elasticsearch/docker-compose.yml down -v cross_language_tests: name: cross-language [${{ matrix.os }}] [python-${{ matrix.python-version}}] [java-${{ matrix.java-version}}] diff --git a/docs/content/docs/development/vector_stores.md b/docs/content/docs/development/vector_stores.md index 66ebea495..07d5eafa7 100644 --- a/docs/content/docs/development/vector_stores.md +++ b/docs/content/docs/development/vector_stores.md @@ -800,7 +800,7 @@ Elasticsearch is currently supported in the Java API only. To use Elasticsearch | `dims` | int | `768` | Vector dimensionality | | `k` | int | None | Number of nearest neighbors to return; can be overridden per query | | `num_candidates` | int | None | Candidate set size for ANN search; can be overridden per query | -| `filter_query` | str | None | Raw JSON Elasticsearch filter query (DSL) applied during KNN candidate selection | +| `filter_query` | str | None | Raw JSON Elasticsearch filter query (DSL) applied as a post-filter | | `host` | str | `"http://localhost:9200"` | Elasticsearch endpoint | | `hosts` | str | None | Comma-separated list of Elasticsearch endpoints | | `username` | str | None | Username for basic authentication | diff --git a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java index 6b9d248a5..f3877af21 100644 --- a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java +++ b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java @@ -83,7 +83,7 @@ *

  • {@code num_candidates} (optional): Candidate set size for ANN search; can be overridden per * query. *
  • {@code filter_query} (optional): A raw JSON Elasticsearch filter query (DSL) that is - * applied during KNN search; can be overridden per query. + * applied as a post-filter; can be overridden per query. *
  • {@code host} or {@code hosts} (optional): Elasticsearch endpoint(s). If omitted, defaults * to {@code localhost:9200}. *
  • Authentication (optional): Either basic auth via {@code username}/{@code password}, or API @@ -584,7 +584,7 @@ private void deleteDocuments( * ..keyword}; multiple entries are combined with AND semantics. Because * Elasticsearch dynamic mapping creates `.keyword` sub-fields only for strings, filters on * non-string metadata values do not match; use a raw {@code filter_query} for those values. The - * filters are applied during KNN candidate selection. + * filters are applied as a post-filter. * *

    A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code * args}. When both filter forms are present, they are combined with AND semantics. @@ -618,19 +618,20 @@ public List queryEmbedding( List queryVector = new ArrayList<>(embedding.length); for (float v : embedding) queryVector.add(v); - SearchRequest.Builder builder = new SearchRequest.Builder().index(index); - final String finalCombined = combined; - builder.knn( - kb -> { - kb.field(this.vectorField) - .queryVector(queryVector) - .k(k) - .numCandidates(numCandidates); - if (finalCombined != null) { - kb.filter(f -> f.withJson(new StringReader(finalCombined))); - } - return kb; - }); + SearchRequest.Builder builder = + new SearchRequest.Builder() + .index(index) + .knn( + kb -> + kb.field(this.vectorField) + .queryVector(queryVector) + .k(k) + .numCandidates(numCandidates)); + + if (combined != null) { + final String finalCombined = combined; + builder = builder.postFilter(f -> f.withJson(new StringReader(finalCombined))); + } final SearchResponse> searchResponse = (SearchResponse) this.client.search(builder.build(), Map.class); diff --git a/integrations/vector-stores/elasticsearch/src/test/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStoreTest.java b/integrations/vector-stores/elasticsearch/src/test/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStoreTest.java index e358e36eb..e89365e1f 100644 --- a/integrations/vector-stores/elasticsearch/src/test/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStoreTest.java +++ b/integrations/vector-stores/elasticsearch/src/test/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStoreTest.java @@ -29,8 +29,8 @@ import org.apache.flink.agents.api.vectorstores.VectorStoreQuery; import org.junit.jupiter.api.Assertions; import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Disabled; import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable; import org.mockito.Mockito; import java.util.ArrayList; @@ -48,7 +48,7 @@ *

    For {@link ElasticsearchVectorStore} doesn't support security check yet, when start the * container, should add "-e xpack.security.enabled=false" option. */ -@EnabledIfEnvironmentVariable(named = "ES_HOST", matches = ".+") +@Disabled("Should setup Elasticsearch server.") public class ElasticsearchVectorStoreTest { public static BaseVectorStore store; @@ -64,7 +64,7 @@ public static Resource getResource(String name, ResourceType type) { } @BeforeAll - public static void initialize() throws Exception { + public static void initialize() { final ResourceDescriptor.Builder builder = ResourceDescriptor.Builder.newBuilder(ElasticsearchVectorStore.class.getName()) .addInitialArgument("embedding_model", "embeddingModel") @@ -76,7 +76,6 @@ public static void initialize() throws Exception { new ElasticsearchVectorStore( builder.build(), ResourceContext.fromGetResource(ElasticsearchVectorStoreTest::getResource)); - store.open(); } @Test @@ -122,13 +121,7 @@ public void testDocumentManagement() throws Exception { // test get all documents List all = store.get(null, name, null, null, Collections.emptyMap()); - Assertions.assertEquals(2, all.size()); - Assertions.assertEquals(documents.get(0).getId(), all.get(0).getId()); - Assertions.assertEquals(documents.get(0).getContent(), all.get(0).getContent()); - Assertions.assertEquals(documents.get(0).getMetadata(), all.get(0).getMetadata()); - Assertions.assertEquals(documents.get(1).getId(), all.get(1).getId()); - Assertions.assertEquals(documents.get(1).getContent(), all.get(1).getContent()); - Assertions.assertEquals(documents.get(1).getMetadata(), all.get(1).getMetadata()); + Assertions.assertEquals(documents, all); // test get specific document List specific = @@ -139,18 +132,14 @@ public void testDocumentManagement() throws Exception { null, Collections.emptyMap()); Assertions.assertEquals(1, specific.size()); - Assertions.assertEquals(documents.get(0).getId(), specific.get(0).getId()); - Assertions.assertEquals(documents.get(0).getContent(), specific.get(0).getContent()); - Assertions.assertEquals(documents.get(0).getMetadata(), specific.get(0).getMetadata()); + Assertions.assertEquals(documents.get(0), specific.get(0)); // test delete specific document store.delete(Collections.singletonList("doc1"), name, null, Collections.emptyMap()); Thread.sleep(1000); List remain = store.get(null, name, null, null, Collections.emptyMap()); Assertions.assertEquals(1, remain.size()); - Assertions.assertEquals(documents.get(1).getId(), remain.get(0).getId()); - Assertions.assertEquals(documents.get(1).getContent(), remain.get(0).getContent()); - Assertions.assertEquals(documents.get(1).getMetadata(), remain.get(0).getMetadata()); + Assertions.assertEquals(documents.get(1), remain.get(0)); // test delete all documents store.delete(null, name, null, Collections.emptyMap()); @@ -202,65 +191,6 @@ public void testFiltersDsl() throws Exception { ((CollectionManageableVectorStore) store).deleteCollection(name); } - @Test - public void testFilteredKnnReturnsMatchingDocumentsOutsideUnfilteredTopK() throws Exception { - String name = "filtered_knn_top_k"; - ((CollectionManageableVectorStore) store).createCollectionIfNotExists(name, Map.of()); - - float[] queryVector = new float[] {1.0f, 0.0f, 0.0f, 0.0f, 0.0f}; - store.add( - List.of( - new Document( - "near bob", - Map.of("user_id", "bob"), - "doc_bob", - new float[] {1.0f, 0.0f, 0.0f, 0.0f, 0.0f}), - new Document( - "near alice", - Map.of("user_id", "alice", "tenant", "allowed"), - "doc_alice", - new float[] {0.9f, 0.1f, 0.0f, 0.0f, 0.0f}), - new Document( - "far alice", - Map.of("user_id", "alice", "tenant", "blocked"), - "doc_alice_far", - new float[] {0.0f, 0.0f, 1.0f, 0.0f, 0.0f})), - name, - Collections.emptyMap()); - - List filteredByDsl = - store.queryEmbedding( - queryVector, 1, name, Map.of("user_id", "alice"), Collections.emptyMap()); - Assertions.assertEquals(1, filteredByDsl.size()); - Assertions.assertEquals("doc_alice", filteredByDsl.get(0).getId()); - - List filteredByRawQuery = - store.queryEmbedding( - queryVector, - 1, - name, - null, - Map.of( - "filter_query", - "{\"term\":{\"_metadata.user_id.keyword\":\"alice\"}}")); - Assertions.assertEquals(1, filteredByRawQuery.size()); - Assertions.assertEquals("doc_alice", filteredByRawQuery.get(0).getId()); - - List filteredByBoth = - store.queryEmbedding( - queryVector, - 1, - name, - Map.of("user_id", "alice"), - Map.of( - "filter_query", - "{\"term\":{\"_metadata.tenant.keyword\":\"allowed\"}}")); - Assertions.assertEquals(1, filteredByBoth.size()); - Assertions.assertEquals("doc_alice", filteredByBoth.get(0).getId()); - - ((CollectionManageableVectorStore) store).deleteCollection(name); - } - @Test public void testUpdateOverwritesExistingDocument() throws Exception { // ES bulk index is upsert by id — update should rewrite the doc in place. From 271818de3ba5765fe1b414f10af695d3130c4ee6 Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Mon, 17 Aug 2026 02:52:09 +0000 Subject: [PATCH 11/12] [hotfix][vector-store][java] Address Elasticsearch Javadoc review --- .../ElasticsearchVectorStore.java | 91 +++++++++++-------- 1 file changed, 53 insertions(+), 38 deletions(-) diff --git a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java index 5d66e479f..13bb89bc8 100644 --- a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java +++ b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java @@ -336,23 +336,29 @@ public Map getStoreKwargs() { /** * Retrieve documents from the vector store. * - *

    If ids is not provided, this method will retrieve documents according to {@code limit}, - * {@code offset}, and {@code filter_query} in additional arguments. If {@code limit} is null, - * up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} documents are returned (an - * Elasticsearch ceiling). + *

    When {@code ids} is non-empty, documents are retrieved directly by ID and the filter, + * limit, offset, and {@code filter_query} arguments are not applied. * - *

    The unified {@code filters} DSL parameter is not yet translated to Elasticsearch's native - * query DSL — callers needing structured filtering should pass a raw {@code filter_query} via - * {@code extraArgs}. TODO: implement equality-DSL translation parallel to the Python Chroma - * implementation. + *

    Otherwise, {@code filters} provides equality-only matching against document metadata. Each + * entry is translated to an Elasticsearch {@code term} query on {@code + * ..keyword}, and multiple entries are combined with AND semantics. Because + * Elasticsearch dynamic mapping creates {@code .keyword} sub-fields only for strings, filters + * on non-string metadata values do not match; use a raw {@code filter_query} for those values. + * A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code + * extraArgs}; when both forms are present, they are combined with AND semantics. * - * @param ids The ids of the documents. - * @param collection The name of the collection to be retrieved. If is null, retrieve the - * default collection. - * @param filters Unified filter DSL. Currently ignored — see method Javadoc. - * @param limit Maximum number of documents to return; falls back to {@link - * ElasticsearchVectorStore#MAX_RESULT_WINDOW} when null. - * @param extraArgs Additional arguments. (offset, filter_query, etc.) + *

    The {@code limit} parameter takes precedence over a {@code limit} value in {@code + * extraArgs}. If neither is provided, up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} + * documents are returned. This is the Elasticsearch result-window ceiling: the combined {@code + * offset} and {@code limit} must not exceed it; an explicit limit above it is rejected by + * Elasticsearch rather than truncated. {@code extraArgs} may also contain an {@code offset}. + * + * @param ids The IDs of documents to retrieve directly. + * @param collection The collection name, or null to use the default collection. + * @param filters Equality-only metadata filters combined with AND semantics. + * @param limit Maximum number of documents to return for filtered or unfiltered searches. + * @param extraArgs Additional arguments, including {@code offset} and raw JSON {@code + * filter_query}. * @return List of documents retrieved. */ @Override @@ -379,22 +385,24 @@ public List get( } /** - * Delete documents in the vector store. + * Delete documents from the vector store. * - *

    If ids is not provided, this method will delete documents matched the {@code filter_query} - * in additional arguments. If neither {@code filter_query} nor {@code filters} is provided, - * this method will delete all the documents. + *

    When {@code ids} is non-empty, documents are deleted directly by ID and {@code filters} + * and {@code filter_query} are not applied. * - *

    The unified {@code filters} DSL parameter is not yet translated to Elasticsearch's native - * query DSL — callers needing structured filtering should pass a raw {@code filter_query} via - * {@code extraArgs}. TODO: implement equality-DSL translation parallel to the Python Chroma - * implementation. + *

    Otherwise, {@code filters} provides equality-only matching against document metadata. Each + * entry is translated to an Elasticsearch {@code term} query on {@code + * ..keyword}, and multiple entries are combined with AND semantics. Because + * Elasticsearch dynamic mapping creates {@code .keyword} sub-fields only for strings, filters + * on non-string metadata values do not match; use a raw {@code filter_query} for those values. + * A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code + * extraArgs}; when both forms are present, they are combined with AND semantics. If neither + * form is supplied, all documents in the collection are deleted. * - * @param ids The ids of the documents. - * @param collection The name of the collection the documents belong to. If is null, use the - * default collection. - * @param filters Unified filter DSL. Currently ignored — see method Javadoc. - * @param extraArgs Additional arguments. (filter_query, etc.) + * @param ids The IDs of documents to delete directly. + * @param collection The collection name, or null to use the default collection. + * @param filters Equality-only metadata filters combined with AND semantics. + * @param extraArgs Additional arguments, including raw JSON {@code filter_query}. */ @Override public void delete( @@ -571,19 +579,26 @@ private void deleteDocuments( * Executes a KNN vector search using a pre-computed embedding. * *

    The method prepares a KNN search request using the supplied {@code embedding} and merges - * default arguments from the store with the provided {@code args}. Optional filter queries - * (JSON DSL) restrict the documents the KNN search may match, so the nearest neighbours are + * default arguments from the store with the provided {@code args}. {@code filters} provides + * equality-only matching against metadata fields. Each entry targets {@code + * ..keyword}; multiple entries are combined with AND semantics. Because + * Elasticsearch dynamic mapping creates {@code .keyword} sub-fields only for strings, filters + * on non-string metadata values do not match; use a raw {@code filter_query} for those values. + * + *

    A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code + * args}. When both filter forms are present, they are combined with AND semantics. The combined + * filter restricts the documents the KNN search may match, so the nearest neighbours are * selected from among the matching documents rather than filtered out afterwards. Up to {@code * k} matching documents are returned even when the closest vectors overall do not match. * - * @param embedding The embedding vector to search with - * @param limit Maximum number of items the caller is interested in; used as a fallback for - * {@code k} if not explicitly provided - * @param collection The index to query search. If is null, search the default index. - * @param args Additional arguments. Supported keys: {@code k}, {@code num_candidates}, {@code - * filter_query} - * @return A list of matching documents, possibly empty - * @throws RuntimeException if the search request fails + * @param embedding The embedding vector to search with. + * @param limit Maximum number of items requested; used as a fallback for {@code k}. + * @param collection The collection name, or null to use the default collection. + * @param filters Equality-only metadata filters combined with AND semantics. + * @param args Additional arguments. Supported keys are {@code k}, {@code num_candidates}, and + * raw JSON {@code filter_query}. + * @return A list of matching documents, possibly empty. + * @throws RuntimeException if the search request fails. */ @SuppressWarnings({"rawtypes", "unchecked"}) @Override From 608312b27bd70661f3c92e33dfd42bf12487184a Mon Sep 17 00:00:00 2001 From: Leo Wang Date: Tue, 25 Aug 2026 07:36:59 +0000 Subject: [PATCH 12/12] [hotfix][vector-store][java] Clarify Elasticsearch result-window documentation --- .../elasticsearch/ElasticsearchVectorStore.java | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java index 13bb89bc8..4be3d6be1 100644 --- a/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java +++ b/integrations/vector-stores/elasticsearch/src/main/java/org/apache/flink/agents/integrations/vectorstores/elasticsearch/ElasticsearchVectorStore.java @@ -112,7 +112,7 @@ public class ElasticsearchVectorStore extends BaseVectorStore /** Default vector dimensionality used when {@code dims} is not provided. */ public static final int DEFAULT_DIMENSION = 768; - /** The maximum number of documents that can be retrieved in get. */ + /** Default request size used by {@code get} when no limit is provided. */ public static final int MAX_RESULT_WINDOW = 10000; public static final String DEFAULT_METADATA_FIELD = "_metadata"; @@ -348,10 +348,12 @@ public Map getStoreKwargs() { * extraArgs}; when both forms are present, they are combined with AND semantics. * *

    The {@code limit} parameter takes precedence over a {@code limit} value in {@code - * extraArgs}. If neither is provided, up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} - * documents are returned. This is the Elasticsearch result-window ceiling: the combined {@code - * offset} and {@code limit} must not exceed it; an explicit limit above it is rejected by - * Elasticsearch rather than truncated. {@code extraArgs} may also contain an {@code offset}. + * extraArgs}. If neither limit is provided, the request size defaults to {@link + * ElasticsearchVectorStore#MAX_RESULT_WINDOW} (10,000). The effective Elasticsearch + * result-window limit is controlled by the target index's {@code index.max_result_window} + * setting, which defaults to 10,000 but is configurable. Elasticsearch rejects requests when + * the combined {@code offset} and request size exceed that setting rather than truncating them. + * {@code extraArgs} may also contain an {@code offset}. * * @param ids The IDs of documents to retrieve directly. * @param collection The collection name, or null to use the default collection.