Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ public class ElasticsearchVectorStore extends BaseVectorStore

/** Default vector dimensionality used when {@code dims} is not provided. */
public static final int DEFAULT_DIMENSION = 768;
/** The maximum number of documents that can be retrieved in get. */
/** Default request size used by {@code get} when no limit is provided. */
public static final int MAX_RESULT_WINDOW = 10000;

public static final String DEFAULT_METADATA_FIELD = "_metadata";
Expand Down Expand Up @@ -336,23 +336,31 @@ public Map<String, Object> getStoreKwargs() {
/**
* Retrieve documents from the vector store.
*
* <p>If ids is not provided, this method will retrieve documents according to {@code limit},
* {@code offset}, and {@code filter_query} in additional arguments. If {@code limit} is null,
* up to {@link ElasticsearchVectorStore#MAX_RESULT_WINDOW} documents are returned (an
* Elasticsearch ceiling).
* <p>When {@code ids} is non-empty, documents are retrieved directly by ID and the filter,
* limit, offset, and {@code filter_query} arguments are not applied.
*
* <p>The unified {@code filters} DSL parameter is not yet translated to Elasticsearch's native
* query DSL — callers needing structured filtering should pass a raw {@code filter_query} via
* {@code extraArgs}. TODO: implement equality-DSL translation parallel to the Python Chroma
* implementation.
* <p>Otherwise, {@code filters} provides equality-only matching against document metadata. Each
* entry is translated to an Elasticsearch {@code term} query on {@code
* <metadataField>.<key>.keyword}, and multiple entries are combined with AND semantics. Because
* Elasticsearch dynamic mapping creates {@code .keyword} sub-fields only for strings, filters
* on non-string metadata values do not match; use a raw {@code filter_query} for those values.
* A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code
* extraArgs}; when both forms are present, they are combined with AND semantics.
*
* @param ids The ids of the documents.
* @param collection The name of the collection to be retrieved. If is null, retrieve the
* default collection.
* @param filters Unified filter DSL. Currently ignored — see method Javadoc.
* @param limit Maximum number of documents to return; falls back to {@link
* ElasticsearchVectorStore#MAX_RESULT_WINDOW} when null.
* @param extraArgs Additional arguments. (offset, filter_query, etc.)
* <p>The {@code limit} parameter takes precedence over a {@code limit} value in {@code
* extraArgs}. If neither limit is provided, the request size defaults to {@link
* ElasticsearchVectorStore#MAX_RESULT_WINDOW} (10,000). The effective Elasticsearch
* result-window limit is controlled by the target index's {@code index.max_result_window}
* setting, which defaults to 10,000 but is configurable. Elasticsearch rejects requests when
* the combined {@code offset} and request size exceed that setting rather than truncating them.
* {@code extraArgs} may also contain an {@code offset}.
*
* @param ids The IDs of documents to retrieve directly.
* @param collection The collection name, or null to use the default collection.
* @param filters Equality-only metadata filters combined with AND semantics.
* @param limit Maximum number of documents to return for filtered or unfiltered searches.
* @param extraArgs Additional arguments, including {@code offset} and raw JSON {@code
* filter_query}.
* @return List of documents retrieved.
*/
@Override
Expand All @@ -379,22 +387,24 @@ public List<Document> get(
}

/**
* Delete documents in the vector store.
* Delete documents from the vector store.
*
* <p>If ids is not provided, this method will delete documents matched the {@code filter_query}
* in additional arguments. If neither {@code filter_query} nor {@code filters} is provided,
* this method will delete all the documents.
* <p>When {@code ids} is non-empty, documents are deleted directly by ID and {@code filters}
* and {@code filter_query} are not applied.
*
* <p>The unified {@code filters} DSL parameter is not yet translated to Elasticsearch's native
* query DSL — callers needing structured filtering should pass a raw {@code filter_query} via
* {@code extraArgs}. TODO: implement equality-DSL translation parallel to the Python Chroma
* implementation.
* <p>Otherwise, {@code filters} provides equality-only matching against document metadata. Each
* entry is translated to an Elasticsearch {@code term} query on {@code
* <metadataField>.<key>.keyword}, and multiple entries are combined with AND semantics. Because
* Elasticsearch dynamic mapping creates {@code .keyword} sub-fields only for strings, filters
* on non-string metadata values do not match; use a raw {@code filter_query} for those values.
* A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code
* extraArgs}; when both forms are present, they are combined with AND semantics. If neither
* form is supplied, all documents in the collection are deleted.
*
* @param ids The ids of the documents.
* @param collection The name of the collection the documents belong to. If is null, use the
* default collection.
* @param filters Unified filter DSL. Currently ignored — see method Javadoc.
* @param extraArgs Additional arguments. (filter_query, etc.)
* @param ids The IDs of documents to delete directly.
* @param collection The collection name, or null to use the default collection.
* @param filters Equality-only metadata filters combined with AND semantics.
* @param extraArgs Additional arguments, including raw JSON {@code filter_query}.
*/
@Override
public void delete(
Expand Down Expand Up @@ -571,19 +581,26 @@ private void deleteDocuments(
* Executes a KNN vector search using a pre-computed embedding.
*
* <p>The method prepares a KNN search request using the supplied {@code embedding} and merges
* default arguments from the store with the provided {@code args}. Optional filter queries
* (JSON DSL) restrict the documents the KNN search may match, so the nearest neighbours are
* default arguments from the store with the provided {@code args}. {@code filters} provides
* equality-only matching against metadata fields. Each entry targets {@code
* <metadataField>.<key>.keyword}; multiple entries are combined with AND semantics. Because
* Elasticsearch dynamic mapping creates {@code .keyword} sub-fields only for strings, filters
* on non-string metadata values do not match; use a raw {@code filter_query} for those values.
*
* <p>A raw Elasticsearch JSON query may also be supplied as {@code filter_query} in {@code
* args}. When both filter forms are present, they are combined with AND semantics. The combined
* filter restricts the documents the KNN search may match, so the nearest neighbours are
* selected from among the matching documents rather than filtered out afterwards. Up to {@code
* k} matching documents are returned even when the closest vectors overall do not match.
*
* @param embedding The embedding vector to search with
* @param limit Maximum number of items the caller is interested in; used as a fallback for
* {@code k} if not explicitly provided
* @param collection The index to query search. If is null, search the default index.
* @param args Additional arguments. Supported keys: {@code k}, {@code num_candidates}, {@code
* filter_query}
* @return A list of matching documents, possibly empty
* @throws RuntimeException if the search request fails
* @param embedding The embedding vector to search with.
* @param limit Maximum number of items requested; used as a fallback for {@code k}.
* @param collection The collection name, or null to use the default collection.
* @param filters Equality-only metadata filters combined with AND semantics.
* @param args Additional arguments. Supported keys are {@code k}, {@code num_candidates}, and
* raw JSON {@code filter_query}.
* @return A list of matching documents, possibly empty.
* @throws RuntimeException if the search request fails.
*/
@SuppressWarnings({"rawtypes", "unchecked"})
@Override
Expand Down
Loading