[IEP] Enhance TextQuery for Hybrid Search: Integrate Keyword and Vector Retrieval using Lucene9+

[junphine]

Proposal

Extend the existing TextQuery API to support hybrid search, combining Lucene-based keyword matching with vector-based Approximate Nearest Neighbor (ANN) search. This aligns with the vision of IEP-141 to enable "hybrid queries, where filtering by metadata (structured data) and ranking by vector happen in a single execution layer" [citation:2][citation:3].

This approach reuses the familiar TextQuery interface, making it intuitive for existing users and avoiding API fragmentation.

Motivation & Design Goal

• Unified Query Experience: Users can perform a search that considers both exact keyword relevance and semantic similarity in one query.
• Leverage Lucene: Reuse the existing GridLuceneIndex infrastructure to manage both inverted indexes (for text) and vector indexes (e.g., Lucene's KnnFloatVectorQuery) [citation:11].
• Single Execution Layer: Perform filtering by structured data and ranking by vector similarity within the same query engine, optimizing complex AI workloads like RAG and semantic search [citation:2][citation:3].

Proposed Interface: Enhanced TextQuery

The TextQuery class will be enhanced with new builder-style methods to accept a vector query component.

package org.apache.ignite.cache.query;

/**
 * A hybrid query that performs both text and vector search.
 * By default, it behaves as a standard TextQuery. When a vector is provided,
 * it performs a hybrid search combining Lucene's keyword and KNN queries.
 */
public final class TextQuery<K, V> extends Query<Cache.Entry<K, V>> {
    /**
     * Standard constructor for pure text search.
     */
    public TextQuery(Class<V> type, String txt) { ... }

    /**
     * NEW: Sets the vector field and query vector for hybrid search.
     *
     * @param vectorFieldName Name of the vector field (annotated with @QueryVectorField).
     * @param vector The query vector (float array).
     * @param k Number of nearest neighbors to return from the vector portion.
     * @return this TextQuery instance for chaining.
     */
    public TextQuery<K, V> setVectorQuery(String vectorFieldName, float[] vector, int k) { ... }

    /**
     * NEW: Sets the distance metric for the vector part (defaults to COSINE).
     */
    public TextQuery<K, V> setDistanceMetric(DistanceMetric metric) { ... }

    /**
     * NEW: Sets the hybrid ranking strategy.
     * - RRF: Reciprocal Rank Fusion (default).
     * - WEIGHTED_SUM: Weighted sum of text and vector scores.
     */
    public TextQuery<K, V> setHybridStrategy(HybridStrategy strategy) { ... }

    /**
     * NEW: Sets the weight for the vector portion in a WEIGHTED_SUM strategy.
     */
    public TextQuery<K, V> setVectorWeight(float weight) { ... }

    // Existing methods: setPageSize, setLocal, etc. remain unchanged [citation:1][citation:4][citation:5].
}

Proposed Annotation: @QueryVectorField

Same as previously proposed. This annotation marks a field for vector indexing, specifying dimension, data type (FP32, INT8, etc.), and the distance metric .

java
package org.apache.ignite.cache.query.annotations;

public @interface QueryVectorField {
    int dimension();
    VectorDataType dataType() default VectorDataType.FP32;
    DistanceMetric metric() default DistanceMetric.COSINE;
}