Make AOSS insert batch size configurable - #843
Conversation
Signed-off-by: norrishuang <12380647@qq.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: norrishuang The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
| ) -> tuple[int, Exception]: | ||
| embeddings_list = list(embeddings) | ||
| batch_size = 100 if self._is_serverless else len(embeddings_list) | ||
| batch_size = config.NUM_PER_BATCH if self._is_serverless else len(embeddings_list) |
There was a problem hiding this comment.
vectordb_bench/backend/clients/aws_opensearch/aws_opensearch.py line:260
Medium ---- NUM_PER_BATCH comes directly from an environment variable without a positivity check. If it is set to 0, range(..., batch_size) raises ValueError: range() arg 3 must not be zero before any request; a negative value makes the range empty and returns (0, None), falsely reporting a successful load. Please validate this setting (or clamp/reject non-positive values) before using it so operator misconfiguration cannot crash or silently skip ingestion.
There was a problem hiding this comment.
Fixed in fc5b23e. OpenSearch Serverless now rejects non-positive NUM_PER_BATCH values with a clear ValueError before constructing the range, and tests cover both zero and negative values.
|
Hi, thank you for the contribution! FYI we have a broader change to make num per batch a CLI variable coming in #813. Once this pr is merged, 813 will reflect upon this by controlling batch behavior through CLI. |
|
/lgtm |
Merging zilliztech#843 reintroduced config.NUM_PER_BATCH, which the insert-batch-size control feature removed. Route the AOSS serverless bulk chunk size through the task-level insert_batch_size instead, keeping the default of 100 and the non-positive guard. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
Merging zilliztech#843 reintroduced config.NUM_PER_BATCH, which the insert-batch-size control feature removed. Route the AOSS serverless bulk chunk size through the task-level insert_batch_size instead, keeping the default of 100 and the non-positive guard. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com>
* feat: add insert batch and streaming rate controls Replace environment and CloudInsert-specific batch sizing with a task-level insert batch size shared by CLI, REST, frontend, datasets, and runners. Keep streaming insert rate case-specific and validate its relationship to batching. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> * fix(milvus): honor configured insert batch size Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> * fix(opensearch): separate health request timeout Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> * fix(opensearch): propagate vector metric type Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> * fix(fts): bypass filter preparation for unfiltered loads Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> * fix(fts): omit filter schema for unfiltered Milvus Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> * fix(aws_opensearch): honor unified insert_batch_size for serverless bulk Merging #843 reintroduced config.NUM_PER_BATCH, which the insert-batch-size control feature removed. Route the AOSS serverless bulk chunk size through the task-level insert_batch_size instead, keeping the default of 100 and the non-positive guard. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> * fix(fts): prepare qrel-preserving capped corpus Materialize capped qrel-preserving documents during dataset preparation and stream the prepared corpus for filtered and unfiltered loads. This keeps semantic ground truth retrievable without scanning the full raw corpus during timed insertion. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> * fix(frontend): allow small streaming insert rates Lower the streaming insert rate control minimum to one and rely on the existing batch/rate validation for valid combinations. Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> --------- Signed-off-by: jamesgao-jpg <james.gao@zilliz.com> Co-authored-by: Ubuntu <ubuntu@ip-10-15-2-233.us-west-2.compute.internal>
4 participants
What changed
NUM_PER_BATCHconfiguration for OpenSearch Serverless bulk requests instead of a hard-coded batch size of 100.Testing