From 19cacfdc08db5652289c32fc8a24e3223a3b9eeb Mon Sep 17 00:00:00 2001 From: Serhiy Bzhezytskyy Date: Fri, 4 Sep 2026 15:52:21 +0300 Subject: [PATCH] nyc_taxis: make the fixed-interval histogram 60 days wide, on the epoch grid OpenSearch Benchmark's date_histogram_fixed_interval asks for "fixed_interval": "60d", and OpenSearch anchors a fixed interval on the epoch, not on the query's lower bound. The port asked Solr for "+30DAY" starting at the query's own 2015-01-01, so it produced buckets of half the width on different edges. Both artifacts, both engines, same 300,649 documents. The Solr operation body is sent to Solr 10.0.0 and the OpenSearch Benchmark operation body to OpenSearch 3.8.0, so the comparison is between the two shipped files rather than a reconstruction: shipped start=2015-01-01 gap=+30DAY 13 buckets width start=2015-01-01 gap=+60DAY 7 buckets this fix start=2014-11-06 gap=+60DAY 8 buckets, identical to OpenSearch Solr OpenSearch 2014-11-06T00:00:00 2996 | 2014-11-06 00:00:00 2996 2015-01-05T00:00:00 52666 | 2015-01-05 00:00:00 52666 2015-03-06T00:00:00 54062 | 2015-03-06 00:00:00 54062 2015-05-05T00:00:00 51300 | 2015-05-05 00:00:00 51300 2015-07-04T00:00:00 45290 | 2015-07-04 00:00:00 45290 2015-09-02T00:00:00 47561 | 2015-09-02 00:00:00 47561 2015-11-01T00:00:00 46009 | 2015-11-01 00:00:00 46009 2015-12-31T00:00:00 756 | 2015-12-31 00:00:00 756 Solr numFound and both bucket sums are 300,640, so no matched document falls outside a bucket on either side. 2014-11-06 is the last 60-day boundary from 1970-01-01 that is not after the corpus; the buckets before the data are empty and mincount 1 drops them. The two sibling operations were checked the same way and needed no change: date_histogram_calendar_interval agrees on 12 buckets and date_histogram_facet on 21, counts identical. The README said "upstream" without saying upstream of what, and here it matters: Elastic's rally-tracks carries a workload of the same name that has diverged (fixed_interval 10d, calendar_interval week), so it names the reference once for the whole table. Co-Authored-By: Claude Opus 5 --- nyc_taxis/README.md | 9 +++++++-- nyc_taxis/operations/default.json | 4 ++-- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/nyc_taxis/README.md b/nyc_taxis/README.md index a22ba46..ab67410 100644 --- a/nyc_taxis/README.md +++ b/nyc_taxis/README.md @@ -95,7 +95,12 @@ parameters. ## Search operations The following named operations are defined in `operations/default.json` and used by the test -procedures above: +procedures above. In the notes, *upstream* means OpenSearch Benchmark's `nyc_taxis` +([opensearch-benchmark-workloads](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/nyc_taxis)), +which these operations were ported from. Elastic's `rally-tracks` carries a workload of the same +name but with different values in places — `fixed_interval: 10d` and `calendar_interval: week` +where OpenSearch Benchmark has `60d` and `month` — so the two are not interchangeable as a +reference. | Operation | Query | Notes | |-----------|-------|-------| @@ -104,7 +109,7 @@ procedures above: | `distance_amount_facet` | `trip_distance:[0 TO 50}` + range facet | Range facet on `trip_distance`. Exclusive upper bound, matching the upstream `lt: 50` | | `date_histogram_facet` | Date range, `+1DAY` gap | Daily dropoff histogram, all 21 days of Jan 2015. Upstream bounds this with `lte: 21/01/2015` in `dd/MM/yyyy`, which covers the whole of the 21st, so the filter runs to `2015-01-22T00:00:00Z` exclusive | | `date_histogram_calendar_interval` | Date range, `+1MONTH` gap | Monthly histogram, calendar-aligned buckets. Exclusive upper bound, matching the upstream `lt: 2016-01-01` | -| `date_histogram_fixed_interval` | Date range, `+30DAY` gap | Monthly histogram, fixed-width buckets. Exclusive upper bound, matching the upstream `lt: 2016-01-01` | +| `date_histogram_fixed_interval` | Date range, `+60DAY` gap | Fixed-width buckets, 60 days each, matching the upstream `fixed_interval: 60d`. OpenSearch anchors a fixed interval on the epoch rather than on the query, so the facet starts at `2014-11-06T00:00:00Z` — the last 60-day boundary at or before the corpus — which puts the bucket edges on the same instants. Exclusive upper bound, matching the upstream `lt: 2016-01-01` | | `desc_sort_tip_amount` | `*:*` sort `tip_amount desc` | Descending sort | | `asc_sort_tip_amount` | `*:*` sort `tip_amount asc` | Ascending sort | | `desc_sort_passenger_count` | `*:*` sort `passenger_count desc` | Descending sort | diff --git a/nyc_taxis/operations/default.json b/nyc_taxis/operations/default.json index ad346df..abc33ec 100644 --- a/nyc_taxis/operations/default.json +++ b/nyc_taxis/operations/default.json @@ -102,9 +102,9 @@ "dropoffs_over_time": { "type": "range", "field": "dropoff_datetime", - "gap": "+30DAY", + "gap": "+60DAY", "mincount": 1, - "start": "2015-01-01T00:00:00Z", + "start": "2014-11-06T00:00:00Z", "end": "2016-01-01T00:00:00Z" } }