Skip to content

fix: Eagerly compute property value count indexes#1007

Merged
RobertJacobsonCDC merged 1 commit into
mainfrom
RobertJacobsonCDC_1006_fix_index_property_counts
Jul 16, 2026
Merged

fix: Eagerly compute property value count indexes#1007
RobertJacobsonCDC merged 1 commit into
mainfrom
RobertJacobsonCDC_1006_fix_index_property_counts

Conversation

@RobertJacobsonCDC

@RobertJacobsonCDC RobertJacobsonCDC commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Fixes a correctness bug and adds regression tests for an issue with value count indexes. Apparently we failed to initialize the index upon creation. Consequently, it's possible for a value count index to incorrectly count the number of entities with particular values.

The fix is simple: just compute the index when it is created.

@RobertJacobsonCDC RobertJacobsonCDC linked an issue Jul 13, 2026 that may be closed by this pull request
@github-actions

This comment was marked as outdated.

@RobertJacobsonCDC
RobertJacobsonCDC force-pushed the RobertJacobsonCDC_1006_fix_index_property_counts branch from 509fc2f to 1d98af5 Compare July 16, 2026 16:27
@github-actions

Copy link
Copy Markdown

Benchmark Results

Hyperfine

Command Mean [ms] Min [ms] Max [ms] Relative
large_sir::baseline 2.7 ± 0.0 2.6 2.9 1.00
large_sir::baseline_households 2.7 ± 0.0 2.6 2.9 1.00 ± 0.03
large_sir::entities 6.2 ± 0.1 6.0 6.4 2.29 ± 0.05
large_sir::households 6.0 ± 0.1 5.8 6.4 2.22 ± 0.05

Criterion

Regressions (slower)
Group Bench Param Change CI Lower CI Upper
sampling sampling_multiple_known_length_entities 11.447% 10.058% 12.902%
large_dataset bench_query_population_indexed_property_entities 3.207% 1.767% 4.725%
indexing with_query_results_indexed_multi-property_entities 2.314% 1.705% 3.046%
sampling sampling_single_l_reservoir_entities 2.229% 2.138% 2.312%
counts multi_property_indexed_entities 1.571% 1.134% 2.120%
Improvements (faster)
Group Bench Param Change CI Lower CI Upper
sample_entity sample_entity_multi_property_indexed 100000 -22.188% -22.919% -21.427%
sample_entity sample_entity_multi_property_indexed 1000 -20.769% -21.873% -19.674%
sample_entity sample_entity_multi_property_indexed 10000 -19.995% -21.032% -18.914%
sample_entity sample_entity_single_property_indexed 1000 -13.519% -15.718% -10.967%
sample_entity sample_entity_whole_population 10000 -9.594% -12.161% -6.869%
sampling sampling_multiple_l_reservoir_entities -8.583% -8.933% -8.168%
sample_entity sample_entity_single_property_indexed 100000 -8.040% -9.476% -6.701%
indexing query_people_multiple_individually_indexed_properties_entities -4.742% -4.905% -4.584%
sample_entity sample_entity_whole_population 100000 -4.615% -6.947% -2.247%
sampling sampling_multiple_unindexed_entities -3.974% -5.603% -2.399%
sampling count_and_sampling_single_unindexed_concrete_plus_derived_entiti -3.476% -4.702% -2.297%
large_dataset bench_query_population_derived_property_entities -3.372% -4.161% -2.626%
sample_entity sample_entity_single_property_unindexed 10000 -3.356% -3.801% -2.818%
indexing query_people_count_multiple_individually_indexed_properties_enti -2.076% -2.234% -1.909%
indexing query_people_count_indexed_multi-property_entities -1.836% -2.328% -1.341%
examples example-births-deaths -1.618% -2.228% -1.071%
Unchanged / inconclusive (CI crosses 0%)
Group Bench Param Change CI Lower CI Upper
sample_entity sample_entity_whole_population 1000 -2.172% -4.841% 0.040%
sample_entity sample_entity_single_property_indexed 10000 -2.140% -4.684% 0.172%
sampling sampling_single_known_length_entities 1.944% 0.582% 3.547%
sampling count_and_sampling_single_known_length_entities -1.456% -3.803% 1.099%
examples example-basic-infection -1.445% -2.193% -0.729%
large_dataset bench_query_population_multi_indexed_entities -1.197% -1.720% -0.729%
sampling sampling_single_unindexed_entities -1.053% -2.781% 0.763%
large_dataset bench_query_population_multi_unindexed_entities 0.666% 0.250% 0.965%
counts index_after_adding_entities 0.628% 0.414% 0.845%
indexing query_people_indexed_multi-property_entities 0.578% 0.192% 1.015%
counts single_property_unindexed_entities 0.573% -0.451% 1.795%
algorithm_benches algorithm_sampling_multiple_l_reservoir -0.557% -0.964% -0.164%
large_dataset bench_filter_unindexed_entity -0.540% -4.163% 3.269%
counts single_property_indexed_entities 0.453% -0.030% 0.979%
sample_entity sample_entity_single_property_unindexed 1000 -0.445% -0.822% -0.070%
sample_entity sample_entity_single_property_unindexed 100000 0.372% -1.587% 2.262%
counts concrete_plus_derived_unindexed_entities -0.349% -0.846% -0.009%
indexing with_query_results_single_indexed_property_entities -0.336% -1.000% 0.458%
large_dataset bench_match_entity 0.300% -1.439% 2.017%
sampling sampling_single_unindexed_concrete_plus_derived_entities 0.278% -1.133% 1.665%
algorithm_benches algorithm_sampling_single_l_reservoir 0.270% -0.315% 1.110%
algorithm_benches algorithm_sampling_single_known_length -0.258% -1.798% 1.270%
indexing query_people_count_single_indexed_property_entities 0.236% -0.182% 0.615%
indexing query_people_single_indexed_property_entities -0.172% -0.272% -0.090%
large_dataset bench_query_population_property_entities -0.156% -0.807% 0.318%
algorithm_benches algorithm_sampling_multiple_known_length 0.127% -0.214% 0.472%
large_dataset bench_filter_indexed_entity 0.063% -1.830% 1.927%
algorithm_benches algorithm_sampling_single_rand_reservoir -0.058% -0.551% 0.472%
counts reindex_after_adding_more_entities 0.049% -0.090% 0.179%
counts multi_property_unindexed_entities -0.043% -0.662% 0.518%
indexing with_query_results_multiple_individually_indexed_properties_enti -0.021% -0.636% 0.649%
Not Compared (no baseline yet)
Group Bench Reason
(none)

github-actions Bot added a commit that referenced this pull request Jul 16, 2026
@RobertJacobsonCDC
RobertJacobsonCDC merged commit e52e1e2 into main Jul 16, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

index_property_counts can double-count changed entities

3 participants