Skip to content

chore: Index multi-properties by default#951

Open
cdc-as81 wants to merge 6 commits into
mainfrom
cdc-as81-multi-properties-should-be-indexed-by-default
Open

chore: Index multi-properties by default#951
cdc-as81 wants to merge 6 commits into
mainfrom
cdc-as81-multi-properties-should-be-indexed-by-default

Conversation

@cdc-as81

@cdc-as81 cdc-as81 commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Make define_multi_property! create a FullIndex by default via
    Property::default_index_type.
  • Allow a multi-property definition to opt into ValueCountIndex or opt out
    with PropertyIndexType::Unindexed.
  • Preserve feat: Replace universal Property: Eq + Hash constraint with IndexableProperty #937’s no-shared-storage design: equivalent multi-properties remain
    distinct, while the first registered representative handles query routing and
    is the only one eligible for a default index.
  • Keep explicit Context::index_property calls compatible with populated
    multi-properties, including default-indexed and unindexed definitions.
  • Update documentation and regression coverage for defaults, overrides,
    equivalent property ordering, and explicit indexing.

@cdc-as81 cdc-as81 linked an issue Jun 10, 2026 that may be closed by this pull request
github-actions Bot added a commit that referenced this pull request Jun 10, 2026
@RobertJacobsonCDC

Copy link
Copy Markdown
Collaborator

I think this needs to be blocked by #937. Also, it's not clear which kind of index should be created. We should discuss how this interacts with these issues, too:

@RobertJacobsonCDC RobertJacobsonCDC added blocked Currently blocked by another issue needs discussion Needs discussion before implementation labels Jun 11, 2026
@github-actions

Copy link
Copy Markdown

Benchmark Results

Hyperfine

Command Mean [ms] Min [ms] Max [ms] Relative
large_sir::baseline 3.4 ± 0.3 3.2 4.6 1.01 ± 0.10
large_sir::baseline_households 3.3 ± 0.2 3.2 4.2 1.00
large_sir::entities 7.4 ± 0.7 6.9 10.3 2.22 ± 0.25
large_sir::households 7.0 ± 0.2 6.8 7.7 2.11 ± 0.12

Criterion

Regressions (slower)
Group Bench Param Change CI Lower CI Upper
counts reindex_after_adding_more_entities 77.234% 76.866% 77.583%
counts index_after_adding_entities 30.538% 30.216% 30.810%
sample_entity sample_entity_single_property_unindexed 100000 26.597% 25.712% 27.265%
large_dataset bench_filter_unindexed_entity 7.826% 3.670% 12.240%
sample_entity sample_entity_multi_property_indexed 1000 7.546% 5.303% 10.317%
sample_entity sample_entity_single_property_unindexed 10000 6.332% 5.689% 7.019%
counts multi_property_indexed_entities 4.625% 2.325% 6.610%
examples example-basic-infection 4.482% 4.055% 4.878%
sample_entity sample_entity_multi_property_indexed 100000 4.034% 2.692% 5.369%
indexing with_query_results_multiple_individually_indexed_properties_enti 3.642% 3.298% 3.934%
sampling sampling_multiple_known_length_entities 2.831% 2.227% 3.428%
sample_entity sample_entity_single_property_unindexed 1000 2.785% 2.562% 3.023%
sampling sampling_multiple_l_reservoir_entities 2.499% 1.872% 3.195%
sampling sampling_single_l_reservoir_entities 2.471% 1.704% 3.258%
indexing query_people_count_multiple_individually_indexed_properties_enti 2.115% 1.707% 2.516%
Improvements (faster)
Group Bench Param Change CI Lower CI Upper
large_dataset bench_query_population_multi_unindexed_entities -99.287% -99.291% -99.284%
indexing query_people_count_single_indexed_property_entities -14.049% -14.576% -13.404%
indexing query_people_indexed_multi-property_entities -11.495% -11.815% -11.160%
sampling count_and_sampling_single_known_length_entities -11.053% -13.757% -8.161%
sample_entity sample_entity_whole_population 10000 -9.614% -15.313% -3.139%
large_dataset bench_match_entity -5.564% -5.933% -5.175%
indexing with_query_results_single_indexed_property_entities -5.410% -6.246% -4.708%
sampling sampling_multiple_unindexed_entities -5.096% -6.279% -3.935%
sample_entity sample_entity_single_property_indexed 1000 -4.095% -5.279% -2.842%
sampling sampling_single_unindexed_entities -2.963% -3.889% -2.044%
sampling count_and_sampling_single_unindexed_concrete_plus_derived_entiti -2.771% -3.098% -2.502%
sample_entity sample_entity_single_property_indexed 100000 -2.715% -4.069% -1.374%
large_dataset bench_query_population_indexed_property_entities -2.549% -2.990% -2.059%
sample_entity sample_entity_single_property_indexed 10000 -2.444% -3.613% -1.222%
large_dataset bench_query_population_derived_property_entities -1.983% -2.459% -1.560%
indexing query_people_multiple_individually_indexed_properties_entities -1.630% -1.903% -1.404%
sampling sampling_single_unindexed_concrete_plus_derived_entities -1.564% -1.837% -1.316%
Unchanged / inconclusive (CI crosses 0%)
Group Bench Param Change CI Lower CI Upper
algorithm_benches algorithm_sampling_single_known_length -5.937% -10.708% -0.909%
sample_entity sample_entity_whole_population 100000 4.540% -1.014% 11.098%
large_dataset bench_filter_indexed_entity 2.818% 0.901% 4.816%
sample_entity sample_entity_multi_property_indexed 10000 1.894% -0.348% 3.966%
sampling sampling_single_known_length_entities -1.885% -4.292% 0.661%
counts single_property_indexed_entities -1.386% -2.523% -0.298%
indexing with_query_results_indexed_multi-property_entities 1.370% 0.842% 1.825%
counts multi_property_unindexed_entities -1.334% -1.716% -0.886%
large_dataset bench_query_population_multi_indexed_entities -0.907% -1.265% -0.571%
examples example-births-deaths 0.887% 0.146% 1.513%
algorithm_benches algorithm_sampling_single_l_reservoir -0.512% -0.881% -0.269%
indexing query_people_count_indexed_multi-property_entities -0.478% -0.679% -0.248%
sample_entity sample_entity_whole_population 1000 -0.400% -8.087% 7.265%
algorithm_benches algorithm_sampling_single_rand_reservoir -0.371% -0.735% -0.066%
indexing query_people_single_indexed_property_entities -0.242% -0.329% -0.167%
large_dataset bench_query_population_property_entities -0.223% -0.738% 0.124%
algorithm_benches algorithm_sampling_multiple_known_length -0.145% -0.710% 0.300%
counts concrete_plus_derived_unindexed_entities 0.104% -0.149% 0.311%
counts single_property_unindexed_entities 0.027% -0.216% 0.314%
algorithm_benches algorithm_sampling_multiple_l_reservoir -0.004% -0.396% 0.413%
Not Compared (no baseline yet)
Group Bench Reason
(none)

github-actions Bot added a commit that referenced this pull request Jun 25, 2026
@CDCgov CDCgov deleted a comment from github-actions Bot Jun 29, 2026
@cdc-as81

Copy link
Copy Markdown
Collaborator Author

lets add an option to the macro to set the index type.
property_store.set_property_indexed::

(PropertyIndexType::FullIndex);

@github-actions

Copy link
Copy Markdown

Benchmark Results

Hyperfine

Command Mean [ms] Min [ms] Max [ms] Relative
large_sir::baseline 3.2 ± 0.0 3.2 3.4 1.00
large_sir::baseline_households 3.5 ± 0.3 3.2 4.4 1.07 ± 0.09
large_sir::entities 6.8 ± 0.3 6.6 8.0 2.12 ± 0.09
large_sir::households 6.7 ± 0.2 6.5 8.0 2.08 ± 0.07

Criterion

Regressions (slower)
Group Bench Param Change CI Lower CI Upper
counts reindex_after_adding_more_entities 101.578% 100.480% 102.609%
large_dataset bench_filter_indexed_entity 79.448% 76.818% 82.235%
counts index_after_adding_entities 32.636% 32.071% 33.094%
counts multi_property_indexed_entities 17.028% 13.988% 19.561%
indexing query_people_single_indexed_property_entities 9.734% 8.928% 10.943%
sample_entity sample_entity_single_property_unindexed 10000 8.915% 7.164% 10.718%
indexing query_people_count_indexed_multi-property_entities 7.955% 6.814% 8.983%
sample_entity sample_entity_whole_population 1000 7.759% 7.368% 8.152%
sample_entity sample_entity_single_property_unindexed 1000 7.584% 7.200% 7.992%
sample_entity sample_entity_whole_population 10000 6.414% 5.468% 7.220%
indexing with_query_results_multiple_individually_indexed_properties_enti 6.101% 5.744% 6.491%
sample_entity sample_entity_whole_population 100000 5.105% 4.698% 5.540%
counts single_property_unindexed_entities 5.042% 3.013% 7.269%
sampling count_and_sampling_single_unindexed_concrete_plus_derived_entiti 4.640% 4.458% 4.797%
large_dataset bench_match_entity 4.118% 3.556% 4.650%
sampling sampling_single_unindexed_concrete_plus_derived_entities 3.541% 3.319% 3.739%
large_dataset bench_query_population_derived_property_entities 2.874% 2.481% 3.297%
large_dataset bench_query_population_multi_indexed_entities 2.870% 2.416% 3.353%
sampling sampling_multiple_l_reservoir_entities 2.783% 2.611% 2.960%
sampling sampling_single_l_reservoir_entities 2.550% 2.228% 2.956%
sampling sampling_multiple_known_length_entities 2.253% 1.821% 2.566%
indexing with_query_results_indexed_multi-property_entities 1.992% 1.409% 2.423%
sampling sampling_multiple_unindexed_entities 1.063% 1.024% 1.102%
Improvements (faster)
Group Bench Param Change CI Lower CI Upper
large_dataset bench_query_population_multi_unindexed_entities -99.130% -99.143% -99.117%
sample_entity sample_entity_multi_property_indexed 1000 -28.507% -29.204% -27.861%
sample_entity sample_entity_multi_property_indexed 10000 -28.356% -29.086% -27.583%
sample_entity sample_entity_multi_property_indexed 100000 -26.040% -26.663% -25.337%
large_dataset bench_filter_unindexed_entity -18.810% -21.640% -15.971%
sample_entity sample_entity_single_property_indexed 100000 -14.514% -15.925% -13.223%
sample_entity sample_entity_single_property_indexed 10000 -14.064% -15.164% -13.062%
sample_entity sample_entity_single_property_indexed 1000 -13.986% -14.999% -12.898%
indexing query_people_count_single_indexed_property_entities -8.030% -8.527% -7.556%
sampling count_and_sampling_single_known_length_entities -3.516% -4.179% -2.855%
indexing query_people_multiple_individually_indexed_properties_entities -1.557% -1.700% -1.422%
sampling sampling_single_unindexed_entities -1.414% -1.522% -1.301%
Unchanged / inconclusive (CI crosses 0%)
Group Bench Param Change CI Lower CI Upper
algorithm_benches algorithm_sampling_single_known_length 1.347% 0.820% 1.845%
examples example-births-deaths -1.089% -2.686% -0.018%
sampling sampling_single_known_length_entities 0.878% -0.369% 2.340%
indexing query_people_count_multiple_individually_indexed_properties_enti -0.842% -1.051% -0.612%
examples example-basic-infection 0.446% -0.076% 1.030%
large_dataset bench_query_population_indexed_property_entities 0.394% -0.003% 0.900%
indexing query_people_indexed_multi-property_entities 0.368% -0.543% 1.065%
large_dataset bench_query_population_property_entities -0.335% -1.025% 0.544%
indexing with_query_results_single_indexed_property_entities -0.252% -0.565% -0.003%
algorithm_benches algorithm_sampling_multiple_l_reservoir 0.217% -0.242% 0.627%
counts single_property_indexed_entities -0.141% -0.821% 0.515%
counts multi_property_unindexed_entities 0.103% -2.472% 2.557%
algorithm_benches algorithm_sampling_multiple_known_length -0.097% -0.838% 0.755%
sample_entity sample_entity_single_property_unindexed 100000 -0.054% -1.171% 0.781%
algorithm_benches algorithm_sampling_single_l_reservoir -0.016% -0.253% 0.222%
counts concrete_plus_derived_unindexed_entities -0.010% -0.703% 0.622%
algorithm_benches algorithm_sampling_single_rand_reservoir -0.003% -0.537% 0.591%
Not Compared (no baseline yet)
Group Bench Reason
(none)

github-actions Bot added a commit that referenced this pull request Jun 29, 2026

@RobertJacobsonCDC RobertJacobsonCDC left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #937 removed sharing indexes between properties. This PR appears to reintroduce shared properties. I don't think we want to do this. Instead, let's retain the warning and error behavior explicitly introduced in #937. I think this will simplify this PR quite a bit.

There's also the philosophical question of whether we want to expose Property::default_index_type as public API. So for example, we could add a parameter to define_*_property! / impl_*_property that lets client code set the return value of Property::default_index_type. But let's defer that question to another issue / PR. (I don't think we have a use case for it, so let's not pay the cost of added code complexity.)

Comment thread docs/book/src/topics/properties.md Outdated
Comment thread docs/book/src/topics/indexing.md Outdated
Comment thread src/entity/query/mod.rs
Comment thread src/entity/context_extension.rs Outdated
Comment thread src/entity/property_store.rs
Comment thread src/entity/property_value_store_core.rs Outdated
Comment thread src/entity/property_value_store_core.rs Outdated
@cdc-as81

Copy link
Copy Markdown
Collaborator Author

Fixed. Equivalent multi-properties now use distinct storage; only the registered representative receives the default index.

@github-actions

Copy link
Copy Markdown

Benchmark Results

Hyperfine

Command Mean [ms] Min [ms] Max [ms] Relative
large_sir::baseline 3.2 ± 0.0 3.2 3.4 1.00
large_sir::baseline_households 3.3 ± 0.1 3.2 3.9 1.01 ± 0.04
large_sir::entities 7.4 ± 0.5 6.9 8.6 2.28 ± 0.16
large_sir::households 6.9 ± 0.2 6.7 7.7 2.12 ± 0.06

Criterion

Regressions (slower)
Group Bench Param Change CI Lower CI Upper
counts reindex_after_adding_more_entities 104.794% 104.192% 105.294%
sampling sampling_single_unindexed_concrete_plus_derived_entities 41.245% 40.956% 41.492%
counts index_after_adding_entities 33.439% 33.186% 33.670%
sample_entity sample_entity_single_property_unindexed 10000 18.727% 18.224% 19.231%
sample_entity sample_entity_single_property_unindexed 1000 17.851% 16.129% 19.300%
large_dataset bench_query_population_multi_indexed_entities 17.743% 16.658% 18.991%
sampling count_and_sampling_single_unindexed_concrete_plus_derived_entiti 16.843% 16.481% 17.174%
sampling sampling_single_known_length_entities 16.484% 14.594% 18.382%
indexing with_query_results_single_indexed_property_entities 15.866% 14.113% 17.649%
counts multi_property_unindexed_entities 15.589% 13.318% 17.855%
algorithm_benches algorithm_sampling_single_known_length 11.490% 9.001% 14.491%
counts single_property_unindexed_entities 8.995% 6.438% 11.810%
counts concrete_plus_derived_unindexed_entities 7.494% 5.122% 10.120%
sample_entity sample_entity_single_property_indexed 100000 4.479% 3.183% 6.044%
sample_entity sample_entity_whole_population 10000 4.131% 2.972% 5.503%
large_dataset bench_query_population_derived_property_entities 3.901% 3.072% 4.764%
sampling sampling_single_l_reservoir_entities 3.797% 3.428% 4.395%
examples example-basic-infection 3.710% 3.204% 4.254%
sample_entity sample_entity_whole_population 1000 2.865% 1.320% 4.218%
indexing query_people_count_single_indexed_property_entities 2.775% 1.676% 3.880%
indexing query_people_single_indexed_property_entities 1.449% 1.030% 1.934%
Improvements (faster)
Group Bench Param Change CI Lower CI Upper
large_dataset bench_query_population_multi_unindexed_entities -99.157% -99.166% -99.149%
large_dataset bench_match_entity -16.547% -17.015% -16.038%
sample_entity sample_entity_multi_property_indexed 10000 -13.960% -14.723% -13.215%
sample_entity sample_entity_multi_property_indexed 1000 -12.239% -13.126% -11.384%
sample_entity sample_entity_multi_property_indexed 100000 -11.150% -11.756% -10.540%
indexing with_query_results_indexed_multi-property_entities -8.192% -9.027% -7.507%
large_dataset bench_query_population_indexed_property_entities -7.744% -8.243% -7.344%
large_dataset bench_query_population_property_entities -6.105% -8.028% -4.390%
large_dataset bench_filter_indexed_entity -5.583% -7.236% -3.877%
indexing query_people_multiple_individually_indexed_properties_entities -5.520% -6.674% -4.639%
sampling sampling_multiple_l_reservoir_entities -3.503% -4.037% -3.068%
sampling count_and_sampling_single_known_length_entities -2.109% -2.686% -1.529%
sampling sampling_multiple_known_length_entities -2.091% -2.778% -1.429%
counts multi_property_indexed_entities -1.336% -1.688% -1.071%
Unchanged / inconclusive (CI crosses 0%)
Group Bench Param Change CI Lower CI Upper
sample_entity sample_entity_whole_population 100000 1.632% 0.283% 2.991%
large_dataset bench_filter_unindexed_entity -0.971% -4.688% 2.886%
counts single_property_indexed_entities -0.961% -1.724% -0.275%
indexing query_people_indexed_multi-property_entities 0.854% 0.417% 1.353%
indexing query_people_count_indexed_multi-property_entities -0.728% -1.129% -0.442%
sample_entity sample_entity_single_property_unindexed 100000 0.640% 0.225% 1.013%
indexing with_query_results_multiple_individually_indexed_properties_enti 0.583% -0.387% 1.291%
sample_entity sample_entity_single_property_indexed 1000 -0.509% -1.659% 0.628%
algorithm_benches algorithm_sampling_single_l_reservoir 0.349% -0.294% 1.111%
indexing query_people_count_multiple_individually_indexed_properties_enti -0.245% -0.426% -0.056%
sample_entity sample_entity_single_property_indexed 10000 0.239% -0.974% 1.334%
algorithm_benches algorithm_sampling_multiple_known_length 0.232% -0.436% 0.913%
algorithm_benches algorithm_sampling_single_rand_reservoir -0.208% -0.648% 0.229%
algorithm_benches algorithm_sampling_multiple_l_reservoir 0.120% -0.458% 0.746%
sampling sampling_single_unindexed_entities -0.096% -0.224% 0.048%
examples example-births-deaths -0.054% -0.643% 0.586%
sampling sampling_multiple_unindexed_entities 0.033% -0.329% 0.424%
Not Compared (no baseline yet)
Group Bench Reason
(none)

github-actions Bot added a commit that referenced this pull request Jul 10, 2026
@github-actions

Copy link
Copy Markdown

Benchmark Results

Hyperfine

Command Mean [ms] Min [ms] Max [ms] Relative
large_sir::baseline 3.4 ± 0.1 3.3 3.6 1.00
large_sir::baseline_households 3.4 ± 0.1 3.3 3.8 1.02 ± 0.03
large_sir::entities 7.9 ± 0.5 7.4 9.5 2.33 ± 0.15
large_sir::households 7.7 ± 0.3 7.1 8.8 2.27 ± 0.10

Criterion

Regressions (slower)
Group Bench Param Change CI Lower CI Upper
counts reindex_after_adding_more_entities 77.518% 77.151% 77.889%
algorithm_benches algorithm_sampling_single_known_length 56.396% 46.439% 65.327%
sampling sampling_single_unindexed_concrete_plus_derived_entities 42.741% 42.180% 43.340%
counts index_after_adding_entities 32.745% 31.491% 33.902%
sample_entity sample_entity_single_property_unindexed 1000 22.151% 21.339% 23.039%
sampling sampling_single_known_length_entities 21.361% 20.030% 22.849%
sampling count_and_sampling_single_unindexed_concrete_plus_derived_entiti 21.298% 20.434% 22.121%
indexing query_people_indexed_multi-property_entities 10.810% 10.212% 11.273%
indexing query_people_single_indexed_property_entities 8.852% 8.608% 9.070%
large_dataset bench_query_population_derived_property_entities 6.772% 6.082% 7.493%
examples example-births-deaths 5.315% 4.723% 5.983%
large_dataset bench_query_population_property_entities 5.187% 2.674% 7.925%
algorithm_benches algorithm_sampling_multiple_known_length 4.434% 3.799% 5.116%
indexing with_query_results_multiple_individually_indexed_properties_enti 3.630% 2.104% 4.870%
sample_entity sample_entity_whole_population 1000 2.343% 1.440% 3.112%
sampling sampling_single_l_reservoir_entities 2.264% 1.998% 2.517%
Improvements (faster)
Group Bench Param Change CI Lower CI Upper
large_dataset bench_query_population_multi_unindexed_entities -99.242% -99.250% -99.235%
large_dataset bench_query_population_multi_indexed_entities -38.508% -39.272% -37.625%
sample_entity sample_entity_single_property_unindexed 10000 -21.798% -22.792% -21.136%
sample_entity sample_entity_single_property_indexed 10000 -16.265% -16.692% -15.872%
large_dataset bench_match_entity -11.082% -11.532% -10.628%
large_dataset bench_filter_indexed_entity -6.732% -8.490% -4.940%
examples example-basic-infection -5.247% -7.323% -3.553%
large_dataset bench_query_population_indexed_property_entities -4.828% -5.404% -4.243%
sample_entity sample_entity_multi_property_indexed 100000 -4.664% -5.557% -3.740%
sample_entity sample_entity_multi_property_indexed 1000 -4.546% -5.553% -3.621%
sample_entity sample_entity_multi_property_indexed 10000 -3.797% -4.797% -2.926%
indexing query_people_count_indexed_multi-property_entities -3.219% -3.656% -2.720%
indexing query_people_count_multiple_individually_indexed_properties_enti -1.956% -2.289% -1.645%
counts multi_property_indexed_entities -1.907% -2.542% -1.240%
sampling sampling_multiple_l_reservoir_entities -1.555% -1.779% -1.335%
sampling sampling_multiple_known_length_entities -1.537% -1.904% -1.231%
Unchanged / inconclusive (CI crosses 0%)
Group Bench Param Change CI Lower CI Upper
large_dataset bench_filter_unindexed_entity -1.308% -5.337% 2.846%
sampling count_and_sampling_single_known_length_entities 1.291% 0.333% 2.599%
sample_entity sample_entity_single_property_unindexed 100000 -1.283% -1.953% -0.650%
sample_entity sample_entity_whole_population 100000 -1.157% -1.619% -0.713%
sample_entity sample_entity_whole_population 10000 -1.143% -1.960% -0.333%
sample_entity sample_entity_single_property_indexed 100000 -1.057% -1.635% -0.664%
indexing query_people_multiple_individually_indexed_properties_entities 0.752% 0.406% 1.005%
algorithm_benches algorithm_sampling_multiple_l_reservoir -0.744% -1.241% -0.253%
counts single_property_unindexed_entities -0.665% -1.560% -0.005%
counts single_property_indexed_entities -0.446% -1.058% 0.083%
counts multi_property_unindexed_entities -0.418% -1.006% -0.009%
sampling sampling_single_unindexed_entities -0.289% -0.320% -0.249%
counts concrete_plus_derived_unindexed_entities 0.260% -0.066% 0.612%
indexing with_query_results_indexed_multi-property_entities -0.251% -0.682% 0.199%
indexing with_query_results_single_indexed_property_entities 0.239% -0.452% 0.819%
sample_entity sample_entity_single_property_indexed 1000 -0.215% -0.701% 0.204%
algorithm_benches algorithm_sampling_single_l_reservoir -0.049% -0.719% 0.553%
sampling sampling_multiple_unindexed_entities 0.048% -0.081% 0.270%
algorithm_benches algorithm_sampling_single_rand_reservoir -0.013% -0.346% 0.454%
indexing query_people_count_single_indexed_property_entities 0.012% -0.551% 0.641%
Not Compared (no baseline yet)
Group Bench Reason
(none)

github-actions Bot added a commit that referenced this pull request Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blocked Currently blocked by another issue needs discussion Needs discussion before implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multi-properties should be indexed by default.

3 participants