Skip to content

Fix MODE datapoint queries to honor gapFill - #3482

Open
ebariaux wants to merge 17 commits into
mainfrom
bugfix/mode-query-gap-fill
Open

ebariaux wants to merge 17 commits into
mainfrom
bugfix/mode-query-gap-fill

Conversation

@ebariaux

Copy link
Copy Markdown
Contributor

Description

MODE queries previously returned only buckets containing datapoints, even
when gapFill=true. They now include empty buckets throughout the requested
range, including leading/trailing gaps and entirely empty ranges.

The fix uses grouped PostgreSQL MODE aggregates for historical and predicted
numeric and boolean datapoints. Existing null handling and tie-breaking are
preserved, and the dependency on PostgreSQL’s tie-breaking behavior is
documented.

This follows the JSON-null query fixes in #3447.

Compatibility / migration

This changes results for existing callers using MODE with gapFill=true:

  • Responses can contain additional empty buckets with unavailable values.
    These serialize as {"x":…}, with y omitted.
  • Generated buckets count toward OR_DATA_POINTS_QUERY_LIMIT, which defaults
    to 100,000. Sparse queries over large ranges with small intervals may now
    exceed this limit.
  • Set gapFill=false to retain populated-bucket-only results. Alternatively,
    shorten the requested range or use a larger interval.

Release notes

MODE datapoint queries now honor gapFill=true, returning empty buckets
with unavailable values across the requested range. This applies to
historical and predicted numeric and boolean datapoints.

Existing callers may receive more datapoints and reach the configured query
limit (100,000 by default). Use gapFill=false to preserve the previous
populated-bucket-only behavior, or request a shorter range or larger interval.
Empty buckets omit the y property in JSON responses.

ebariaux and others added 17 commits October 19, 2026 11:24
Replace correlated frequency queries with grouped MODE aggregates.
Preserve null handling and tie-breaking, and cover empty ranges and leading/trailing gaps with regression tests.

BREAKING CHANGE: MODE with gapFill=true now includes empty buckets as null-valued results across the inclusive query range.
Results may contain more rows and exceed the configured datapoint query limit.
Use gapFill=false to retain populated-bucket-only results.
Normalize JSON nulls before casting and remove the fallback that replaced unavailable differences with absolute readings.
Document carry-forward semantics and add regression coverage.

BREAKING CHANGE: With gap filling enabled, the first reading after a leading empty bucket now yields null instead of its absolute value as the difference.
Restore MODE's existing populated-bucket-only behavior for both gapFill
settings, retaining JSON-null handling and numeric/boolean tie-breaking.
Restore the corresponding expectations and document the MODE limitation.

The gap-fill behavior change and its regression coverage will be carried
by bugfix/mode-query-gap-fill as a separate follow-up.
Use grouped MODE aggregates so time_bucket_gapfill can produce empty
buckets across the inclusive requested range. Preserve JSON-null handling,
numeric and boolean tie-breaking, and populated-only results when disabled.

Migrate the internal, leading/trailing, entirely empty range and inclusive
end-boundary assertions from the JSON-null PR. Add coverage for sparse
queries exceeding the configured result limit after gap filling.

Release note / compatibility:
MODE with gapFill=true now returns null-valued empty buckets. Existing
callers may receive more results and exceed OR_DATA_POINTS_QUERY_LIMIT
(100,000 by default). Use gapFill=false to preserve populated-bucket-only
results, or reduce the range or increase the interval to request fewer
buckets. This applies to historical and predicted numeric/boolean data.
@ebariaux ebariaux added Bug Error or issue in the system Breaking Change Indicates that a PR includes a breaking change labels Oct 23, 2026
@ebariaux
ebariaux requested review from a team and richturner and removed request for a team October 23, 2026 15:40
@ebariaux ebariaux linked an issue Oct 23, 2026 that may be closed by this pull request
6 tasks
@sonarqubecloud

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Breaking Change Indicates that a PR includes a breaking change Bug Error or issue in the system

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MODE datapoint queries ignore gapFill and omit empty buckets

1 participant