CPU and I/O latency reach robust-baseline parity in Darling (#1743 follow-up)

[erikdarlingdata]

Closes the coverage asymmetry #1994 shipped with: Darling's CPU and I/O latency baselines read sum/sumsq rollups that cannot produce a median, so those two families degraded to the classical gate while Lite had robust statistics on all nine non-event metrics.

Measured before built — twice

• The quick fix (4-day raw window) was wrong: measured on the production store, zero of 5,044 full buckets would satisfy the distinct-day trust floors — a regression to the absolute-fallback bar, not an upgrade.
• The premise behind that fix was also wrong: cpu_utilization and file_io_stats are not on the 4-day raw tier at all — they carry their own 30-day service-side retention (CollectorScheduleDefaults: 1-minute cadence, 30-day retention; verified on the production store — no TimescaleDB retention policy, the store's full life present, compressed after a day).

What

• Both arms read their raw hypertables at Lite's exact grain through the shared robust scaffold — the mean/stddev produced are the same per-sample statistics the rollups reconstructed, plus the median/MAD the rollups could not. IO keeps its DOUBLE PRECISION cast and its nullable-ratio row-count semantics (the row_count-vs-ratio_count distinction the retired rollup documented, preserved at raw grain).
• Prod-validated: the composed CPU arm on the fleet's busiest tenant runs in 227 ms for the full window; tier structure exact (168 full + 24 hour-only + 1 flat); 42 full buckets already trustworthy at 16 days of store life with every hour-only sentinel trustworthy — the tier ladder rides store age correctly, robust stats intact at whichever tier selection lands on.
• Pins move with the reasons: the nine aggregate-fed families keep their supply pin; CPU/IO are pinned to their exact raw tables; and a new pin (RawBaselineFamilies_RetentionCoversTheWindow) makes the load-bearing invariant a red build — if either collector's retention ever drops below the 30-day baseline window, the build fails naming PgBaselineProvider times out client-side scanning raw history, and tiered retention will mask it rather than fix it #1757 instead of quietly regressing the baselines.
• The retired cpu_utilization_baseline/file_io_baseline aggregates stay registered (upgrade compatibility); removing them is separate cleanup.

Sequencing

Merges after tonight's nightly was cut, so #1994 gets its clean solo dogfood window on the monitor box; this rides the next nightly through the same soak.

🤖 Generated with Claude Code

[claude]

Review

What this does: brings Darling's CPU and I/O latency baselines to parity with Lite by reading the raw cpu_utilization_stats/file_io_stats hypertables through the shared RobustTierScaffold instead of the old sum/sumsq rollup materialized views, so those two families get median/MAD like the other nine. Confirmed correct: query logic mirrors Lite's v_cpu_utilization_stats/v_file_io_stats reads (mean/stddev math is equivalent to the retired rollup's Bessel-corrected reconstruction), the nullable-ratio row_count-vs-ratio_count semantics for IO are preserved, bare table names resolve via the documented search_path = collect, config, public, and the test updates correctly reflect the new 9-aggregate/2-raw split (BaselineAggregates still has 9 entries including the now-unused cpu_utilization_baseline/file_io_baseline, kept for upgrade compat).

One correctness concern worth a look (posted inline): switching CPU/IO from the aggregate views to raw-table reads removes a structural retention guarantee those two families used to have. I left the details on PgBaselineProvider.cs.

No Lite/Darling parity issues found — this PR is closing a parity gap, and the new SQL text matches Lite's shape wherever the Postgres/DuckDB dialects allow. No security concerns (no new external input paths, no dynamic SQL — the retention lookup is a compiled dictionary key). No missing-index recommendations included, per instructions.

[claude]

Retention guarantee weakened for CPU/IO, with no runtime guard — only a build-time default pin.

Before this PR, CPU and I/O baselines were served by cpu_utilization_baseline/file_io_baseline, which are entries in TimescaleSupport.BaselineAggregates. That array feeds RetentionPolicies (TimescaleSupport.cs ~1921-1926), which gives every baseline aggregate a fixed, product-controlled BaselineRetentionInterval (35d) — structurally independent of anything a user configures, "by construction" per the leaf-rule comment there.

After this PR, CPU/IO read the raw cpu_utilization_stats/file_io_stats hypertables directly. Per the PR description, these tables have no TimescaleDB retention policy — they purge via DarlingRetention.cs's drop_chunks, keyed to the user-editable CollectorScheduleDefaults/schedule-editor retention (CollectorScheduleEditorWindow.xaml.cs enforces only RetentionDays >= 1, not >= the 30-day baseline window).

So a Darling operator who shortens cpu_utilization or file_io_stats retention via the schedule editor (a legitimate, supported action — same UI Lite's WarnIfRetentionUndercutsBaselineWindow exists specifically to guard against) will silently regress these two families back into the #1757 short-supply shape. Nothing in this PR catches that at runtime:

• RawBaselineFamilies_RetentionCoversTheWindow (Darling.Tests) only pins the code default (CollectorScheduleDefaults.All["cpu_utilization"].RetentionDays >= 30) — it can't see a live store's configured retention.
• PgBaselineProvider has no equivalent of Lite BaselineProvider's _retentionDaysForCollector seam / WarnIfRetentionUndercutsBaselineWindow warning at all.

Worth either wiring a runtime warning analogous to Lite's for these two collectors, or enforcing a >=30-day floor specifically for cpu_utilization/file_io_stats in the schedule editor (the way the baseline aggregates' retention is structurally guaranteed regardless of user config). As shipped, this PR trades a structural guarantee for a build-time-only one.