feat(semantic-layer): generate one metric per measure field in build#488
Open
yustme wants to merge 1 commit into
Open
feat(semantic-layer): generate one metric per measure field in build#488yustme wants to merge 1 commit into
yustme wants to merge 1 commit into
Conversation
The heuristic builder emitted a single COUNT(*) placeholder metric per table, regardless of how many measurable columns it had. Match the semantic-layer toolkit: emit one metric per `measure` field, with the aggregate guessed from the column name -- SUM by default, AVG for rate/percent/ratio names (so it does not trip the SUM_ON_PCT validation), MIN/MAX for min/max names. A `count_*` column stays SUM (additive counts sum to a total). The COUNT(*) row-count metric is kept as a baseline, and metric names are deduplicated model-wide. On a 47-measure table the build now yields ~48 metrics instead of 1.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
kbagent semantic-layer buildemitted a singleCOUNT(*)placeholder metric per table, no matter how many measurable columns it had. On a metrics table with dozens of numeric columns you got one metric and had to add the rest by hand — whereas the semantic-layer toolkit generates one metric per measure.Fix
Emit one metric per
measurefield, with the aggregate guessed from the column name (mirrors the toolkit, with two deliberate refinements):avg/average/mean/rate/pct/percent/ratio/shareAVGmax/maximum/peakMAXmin/minimumMINSUMRefinements over a naive port:
SUM_ON_PCTvalidation warning.count_*stays SUM — additive daily counts sum to a total;COUNT()of an already-aggregated column is meaningless.The
COUNT(*)row-count metric is kept as a baseline, metric names aretotal_/avg_/max_/min_-prefixed, snake_cased, and deduplicated model-wide (_2,_3, … on collision).Result
On a 47-measure table the build now yields ~48 metrics (43 SUM + 4 AVG for the pct/rate columns + 1 row count) instead of 1.
Tests
TestEstimateMetricAggregation: default SUM,count_*stays SUM, rate/pct → AVG, min/max.TestHeuristicMetricPerMeasure: one metric per measure + row count; dimension columns produce none; duplicate measure names across tables are deduped.test_heuristic_fallbackfor the measure-metric + row-count pair.tests/test_semantic_layer_service.pypasses;ruff checkclean.Note
Independent of #486 (field-role classifier) and #487 (alias type resolution), but composes with them: #486/#487 make more columns correctly classify as
measure, which this PR turns into metrics. The generated metric SQL reuses the builder's existing Snowflake-style FQN ("KEBOOLA"."<bucket>"."<table>") — pre-existing behaviour, unchanged here.