Support Start TimeStamp (ST) & Created Timestamp (CT) on PRW2 (cortexproject#7371)

* Support Start TimeStamp on PRW2

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* Clear CT

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* Add feature flag

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* pre-allocate v2 samples/histograms

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* fix test

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* Add metric

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

* move debug log to push level

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>

---------

Signed-off-by: SungJin1212 <tjdwls1201@gmail.com>
Signed-off-by: Charlie Le <charlie_le@apple.com>

Author: SungJin1212

CHANGELOG.md

@@ -1,6 +1,7 @@
 # Changelog
 
 ## master / unreleased
+* [FEATURE] Distributor: Add experimental `-distributor.enable-start-timestamp` flag for Prometheus Remote Write 2.0. When enabled, `StartTimestamp (ST)` is ingested. #7371
 * [ENHANCEMENT] Metrics Helper: Add native histogram support for aggregating and merging, including dual-format histogram handling that exposes both native and classic bucket formats. #7359
 * [ENHANCEMENT] Cache: Add per-tenant TTL configuration for query results cache to control cache expiration on a per-tenant basis with separate TTLs for regular and out-of-order data. #7357
 * [ENHANCEMENT] Tenant Federation: Add a local cache to regex resolver. #7363

docs/configuration/config-file-reference.md

@@ -4111,6 +4111,12 @@ The `limits_config` configures default and per-tenant limits imposed by Cortex s
 # CLI flag: -distributor.enable-type-and-unit-labels
 [enable_type_and_unit_labels: <boolean> | default = false]
 
+# EXPERIMENTAL: If true, StartTimestampMs (ST) is handled for remote write v2
+# samples and histograms. CreatedTimestamp (CT) is used as a fallback when ST is
+# not set.
+# CLI flag: -distributor.enable-start-timestamp
+[enable_start_timestamp: <boolean> | default = false]
+
 # The maximum number of active series per user, per ingester. 0 to disable.
 # CLI flag: -ingester.max-series-per-user
 [max_series_per_user: <int> | default = 5000000]

docs/configuration/v1-guarantees.md

@@ -115,6 +115,7 @@ Currently experimental features are:
 - Distributor/Ingester: Stream push connection
   - Enable stream push connection between distributor and ingester by setting `-distributor.use-stream-push=true` on Distributor.
   - Add `__type__` and `__unit__` labels to OTLP and remote write v2 requests (`-distributor.enable-type-and-unit-labels`)
+  - Handle StartTimestampMs (ST) for remote write v2 samples and histograms, using CreatedTimestamp (CT) as a fallback when ST is not set (`-distributor.enable-start-timestamp`)
 - Ingester: Series Queried Metric
   - Enable on Ingester via `-ingester.active-queried-series-metrics-enabled=true`
   - Set the time window to expose via metrics using `-ingester.active-queried-series-metrics-windows=2h`. At least 1 time window is required to expose the metric.