feat(sidecar): forward FFE evaluation metrics to OTLP intake#2052
Open
leoromanovsky wants to merge 3 commits into
Open
feat(sidecar): forward FFE evaluation metrics to OTLP intake#2052leoromanovsky wants to merge 3 commits into
leoromanovsky wants to merge 3 commits into
Conversation
Contributor
📚 Documentation Check Results📦
|
Contributor
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
Contributor
🔒 Cargo Deny Results📦
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2052 +/- ##
==========================================
+ Coverage 72.90% 73.07% +0.16%
==========================================
Files 460 462 +2
Lines 76396 76939 +543
==========================================
+ Hits 55696 56220 +524
- Misses 20700 20719 +19
🚀 New features to boost your workflow:
|
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 96d9a7b | Docs | Datadog PR Page | Give us feedback! |
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
This was referenced May 28, 2026
leoromanovsky
requested review from
dd-oleksii and
typotter
and removed request for
a team
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
PHP FFE evaluation metrics need a native path for aggregation, OTLP encoding, and delivery without building PHP OTLP writer/transport machinery. The shared design doc is the cross-PR reference: https://docs.google.com/document/d/1NvMfTpZWLBlFmEFNjdnlMyeVpy5l7KD8qujGFco6w2w/edit?tab=t.0
This PR is metric-only. Exposures remain in #2026 so reviewers can evaluate OTLP metric delivery independently from exposure cache semantics.
Changes
This adds caller-driven FFE evaluation metric sidecar actions and OTLP export for
feature_flag.evaluations.The reusable FFE-domain pieces now live in
datadog-ffebehind theevaluation-metricsfeature: evaluation metric input types, metric attribute normalization, aggregation by matching attribute sets, and OTLP/protobuf payload encoding.datadog-sidecarkeeps only sidecar-specific work: parsing the configured endpoint URL, building the HTTP request, applying the timeout, logging delivery failures, and integrating with sidecar lifecycle/actions.The PHP companion PR uses this from native/C code for raw
DDTrace\ffe_evaluatecalls and from a thin PHP OpenFeature adapter for final OpenFeature-aware results. PHP does not aggregate, encode, or transport OTLP payloads.Current PHP MVP path:
flowchart LR Eval["PHP evaluation<br/>raw API or OpenFeature adapter"] Record["PHP tracer native call<br/>record typed evaluation metric"] Action["sidecar action<br/>record FFE evaluation metrics"] Domain["datadog-ffe<br/>feature: evaluation-metrics<br/>attributes + aggregation + OTLP encoder"] Sidecar["shared sidecar<br/>metric flush lifecycle"] Collector["OTLP endpoint<br/>Agent or local collector"] Intake["feature_flag.evaluations"] Eval --> Record Record --> Action Action --> Domain Domain --> Sidecar Sidecar --> Collector Collector --> IntakeFuture Python/Ruby connection:
flowchart LR PyToday["dd-trace-py today<br/>OpenFeature hook + host metric writer"] RbToday["dd-trace-rb today<br/>OpenFeature hook + host metric writer"] PyFuture["dd-trace-py future<br/>explicit native opt-in"] RbFuture["dd-trace-rb future<br/>explicit native opt-in"] Native["libdatadog caller-driven<br/>FFE metric action"] Shared["shared sidecar<br/>aggregation + OTLP delivery"] Otlp["OTLP endpoint"] PyToday -. "current host metric path" .-> Otlp RbToday -. "current host metric path" .-> Otlp PyFuture -. "after ownership switch" .-> Native RbFuture -. "after ownership switch" .-> Native Native --> Shared Shared --> OtlpThe future Python/Ruby arrows are intentionally not active behavior in this PR. They show the reusable target for a later migration while preserving today's host-language metric writers.
Why Python/Ruby do not double count today:
feature_flag.evaluationsas a side effect.Reference implementation check: dd-trace-java's canonical metric path is OpenFeature hook based. Java's
ProvidercreatesFlagEvalMetricsand returns aFlagEvalHook; the hook runs infinallyAfter, reads the final OpenFeatureFlagEvaluationDetailsincluding flag key, variant, reason, error code, and allocation metadata, and records onefeature_flag.evaluationscounter. Application code only calls OpenFeature; it does not call a metric API.PHP mirrors that canonical OpenFeature shape. The PHP OpenFeature provider disables raw native metric recording while it asks the native evaluator for an assignment, then records exactly one final OpenFeature-aware metric through the Datadog-owned recorder. The raw Datadog PHP client has no direct Java equivalent, but it keeps the same SDK-owned ergonomics: normal evaluation APIs record one native metric per evaluation internally. For future Python/Ruby migration, the same rule applies: either keep the existing host-language OpenFeature metric hook, or switch ownership to the native recorder and disable/bypass the host metric writer for those evaluations.
Decisions
No telemetry is emitted automatically from shared libdatadog evaluator calls. SDKs must explicitly enqueue FFE telemetry actions. This avoids double counting for Python/Ruby, which currently log feature-flag telemetry in host-language code.
Evaluation metrics intentionally count evaluations and do not use exposure-cache deduplication semantics.
Future Python/Ruby migration must be an ownership switch, not an additional writer. If those SDKs opt into this native metric path, their host-language OpenFeature metric writers must stop recording the same evaluations.
Validation
Current head (
96d9a7bae) local validation:Results: datadog-ffe metric tests passed (2 passed), sidecar metric tests passed (6 passed), default datadog-ffe check passed, sidecar FFI check passed, fmt check passed with only the repo stable-rustfmt warnings.
Prior downstream PHP behavior validation before the reusable-crate refactor, from DataDog/dd-trace-php#3911 using this PR at
1f1fca439:System-tests downstream validation:
Result: 17 passed in 81.26 seconds.
Related PRs: DataDog/dd-trace-php#3906, DataDog/dd-trace-php#3911, #2026, DataDog/system-tests#7033.