Skip to content

Add span tags to AI Guard spans for Anomaly Detection#5752

Open
y9v wants to merge 8 commits into
masterfrom
add-request-tags-to-ai-guard-span
Open

Add span tags to AI Guard spans for Anomaly Detection#5752
y9v wants to merge 8 commits into
masterfrom
add-request-tags-to-ai-guard-span

Conversation

@y9v
Copy link
Copy Markdown
Member

@y9v y9v commented May 13, 2026

What does this PR do?
This PR adds tags that are required for Anomaly Detection to AI Guard spans: ai_guard.http.useragent, ai_guard.http.client_ip, and ai_guard.network.client.ip.

Since AI Guard has no access to Rack env, we have to stash those attributes in the active trace under _dd.ai_guard prefix, set them on AI Guard span when AI Guard evaluation runs, and remove them from the active trace.

Out of scope for this PR:

  • Fixing the signature for Datadog::Tracing.active_trace - since it will require review from tracing team.
  • ai_guard.usr.id and ai_guard.session_id tags - this requires code changes for AppSec, and needs to be well thought through

Motivation:
We want to support Anomaly Detection in Ruby AI Guard SDK.

Change log entry
No. The follow-up PR with user and session IDs will have a changelog entry.

Additional Notes:
APPSEC-62451

How to test the change?
CI and manual testing.

@y9v y9v self-assigned this May 13, 2026
@y9v y9v requested review from a team as code owners May 13, 2026 13:49
@dd-octo-sts dd-octo-sts Bot added the ai-guard label May 13, 2026
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 13, 2026

Typing analysis

Note: Ignored files are excluded from the next sections.

steep:ignore comments

This PR introduces 6 steep:ignore comments, and clears 3 steep:ignore comments.

steep:ignore comments (+6-3)Introduced:
lib/datadog/ai_guard/contrib/rack/request_middleware.rb:43
lib/datadog/ai_guard/contrib/rack/request_middleware.rb:59
lib/datadog/ai_guard/contrib/rack/request_middleware.rb:71
lib/datadog/ai_guard/contrib/rack/request_middleware.rb:77
lib/datadog/ai_guard/evaluation.rb:34
lib/datadog/ai_guard/evaluation.rb:84
Cleared:
lib/datadog/ai_guard/contrib/rack/request_middleware.rb:49
lib/datadog/ai_guard/evaluation.rb:29
lib/datadog/ai_guard/evaluation.rb:79

@datadog-official
Copy link
Copy Markdown

datadog-official Bot commented May 13, 2026

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 97.09% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1c93f9e | Docs | Datadog PR Page | Give us feedback!

Comment thread lib/datadog/ai_guard/contrib/rack/request_middleware.rb Outdated
Comment thread lib/datadog/ai_guard/contrib/rack/request_middleware.rb Outdated
Comment thread lib/datadog/ai_guard/contrib/rack/request_middleware.rb
Comment thread lib/datadog/ai_guard/contrib/rack/request_middleware.rb Outdated
Comment thread lib/datadog/ai_guard/contrib/rack/request_middleware.rb Outdated
@y9v y9v force-pushed the add-request-tags-to-ai-guard-span branch from 080a84a to cd71108 Compare May 18, 2026 18:35
@y9v y9v requested a review from Strech May 18, 2026 18:36
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented May 18, 2026

Benchmarks

Benchmark execution time: 2026-05-28 08:53:15

Comparing candidate commit 1c93f9e in PR branch add-request-tags-to-ai-guard-span with baseline commit b1a5c02 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Comment thread lib/datadog/ai_guard/contrib/rack/request_middleware.rb Outdated
Comment thread lib/datadog/ai_guard/contrib/rack/request_middleware.rb Outdated
Comment thread lib/datadog/ai_guard/ext.rb
@y9v y9v force-pushed the add-request-tags-to-ai-guard-span branch from ad91751 to de1d85b Compare May 22, 2026 14:31
@y9v y9v requested a review from Strech May 26, 2026 08:10
Copy link
Copy Markdown
Member

@Strech Strech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have 2 minor concerns to raise and it would be good to know how complex it to type with steep as most of the methods excluded

Comment thread lib/datadog/ai_guard/contrib/rack/request_middleware.rb Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants