feat(profiling): expose profiler settings on each uploaded profile

[taegyunkim]

Description

Publishes a snapshot of the effective profiler configuration on each uploaded profile via the upload event's user-visible info channel (ddog_prof_Exporter_send_blocking's optional_info_json argument). Settings are nested under the info.profiler.settings header, matching dd-trace-go and dd-trace-php conventions. Each setting is keyed by its dotted config path relative to ProfilingConfig (e.g. stack.enabled, upload_interval).

Per the Confluence guidance, the internal channel is auto-indexed but not user-visible, while info is both indexed and visible. The profiling configuration knobs are already public (DDConfig entries shipped in the library source), and we already export profiler_config tag, which has some of those information. Adding more to profiler_config doesn't help with searching such bits of configurations. So we simply export them as info field.

Example shape of the uploaded info JSON:

{
  "profiler": {
    "settings": {
      "enabled": true,
      "upload_interval": 60.0,
      "stack.enabled": true,
      "stack.adaptive_sampling": true,
      "stack.adaptive_sampling_target_overhead": 1.0,
      "stack.adaptive_sampling_max_interval": 1000000,
      "stack.fast_copy": false,
      "stack.max_threads": 25,
      "exception.enabled": false,
      "exception.sampling_interval": 100,
      "lock.enabled": true,
      "memory.enabled": true,
      "heap.enabled": true,
      "...": "..."
    }
  }
}

Today, report_configuration() is the only path that surfaces profiler config externally, but it skips every entry declared with private=True. That excludes precisely the knobs that matter for interpreting a profile (stack.adaptive_sampling, stack.adaptive_sampling_target_overhead, stack.adaptive_sampling_max_interval, stack.fast_copy, stack.max_threads, etc.). Additionally, telemetry is a process-level aggregate, not per-profile.

Changes:

• ddtrace/internal/settings/_core.py — adds DDConfig.dump_settings() (a method on the config) that walks the tree including private=True items and emits a JSON-safe {dotted_name: value} dict with keys relative to the config root (no __prefix__ baked in). Wrapping is a channel concern, handled at the call site.
• dd_wrapper/include/profiler_state.hpp — adds a process-singleton profiler_settings_info_json string. Must live on ProfilerState rather than ProfilerStats because UploaderBuilder::build() does a std::swap of ProfilerStats on every upload, which would wipe the snapshot.
• dd_wrapper/src/ddup_interface.cpp — ddup_set_profiler_settings_json stores the JSON object verbatim on ProfilerState. Empty objects and inputs without matching braces are dropped silently.
• dd_wrapper/src/uploader.cpp — builds a ddog_CharSlice from ProfilerState::profiler_settings_info_json and passes it as optional_info_json to ddog_prof_Exporter_send_blocking. Also writes a sibling *.info.json next to *.internal_metadata.json in export_to_file, used when output_filename is set for tests.
• dd_wrapper/include/ddup_interface.hpp, ddtrace/internal/datadog/profiling/ddup/_ddup.pyx, _ddup.pyi — Cython binding set_profiler_settings_json(settings_json).
• ddtrace/profiling/profiler.py — wraps the settings dict as {"profiler": {"settings": ...}} and calls the binding once after ddup.start() with json.dumps(...).

JIRA: PROF-14617

Cross-language references:

• dd-trace-go profilerInfo (upload.go:141-143)
• dd-trace-php info JSON (uploader.rs:90-97)

Testing

• New unit tests in tests/profiling/test_profiling_config.py::TestDumpSettings: verify private keys are included, keys are bare dotted paths (no DD_, _DD_, or dd.profiling. prefix), all values are JSON-serializable, and env-var overrides are reflected.
• End-to-end smoke test (manual): three back-to-back ddup.upload() calls in the same process. All three resulting *.info.json files carry info.profiler.settings with 35 entries and the expected env-override values applied (stack.adaptive_sampling = false after _DD_PROFILING_STACK_ADAPTIVE_SAMPLING_ENABLED=0). The *.internal_metadata.json files are unaffected, with only their original stats fields. Payload size went from ~1596 to ~1140 bytes per profile after dropping the redundant dd.profiling. prefix.
• ruff format and ruff check clean on modified Python files.

Risks

• Low. The settings ride on a separate info channel that was previously nullptr; no existing field of internal_metadata.json changes. If dump_settings or json.dumps raises, the call site swallows the exception and logs at debug level, so a serialization bug can't take down profiling startup.
• Telemetry behavior is unchanged (still skips private=True).
• The C++ side trusts json.dumps to produce a valid compact JSON object; it only does a basic front() == '{' && back() == '}' sanity check before storing. If a future caller passes a non-object payload, the field is cleared and optional_info_json is set to nullptr (the previous behavior).

Additional Notes

• Settings names follow the dotted Python config path relative to ProfilingConfig (stack.adaptive_sampling, upload_interval, ...). The info.profiler.settings wrapping already carries the scope, so the dd.profiling. prefix is dropped to save payload.
• The snapshot is one-shot at startup, mirroring dd-trace-go. Runtime-mutable values like the effective sampling interval are still exposed via the existing sampling_interval_us field in internal_metadata.json.
• Storage on ProfilerState (not ProfilerStats) is deliberate: ProfilerStats is std::swap-ped on every upload (UploaderBuilder::build()), so a snapshot stored there would survive only one profile.

See this screenshot for how it shows up on the UI

PROMPTS.md

> Do we log anything or upload instrumentation telemetry whether adaptive sampling for profiling is turned off or not?

> How are the sampling interval related things are reported?

> I was looking at profile pages from other profiles, for example go, and it seems to output things like
>
> Profiler
> activation: manual
> settings.agentless: false
> ...
> ssi.mechanism: none
>
> which would have been built from some code in ~/dd/dd-trace-go/profiler/upload.go,
> especially the lines related to profilerInfo struct and relevant code
>
> I believe it would be also nice to move some of our config vars, especially those
> in ddtrace/internal/settings/profiling.py and some of those internal settings
> related to stack v2, what do you think?

> That sounds reasonable, write plan.md and go ahead to implement this

> So this is basically dumping everything at once, i.e. everything is under a single
> 'profiler_settings' key, then it wouldn't be easy to filter profiles using a single
> configuration knob. Each profiling config item should be a single key, for example
> dd.profiling.stack.enabled: true, dd.profiling.stack.adaptive_sampling_enabled: true,
> dd.profiling.lock.enabled: true, ... like these

> In this approach, we unescape json everytime upload happens, which is unnecessary.
> We should store the unescaped json key value pair strings, and just pass that when
> uploading.

> It would be more appropriate to update ddtrace/internal/settings/_core.py instead of
> ddtrace/internal/telemetry/__init__.py to define dump_settings

> According to https://datadoghq.atlassian.net/wiki/spaces/PROF/pages/4004775041/Add+extra+attributes+data+info+to+profiles
> internal field in the event.json are also auto-indexed and searchable but not visible
> to the users. Given that the configurations knobs are already public, part of the code,
> let's just export via the info field.

> Looking at dd-trace-php uploader.rs and dd-trace-go upload.go, it looks customary to
> have these headers (info.profiler), so for python, we'd also use profiler header, and
> could also trim dd.profiling. prefixes given that they're all common, and save a
> little bit of payload

PLAN.md

# PROF-14617 — Expose profiler settings on each profile

JIRA: https://datadoghq.atlassian.net/browse/PROF-14617
Branch: taegyunkim/prof-14617-profile-settings-metadata

## Approach (final)

1. Python: DDConfig.dump_settings() walks the tree (including private items)
   and emits a flat {dotted_name: value} dict. Keys are relative to the
   config root (no __prefix__ baked in). Lives on DDConfig in
   ddtrace/internal/settings/_core.py.
2. C++: ProfilerState owns a profiler_settings_info_json string (full JSON
   object). The setter stores the value verbatim after a basic shape check.
3. Uploader builds a ddog_CharSlice from that string and passes it as
   optional_info_json to ddog_prof_Exporter_send_blocking. The `info`
   channel is user-visible and auto-indexed on the backend.
4. Storage on ProfilerState (not ProfilerStats) because ProfilerStats is
   std::swap-ped on every upload in UploaderBuilder::build().
5. Binding: ddup_set_profiler_settings_json (interface + Cython).
6. Startup: profiler.py wraps the dict as {"profiler": {"settings": ...}}
   and calls the binding once after ddup.start() with json.dumps(...).
   Matches dd-trace-go and dd-trace-php conventions.
7. Tests + release note (category: other).

## Out of scope

- Reporting settings for the tracer (only profiler today).
- Refreshing settings mid-process.
- Changing telemetry's behavior. report_configuration() keeps skipping
  private items as before.

[cit-pr-commenter-54b7da]

Codeowners resolved as

ddtrace/internal/datadog/profiling/dd_wrapper/include/ddup_interface.hpp  @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/include/profiler_state.hpp  @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/src/ddup_interface.cpp    @DataDog/profiling-python
ddtrace/internal/datadog/profiling/dd_wrapper/src/uploader.cpp          @DataDog/profiling-python
ddtrace/internal/datadog/profiling/ddup/_ddup.pyi                       @DataDog/profiling-python
ddtrace/internal/datadog/profiling/ddup/_ddup.pyx                       @DataDog/profiling-python
ddtrace/internal/settings/_core.py                                      @DataDog/apm-core-python
ddtrace/profiling/profiler.py                                           @DataDog/profiling-python
releasenotes/notes/profiling-per-profile-settings-metadata-b5aeab16b998e71f.yaml  @DataDog/apm-python
tests/profiling/test_profiling_config.py                                @DataDog/profiling-python

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0508bb31f3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[datadog-datadog-prod-us1-2]

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 8ae8640 | Docs | Datadog PR Page | Give us feedback!}

[KowalskiThomas]

Overall LGTM, can stamp once you confirm the testing in staging bit!