Skip to content

Enable PHP FFE evaluation system tests#7003

Draft
leoromanovsky wants to merge 5 commits into
mainfrom
leo.romanovsky/pr-g-php-ffe-scaffold
Draft

Enable PHP FFE evaluation system tests#7003
leoromanovsky wants to merge 5 commits into
mainfrom
leo.romanovsky/pr-g-php-ffe-scaffold

Conversation

@leoromanovsky
Copy link
Copy Markdown
Contributor

@leoromanovsky leoromanovsky commented May 21, 2026

Motivation

DataDog/dd-trace-php#3906 adds the PHP runtime path for Feature Flags and Experimentation evaluation through live FFE_FLAGS Remote Config and the native libdatadog evaluator. PHP should run the shared FFE evaluation system tests so that the runtime work has both end-to-end Remote Config coverage and canonical JSON fixture coverage before review and merge.

Reference design doc: https://docs.google.com/document/d/1NvMfTpZWLBlFmEFNjdnlMyeVpy5l7KD8qujGFco6w2w/edit?tab=t.0

Changes

This PR enables the existing end-to-end tests/ffe/test_dynamic_evaluation.py suite for PHP starting at v1.20.0-dev and adds the minimal PHP weblog /ffe endpoint needed by that shared suite. The endpoint calls the DDTrace\FeatureFlags\Client details API exposed by the PHP runtime work, normalizes the request shape used by system-tests, and returns provider-not-ready defaults when the runtime is unavailable.

This PR also enables the parametric canonical JSON fixture evaluation suite for PHP. The PHP parametric server now implements /ffe/start and /ffe/evaluate using the same Datadog client API.

Decisions

The shared parametric test still requires the test-agent to observe an acknowledged FFE_FLAGS Remote Config update. For PHP specifically, the test also passes the canonical UFC fixture into /ffe/start; the PHP parametric server is a single long-lived CLI request, so this seeds that process with the same fixture via the extension testing loader before the real libdatadog evaluator is exercised. That keeps the parametric suite focused on evaluator correctness while the end-to-end suite validates live RC delivery into PHP request handling.

The scope is deliberately evaluation-only. Exposure forwarding and feature_flag.evaluations metrics remain disabled here so they can be enabled in separate stacked PRs after their runtime implementations are ready.

Verification

Local validation was run against locally built dd-trace-php artifacts from DataDog/dd-trace-php#3906, not a released tracer.

Build inputs:

  • system-tests commit: 224c241208532a6b16b1ea907e4638516201a50d
  • dd-trace-php PR 3906 head: 4cf261de35a6915711a7998d60f921bd8d14d502
  • local dd-trace-php checkout: /Users/leo.romanovsky/go/src/github.com/DataDog/dd-trace-php-pr3906-review

Proof from local runs:

$ CI=true PYTEST_XDIST_AUTO_NUM_WORKERS=4 DOCKER_CONFIG=/tmp/codex-docker-no-creds DOCKER_HOST=unix://$HOME/.colima/default/docker.sock DOCKER_BUILDKIT=1 TEST_LIBRARY=php ./run.sh PARAMETRIC --skip-parametric-build tests/parametric/test_parametric_endpoints.py::Test_Parametric_FFE_Start tests/parametric/test_ffe/test_dynamic_evaluation.py::Test_Feature_Flag_Dynamic_Evaluation -q
26 passed in 55.42s
$ CI=true DOCKER_CONFIG=/tmp/codex-docker-no-creds DOCKER_HOST=unix://$HOME/.colima/default/docker.sock DOCKER_BUILDKIT=1 TEST_LIBRARY=php WEBLOG_VARIANT=apache-mod-8.0 ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION tests/ffe/test_dynamic_evaluation.py -q
================================= test context =================================
Scenario: FEATURE_FLAGGING_AND_EXPERIMENTATION
Logs folder: ./logs_feature_flagging_and_experimentation
Starting containers...
Agent: 7.79.0
Backend: datad0g.com
Library: php@1.20.0
Weblog variant: apache-mod-8.0
Weblog system: Linux weblog 6.8.0-100-generic #100-Ubuntu SMP PREEMPT_DYNAMIC Tue Jan 13 16:39:21 UTC 2026 aarch64 GNU/Linux

tests/ffe/test_dynamic_evaluation.py .......

7 passed in 49.23s

Supporting commands run locally:

  • DOCKER_CONFIG=/tmp/codex-docker-no-creds DOCKER_HOST=unix://$HOME/.colima/default/docker.sock ./format.sh: passed
  • ./tooling/bin/build-debug-artifact gnu-aarch64-8.2-nts /Users/leo.romanovsky/go/src/github.com/DataDog/system-tests-pr-g-php-ffe-scaffold/binaries from the PR 3906 checkout: passed
  • ./tooling/bin/build-debug-artifact gnu-aarch64-8.0-nts /Users/leo.romanovsky/go/src/github.com/DataDog/system-tests-pr-g-php-ffe-scaffold/binaries from the PR 3906 checkout: passed
  • CI=true DOCKER_CONFIG=/tmp/codex-docker-no-creds DOCKER_HOST=unix://$HOME/.colima/default/docker.sock DOCKER_BUILDKIT=1 ./build.sh php -w apache-mod-8.0 -i weblog: passed

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 21, 2026

CODEOWNERS have been resolved as:

utils/build/docker/php/common/ffe.php                                   @DataDog/apm-php @DataDog/system-tests-core
docs/understand/weblogs/end-to-end_weblog.md                            @DataDog/system-tests-core
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
tests/parametric/test_ffe/test_dynamic_evaluation.py                    @DataDog/feature-flagging-and-experimentation-sdk @DataDog/system-tests-core
utils/build/docker/php/common/rewrite-rules.conf                        @DataDog/apm-php @DataDog/system-tests-core
utils/build/docker/php/parametric/server.php                            @DataDog/apm-php @DataDog/system-tests-core
utils/docker_fixtures/_test_clients/_test_client_parametric.py          @DataDog/system-tests-core

@leoromanovsky leoromanovsky changed the title Add PHP FFE system-test scaffold Enable PHP FFE parametric system tests May 22, 2026
@datadog-prod-us1-4
Copy link
Copy Markdown

datadog-prod-us1-4 Bot commented May 22, 2026

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 19b9c41 | Docs | Datadog PR Page | Give us feedback!

@leoromanovsky leoromanovsky force-pushed the leo.romanovsky/pr-g-php-ffe-scaffold branch from 4ca7546 to 984548d Compare May 28, 2026 02:54
@leoromanovsky leoromanovsky changed the title Enable PHP FFE parametric system tests Enable PHP FFE evaluation system tests May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant