Enable PHP FFE exposure system tests

[leoromanovsky]

Motivation

PHP FFE exposure logging now lives in the native dd-trace-php/runtime path and needs the shared FFE exposure system-tests enabled so regressions are caught outside local dogfooding. The M1 evaluation PR only proves remote-config evaluation; this stacked PR adds the next validation layer for exposure emission and cache behavior.

Planning/reference doc: https://docs.google.com/document/d/1NvMfTpZWLBlFmEFNjdnlMyeVpy5l7KD8qujGFco6w2w/edit?tab=t.0

Changes

This PR enables tests/ffe/test_exposures.py for PHP at v1.20.0-dev. It intentionally does not enable FFE evaluation-metric tests; those belong to a separate stacked PR for dd-trace-php#3911.

The PR is stacked on #7003, which adds the PHP FFE scaffold and evaluation-test enablement. The PHP behavior being validated here is implemented in DataDog/dd-trace-php#3910.

Decisions

The scope is deliberately one manifest activation. The shared exposure tests already cover the required product behavior: event generation, multiple RC files, malformed/empty RC handling, same-subject deduplication, different-subject emission, allocation/variant changes, doLog=false, missing flags, and empty targeting keys.

The local proof uses a matched PHP 8.0 artifact with the default apache-mod-8.0 weblog. Earlier PHP 8.2 artifact attempts against the 8.0 weblog are not considered valid evidence because the extension ABI did not match the runtime.

Related PRs

• PHP runtime evaluation base: feat(ffe): add runtime-backed PHP feature flag evaluation dd-trace-php#3906
• PHP exposure implementation: Add FFE exposure emission dd-trace-php#3910
• System-tests PHP FFE scaffold/evaluations base: Enable PHP FFE evaluation system tests #7003
• Sidecar delivery/cache: feat(sidecar): forward FFE exposures to EVP proxy libdatadog#2026

Validation

Local build and run against dd-trace-php#3910 head 59d0ea178:

cd /Users/leo.romanovsky/go/src/github.com/DataDog/dd-trace-php-ffe-runtime-first
./tooling/bin/build-debug-artifact gnu-aarch64-8.0-nts \
  /Users/leo.romanovsky/go/src/github.com/DataDog/system-tests-pr-g-php-ffe-scaffold/binaries
passed

cd /Users/leo.romanovsky/go/src/github.com/DataDog/system-tests-pr-g-php-ffe-scaffold
DOCKER_HOST=unix:///Users/leo.romanovsky/.colima/default/docker.sock \
DOCKER_CONFIG=/tmp/system-tests-empty-docker-config \
./build.sh php -i weblog -w apache-mod-8.0
passed

DOCKER_HOST=unix:///Users/leo.romanovsky/.colima/default/docker.sock \
DOCKER_CONFIG=/tmp/system-tests-empty-docker-config \
TEST_LIBRARY=php \
./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION tests/ffe/test_exposures.py -vv
11 passed in 78.45s

Manifest formatting:

./format.sh
All good, the system-tests CI will be happy!

[datadog-prod-us1-4]

 

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 54 Pipeline jobs failed

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0 1     

🔧 Fix in code (Fix with Cursor).

13 failed tests due to assertion errors: expected 'expected', got 'default' in multiple tests.

🧪 1 Test failed

tests.ffe.test_dynamic_evaluation.Test_FFE_Flag_Parse_Error_Isolation.test_valid_flag_unaffected[apache-mod-7.0] from system_tests_suite   (Fix with Cursor)

AssertionError: Valid flag corrupted by malformed neighbour: expected 'expected', got 'default' (parse error on one flag must not abort processing of the whole config)
assert 'default' == 'expected'
  - expected
  + default

self = <tests.ffe.test_dynamic_evaluation.Test_FFE_Flag_Parse_Error_Isolation object at 0x7f9aad3660f0>

    def test_valid_flag_unaffected(self):
        assert self.r_valid.status_code == 200, f"Valid flag evaluation failed: {self.r_valid.text}"
        result = json.loads(self.r_valid.text)
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0-zts 1     

🔧 Fix in code (Fix with Cursor).

13 failed tests. Multiple tests failed due to assertion errors where expected values did not match actual values, resulting in 'default' being returned instead of the expected values.

🧪 1 Test failed

tests.ffe.test_dynamic_evaluation.Test_FFE_Flag_Parse_Error_Isolation.test_valid_flag_unaffected[apache-mod-7.0-zts] from system_tests_suite   (Fix with Cursor)

AssertionError: Valid flag corrupted by malformed neighbour: expected 'expected', got 'default' (parse error on one flag must not abort processing of the whole config)
assert 'default' == 'expected'
  - expected
  + default

self = <tests.ffe.test_dynamic_evaluation.Test_FFE_Flag_Parse_Error_Isolation object at 0x7f11ebe17fe0>

    def test_valid_flag_unaffected(self):
        assert self.r_valid.status_code == 200, f"Valid flag evaluation failed: {self.r_valid.text}"
        result = json.loads(self.r_valid.text)
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.1 1     

🔧 Fix in code (Fix with Cursor).

13 failed tests due to assertion errors about expected flag values. AssertionError: expected 'expected', got 'default' in multiple tests. Failure visible in tests/ffe/test_dynamic_evaluation.py:168 and others.

🧪 1 Test failed

tests.ffe.test_dynamic_evaluation.Test_FFE_Flag_Parse_Error_Isolation.test_valid_flag_unaffected[apache-mod-7.1] from system_tests_suite   (Fix with Cursor)

AssertionError: Valid flag corrupted by malformed neighbour: expected 'expected', got 'default' (parse error on one flag must not abort processing of the whole config)
assert 'default' == 'expected'
  - expected
  + default

self = <tests.ffe.test_dynamic_evaluation.Test_FFE_Flag_Parse_Error_Isolation object at 0x7f52d8d87d40>

    def test_valid_flag_unaffected(self):
        assert self.r_valid.status_code == 200, f"Valid flag evaluation failed: {self.r_valid.text}"
        result = json.loads(self.r_valid.text)
...

View all 54 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a27a887 | Docs | Datadog PR Page | Give us feedback!}