Skip to content

Commit d1a1344

Browse files
docs: add feature flags in E2E tests section (#165)
- Add a new Feature flags in E2E tests section to the shared E2E testing guidelines. - Document the two categories of feature flags (remote/runtime and build-time/compile-time) with guidelines and examples for each. - Include general principles for testing both flag states, keeping flag logic explicit in helpers, and cleaning up flags after rollout. Fixes: [MMQA-1481](https://consensyssoftware.atlassian.net/browse/MMQA-1481?atlOrigin=eyJpIjoiNmFkNTAzY2I0NmVmNGFkOGEyNjE3MDA1ZGY1ZmYwNzgiLCJwIjoiaiJ9) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Low risk documentation-only changes; no runtime or test code behavior is modified. > > **Overview** > Adds a new **Feature flags in E2E tests** section to `docs/testing/e2e-testing.md`, clarifying how to test *remote/runtime* vs *build-time/compile-time* flags, including recommended override patterns and examples. > > Updates `docs/remote-feature-flags.md` to explain Extension’s production remote-flag fetch and E2E behavior via the feature flag registry, and replaces the embedded E2E override snippet with a link to the new E2E feature-flag guidelines. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit f7f987e. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent 4edef16 commit d1a1344

2 files changed

Lines changed: 161 additions & 16 deletions

File tree

docs/remote-feature-flags.md

Lines changed: 3 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -223,6 +223,8 @@ Your selector must include:
223223

224224
#### Extension
225225

226+
In production, MetaMask Extension fetches remote flags from the [client-config API](https://client-config.api.cx.metamask.io/v1/flags?client=extension&distribution=main&environment=prod) at runtime. During E2E tests, a global mock server (`test/e2e/mock-e2e.js`) reads from the [feature flag registry](https://github.com/MetaMask/metamask-extension/blob/main/test/e2e/feature-flags/feature-flag-registry.ts) instead of calling the real API. Each registry entry stores the flag's production default value, so tests reflect real-world behavior unless a specific test explicitly overrides a flag.
227+
226228
##### Local Feature Flag Override
227229

228230
- Developers can override `remoteFeatureFlag` values by defining them in `.manifest-overrides.json` and enable `MANIFEST_OVERRIDES=.manifest-overrides.json` in the `.metamaskrc.dist` locally.
@@ -259,22 +261,7 @@ Your selector must include:
259261

260262
##### B. E2E Test
261263

262-
Add the customized value in your test configuration:
263-
264-
```typescript
265-
await withFixtures({
266-
fixtures: new FixtureBuilder()
267-
.withMetaMetricsController({
268-
metaMetricsId: MOCK_META_METRICS_ID,
269-
participateInMetaMetrics: true,
270-
})
271-
.build(),
272-
manifestFlags: {
273-
remoteFeatureFlags: MOCK_CUSTOMIZED_REMOTE_FEATURE_FLAGS,
274-
},
275-
title: this.test?.fullTitle(),
276-
});
277-
```
264+
For detailed guidelines on handling remote (and build-time) feature flags in E2E tests — including the feature flag registry, override patterns, and general principles — see [Feature flags in E2E tests](testing/e2e-testing.md#feature-flags-in-e2e-tests).
278265

279266
#### Mobile
280267

docs/testing/e2e-testing.md

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,3 +206,161 @@ solution: replace UI steps that build up the application state with the FixtureB
206206
scenario: import Account using private key and remove imported account
207207
solution: replace UI steps that build up the application state with the FixtureBuilder
208208
```
209+
210+
## Feature flags in E2E tests
211+
212+
MetaMask uses two categories of feature flags, and each requires a different approach in E2E tests. Understanding the distinction helps you write tests that accurately reflect production behavior.
213+
214+
### Remote feature flags (runtime)
215+
216+
Remote feature flags are fetched at runtime from a configuration service. In production, the application calls a remote API to retrieve flag values. During E2E tests, a mock server or fixture intercepts these requests and returns controlled values instead.
217+
218+
Each client should maintain a **feature flag registry** — a central source of truth that maps every remote flag to its current production default value. The E2E mock layer reads from this registry so that tests run against production-accurate defaults by default, without calling the real API.
219+
220+
#### Guidelines
221+
222+
- **Default to production values.** Tests should use the same flag values that real users see unless the test is specifically verifying behavior behind a different flag state. The registry makes this automatic.
223+
- **Override only when needed.** When a test must exercise a non-default flag state, use the test framework's override mechanism (e.g. fixture builder methods or manifest flag overrides) rather than changing the registry itself.
224+
- **Register every flag.** Any remote flag referenced in application code should have a corresponding entry in the registry. CI checks can enforce this automatically by scanning for flag references and verifying they exist in the registry.
225+
- **Keep the registry up to date.** When a flag is fully rolled out or removed from the remote API, update or remove its registry entry. Stale entries lead to tests that no longer reflect production.
226+
- **No custom builds required.** Because remote flags are resolved at runtime, you do not need to create a special build to test different remote flag values. Override them at the test level.
227+
228+
#### Examples
229+
230+
✅ Registry entry with production default — tests automatically use this value:
231+
232+
```typescript
233+
redesignedConfirmations: {
234+
name: 'redesignedConfirmations',
235+
type: FeatureFlagType.Remote,
236+
productionDefault: true,
237+
status: FeatureFlagStatus.Active,
238+
},
239+
```
240+
241+
✅ Override a remote flag for a specific test without changing the registry:
242+
243+
```javascript
244+
// Override via fixture builder
245+
new FixtureBuilder()
246+
.withRemoteFeatureFlags({ redesignedConfirmations: false })
247+
.build();
248+
```
249+
250+
```javascript
251+
// Override via manifest flags in test options
252+
await withFixtures(
253+
{
254+
manifestFlags: {
255+
remoteFeatureFlags: { redesignedConfirmations: false },
256+
},
257+
},
258+
async () => {
259+
// test runs with the flag disabled
260+
},
261+
);
262+
```
263+
264+
❌ Modifying the registry to change a flag value for a single test:
265+
266+
```typescript
267+
// DON'T do this — it changes the default for all tests
268+
redesignedConfirmations: {
269+
name: 'redesignedConfirmations',
270+
productionDefault: false, // changed from true just for one test
271+
},
272+
```
273+
274+
### Build-time feature flags (compile-time)
275+
276+
Build-time feature flags are set during the build process and baked into the compiled output. They control which code paths are included in a given build. Changing a build-time flag requires creating a new build before running tests.
277+
278+
#### Guidelines
279+
280+
- **Create a dedicated test build.** To test with a build-time flag enabled, set the flag in the build configuration or pass it as an environment variable, then create a test build.
281+
- **Keep build-time flags separate from remote flags.** Do not conflate the two. A build-time flag controls what code ships; a remote flag controls runtime behavior of code that is already shipped.
282+
- **Document available flags.** Each client should document its build-time flags and how to enable them for test builds, so contributors know which flags exist and how to use them.
283+
284+
#### Examples
285+
286+
✅ Enable a build-time flag via environment variable, then run tests:
287+
288+
```bash
289+
# Create a test build with the flag enabled
290+
MULTICHAIN=1 yarn build:test
291+
292+
# Run E2E tests against that build
293+
yarn test:e2e
294+
```
295+
296+
✅ Enable a build-time flag via local configuration file:
297+
298+
```bash
299+
# In your local config file (e.g. .metamaskrc, .env, etc.)
300+
MULTICHAIN=1
301+
302+
# Then build and test as usual
303+
yarn build:test
304+
yarn test:e2e
305+
```
306+
307+
❌ Trying to override a build-time flag at the test level (this has no effect):
308+
309+
```javascript
310+
// DON'T do this — build-time flags are already compiled in
311+
new FixtureBuilder()
312+
.withBuildFlag({ MULTICHAIN: true }) // has no effect at runtime
313+
.build();
314+
```
315+
316+
### General principles
317+
318+
- **Test both states when possible.** For any flag that gates significant user-facing behavior, consider having tests for both the enabled and disabled states to prevent regressions in either path.
319+
- **Avoid flag-dependent test logic in shared helpers.** If a helper function behaves differently based on a flag, make the flag value an explicit parameter rather than reading it implicitly. This keeps tests predictable and easy to reason about.
320+
- **Clean up after rollout.** Once a feature flag is no longer needed (the feature is fully launched or removed), delete the flag references from application code, tests, and the registry. Leftover flags add confusion and maintenance burden.
321+
322+
#### Examples
323+
324+
✅ Testing both flag states explicitly:
325+
326+
```javascript
327+
describe('token approvals', () => {
328+
it('shows redesigned confirmation when flag is enabled', async () => {
329+
new FixtureBuilder()
330+
.withRemoteFeatureFlags({ redesignedConfirmations: true })
331+
.build();
332+
// assert redesigned UI is shown
333+
});
334+
335+
it('shows legacy confirmation when flag is disabled', async () => {
336+
new FixtureBuilder()
337+
.withRemoteFeatureFlags({ redesignedConfirmations: false })
338+
.build();
339+
// assert legacy UI is shown
340+
});
341+
});
342+
```
343+
344+
✅ Making flag dependency explicit in a helper:
345+
346+
```javascript
347+
function confirmTransaction(driver, { isRedesigned }) {
348+
if (isRedesigned) {
349+
return driver.clickElement('[data-testid="confirm-redesigned"]');
350+
}
351+
return driver.clickElement('[data-testid="confirm-legacy"]');
352+
}
353+
```
354+
355+
❌ Reading the flag implicitly inside a shared helper:
356+
357+
```javascript
358+
// DON'T do this — the helper's behavior depends on hidden global state
359+
function confirmTransaction(driver) {
360+
const flags = getRemoteFeatureFlags();
361+
if (flags.redesignedConfirmations) {
362+
return driver.clickElement('[data-testid="confirm-redesigned"]');
363+
}
364+
return driver.clickElement('[data-testid="confirm-legacy"]');
365+
}
366+
```

0 commit comments

Comments
 (0)