Add route-parameter-driven Playwright regression matrix for mock and real data scenarios

[DevOps-zhuang]

Background

We need a Playwright regression capability that validates whether the dashboard can be accessed successfully across multiple scope combinations using route parameters.

The application already allows route parameters to override .env defaults, for example:

• /enterprises/<enterprise>
• /orgs/<organization>
• /orgs/<organization>/teams/<team>
• /enterprises/<enterprise>/teams/<team>
• /orgs/<organization>?mock=true

This means .env should only be treated as the initial/default configuration, while the regression suite should validate multiple combinations by passing route parameters dynamically.

Goal

Introduce a configuration-driven Playwright regression matrix that can validate both:

1. Mock-data scenarios
2. Real-data scenarios when .env is set to non-mock mode

Scope

• Cover organization, enterprise, organization team, and enterprise team route combinations
• Validate that the basic dashboard page can be opened successfully for each configured scenario
• Validate that the page URL is correct and the route is handled without navigation errors
• Use a scenario/config table so multiple combinations can be added without duplicating test logic

Notes

• .env provides defaults only
• Route parameters are the primary mechanism for switching test scenarios
• Real-data runs should use configured scenario values rather than hardcoded single-environment assumptions

Expected outcome

A reusable regression framework for route-driven page accessibility checks across mock and real environments.

[DevOps-zhuang]

Requirement draft has been written in the working branch at docs/requirements/req-playwright-regression-matrix.md.

Key points captured in the requirement:

• .env is treated as default startup configuration only
• route parameters are the primary mechanism for switching regression scenarios
• the regression matrix must cover organization, enterprise, organization team, and enterprise team combinations
• both mock mode and real-data mode are in scope
• the framework must be configuration-driven so new scenarios can be added without duplicating Playwright specs
• browser coverage must include at least Chrome and Edge
• basic validation focuses on route correctness and successful dashboard page accessibility, not full metric-value correctness

The requirement also includes explicit acceptance criteria and non-functional requirements to keep implementation scope unambiguous.

[DevOps-zhuang]

Technical design draft has been created at docs/design/design-playwright-regression-matrix.md in the working branch.

Design decisions captured:

• .env remains default startup configuration only
• regression scenarios are selected primarily by route parameters
• mock and real-data modes share the same scenario schema and core Playwright spec
• mock mode uses repository-tracked scenarios and an isolated test server
• real mode uses a tracked example plus an ignored local scenario file or external file path
• browser strategy is split between CI-safe Chromium baseline and local/manual Chrome + Edge coverage
• no backend API changes are required; the design builds on the route override behavior already documented in README

The design also defines the scenario model, configuration source split, runtime environment variables, CI execution strategy, and high-level implementation task blocks.

[karpikpl]

Are you sure we don't already have that? My Playwright tests access the app using different scopes