Skip to content

Commit 29ca2f9

Browse files
openshift-merge-bot[bot]openshift-merge-bot[bot]
authored
Merge pull request #649 from etmurasaki/ou-1076
OU-1076: monitoring-plugin E2E testing instructions to AGENTS.md
2 parents 24e74e8 + 83aa06f commit 29ca2f9

6 files changed

Lines changed: 574 additions & 122 deletions

File tree

‎AGENTS.md‎

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,14 +113,52 @@ make lint-frontend
113113
make lint-backend
114114
make test-translations
115115
make test-backend
116-
# Run cypress tests (see web/cypress/README.md)
116+
# future slash command for test execution
117117
```
118118

119119
### PR Requirements:
120120
- **Title format**: `[JIRA_ISSUE]: Description`
121121
- **Testing**: All linting and tests must pass
122122
- **Translations**: Ensure i18next keys are properly added
123123

124+
### Cypress E2E Testing
125+
126+
#### Overview
127+
The Monitoring Plugin uses Cypress for comprehensive End-to-End (E2E) testing to ensure functionality across both the core **monitoring-plugin** (managed by CMO) and the **monitoring-console-plugin** (managed by COO). Our test suite covers test scenarios including alerts, metrics, dashboards, and integration with Virtualization and Fleet Management (ACM).
128+
129+
**Key Testing Documentation:**
130+
- **Setup & Configuration**: `web/cypress/README.md` - Environment variables, installation, troubleshooting
131+
- **Testing Guide**: `web/cypress/CYPRESS_TESTING_GUIDE.md` - Test architecture, creating tests, workflows
132+
- **Test Catalog**: `web/cypress/E2E_TEST_SCENARIOS.md` - Complete list of all test scenarios
133+
134+
#### When to Create New Cypress Tests
135+
136+
You should create new Cypress tests when:
137+
138+
1. **Adding New Features**: Any new UI feature requires corresponding E2E tests
139+
2. **Fixing Bugs**: Bug fixes should include tests to prevent regression
140+
3. **Modifying Existing Features**: Changes to existing functionality require test updates
141+
142+
#### Quick Test Commands
143+
144+
```bash
145+
cd web/cypress
146+
147+
# Run all regression tests
148+
npm run cypress:run --spec "cypress/e2e/**/regression/**"
149+
150+
# Run BVT (Build Verification Tests)
151+
npm run cypress:run --spec "cypress/e2e/monitoring/00.bvt_admin.cy.ts"
152+
153+
# Run COO tests
154+
npm run cypress:run --spec "cypress/e2e/coo/*.cy.ts"
155+
156+
# Interactive mode
157+
npm run cypress:open
158+
```
159+
160+
For detailed testing instructions, see `web/cypress/CYPRESS_TESTING_GUIDE.md`
161+
124162
### Release Pipeline:
125163
- **Konflux**: Handles CI/CD and release automation
126164
- **CMO releases**: Follow OpenShift release cycles

‎CLAUDE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
AGENTS.md

‎web/cypress/CYPRESS_TESTING_GUIDE.md‎

Lines changed: 254 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,254 @@
1+
# Cypress Testing Guide - Monitoring Plugin
2+
3+
> **Complete guide for developers and AI agents on Cypress E2E testing**
4+
5+
---
6+
7+
## Table of Contents
8+
- [Quick Start](#quick-start)
9+
- [Test Architecture](#test-architecture)
10+
- [Creating Tests](#creating-tests)
11+
- [Running Tests](#running-tests)
12+
- [Troubleshooting](#troubleshooting)
13+
14+
---
15+
16+
## Quick Start
17+
18+
### Prerequisites
19+
- Node.js >= 18
20+
- OpenShift cluster with kubeconfig
21+
- Environment variables configured
22+
23+
### 30-Second Setup
24+
```bash
25+
cd web/cypress
26+
source ./configure-env.sh # Interactive configuration
27+
npm install # Install dependencies
28+
npm run cypress:open # Start testing
29+
```
30+
31+
**For detailed setup instructions and environment configuration, see [README.md](README.md)**
32+
33+
---
34+
35+
## Test Architecture
36+
37+
### 3-Layer Organization
38+
39+
The Monitoring Plugin uses a 3-layer architecture for test organization:
40+
41+
```
42+
┌─────────────────────────────────────────────────┐
43+
│ Layer 3: E2E Test Files │
44+
│ (cypress/e2e/) │
45+
│ - Call support scenarios │
46+
│ - Specify perspective (Administrator, etc.) │
47+
└────────────────┬────────────────────────────────┘
48+
│ imports
49+
┌────────────────▼────────────────────────────────┐
50+
│ Layer 2: Support Scenarios │
51+
│ (cypress/support/monitoring or perses │
52+
│ - Reusable test scenarios │
53+
│ - Work across multiple perspectives │
54+
│ - Export functions with perspective parameter │
55+
└────────────────┬────────────────────────────────┘
56+
│ uses
57+
┌────────────────▼────────────────────────────────┐
58+
│ Layer 1: Page Object Views │
59+
│ (cypress/views/) │
60+
│ - Reusable UI actions │
61+
│ - Navigation, clicks, assertions │
62+
│ - Use data-test attributes │
63+
└─────────────────────────────────────────────────┘
64+
```
65+
66+
### File Structure
67+
68+
```
69+
cypress/
70+
├── e2e/
71+
│ ├── monitoring/ # Core monitoring tests (Administrator)
72+
│ │ ├── 00.bvt_admin.cy.ts
73+
│ │ └── regression/
74+
│ ├── coo/ # COO-specific tests
75+
│ │ ├── 01.coo_bvt.cy.ts
76+
│ │ └── 02.acm_alerting_ui.cy.ts
77+
│ └── virtualization/ # Integration tests (Virtualization)
78+
├── support/
79+
│ ├── monitoring/ # Reusable test scenarios
80+
│ │ ├── 01.reg_alerts.cy.ts
81+
│ │ ├── 02.reg_metrics.cy.ts
82+
│ │ └── 03.reg_legacy_dashboards.cy.ts
83+
│ ├── perses/ # COO/Perses scenarios
84+
│ └── commands/ # Custom Cypress commands
85+
└── views/ # Page object models (reusable actions)
86+
```
87+
88+
**Benefits**:
89+
- Test scenarios reusable across Administrator, Virtualization, and Fleet Management perspectives
90+
- Page actions separated from test logic for better maintainability
91+
- UI changes only require updating views, not individual tests
92+
93+
---
94+
95+
## Creating Tests
96+
97+
### Workflow
98+
99+
1. **Layer 1 - Views**: Check/add page actions in `cypress/views/`
100+
- Under `views/` folder, find pre-defined actions per page
101+
- If none fits your needs, add new ones
102+
103+
2. **Layer 2 - Support**: Add test scenarios to `cypress/support/monitoring/`
104+
- Add test scenarios to cypress files under `support/` folder
105+
- Make scenarios reusable across perspectives (Administrator, Virtualization, Fleet Management)
106+
- If it is not applicable, in some cases for Incidents or Fleet Management, test scenarios will be written directly into Layer 3
107+
108+
3. **Layer 3 - E2E**: Verify e2e files call your scenario (usually pre-configured)
109+
- Administrator: `e2e/monitoring/`
110+
- Virtualization: `e2e/virtualization/`
111+
- Fleet Management: `e2e/coo/` (for ACM)
112+
113+
### Example: Support Scenario Structure
114+
115+
```typescript
116+
// In support/monitoring/01.reg_alerts.cy.ts
117+
import { nav } from '../../views/nav';
118+
import { silencesListPage } from '../../views/silences-list-page';
119+
120+
export const runAlertTests = (perspective: string) => {
121+
describe(`${perspective} perspective - Alerting > Alerts page`, () => {
122+
123+
it('should filter alerts by severity', () => {
124+
// Use page object actions from views/
125+
silencesListPage.filter.byName('test-alert');
126+
silencesListPage.rows.shouldBe('test-alert', 'Active');
127+
});
128+
});
129+
};
130+
```
131+
132+
### When to Create New Tests
133+
134+
| Scenario | Action |
135+
|----------|--------|
136+
| New UI feature | Create new test scenario in support/ |
137+
| Bug fix | Add test case to existing support file |
138+
| Component update | Update existing test scenarios |
139+
| New Perses feature | Create new test scenario in support/ |
140+
| ACM integration | Add test in e2e/coo/ |
141+
142+
### Best Practices
143+
144+
1. **Use Page Objects**: Import actions from `cypress/views/`
145+
2. **Data Test Attributes**: Prefer `data-test` over CSS selectors
146+
3. **Keep Tests Isolated**: Each test should run independently
147+
4. **Meaningful Assertions**: Use descriptive error messages
148+
5. **Document Changes**: Update `E2E_TEST_SCENARIOS.md`
149+
150+
---
151+
152+
## Running Tests
153+
154+
### Common Commands
155+
156+
```bash
157+
cd web/cypress
158+
159+
# Run all regression tests
160+
npm run cypress:run --spec "cypress/e2e/**/regression/**"
161+
162+
# Run specific feature regression
163+
npm run cypress:run --spec "cypress/e2e/monitoring/regression/01.reg_alerts_admin.cy.ts"
164+
npm run cypress:run --spec "cypress/e2e/monitoring/regression/02.reg_metrics_admin.cy.ts"
165+
npm run cypress:run --spec "cypress/e2e/monitoring/regression/03.reg_legacy_dashboards_admin.cy.ts"
166+
167+
# Run BVT (Build Verification Tests)
168+
npm run cypress:run --spec "cypress/e2e/monitoring/00.bvt_admin.cy.ts"
169+
170+
# Run COO tests
171+
npm run cypress:run --spec "cypress/e2e/coo/01.coo_bvt.cy.ts"
172+
173+
# Run ACM Alerting tests
174+
npm run cypress:run --spec "cypress/e2e/coo/02.acm_alerting_ui.cy.ts"
175+
176+
# Interactive mode (GUI)
177+
npm run cypress:open
178+
```
179+
180+
### Environment Setup
181+
182+
**Interactive** (Recommended):
183+
```bash
184+
cd web/cypress
185+
source ./configure-env.sh
186+
```
187+
188+
**Manual Setup**: For complete environment variable reference and configuration examples, see [README.md](README.md#environment-variables-reference)
189+
190+
### Regression Testing Strategy
191+
192+
| Change Type | Required Tests |
193+
|-------------|---------------|
194+
| **UI Component Change** | Feature-specific regression + BVT |
195+
| **API Integration Change** | Full regression suite |
196+
| **Console Extension Change** | BVT + Navigation tests |
197+
| **Bug Fix** | New test + Related regression |
198+
199+
### Pre-PR Checklist
200+
201+
- [ ] `make lint-frontend` (no errors)
202+
- [ ] `make lint-backend` (no errors)
203+
- [ ] Ran BVT tests locally (all passing)
204+
- [ ] Ran regression tests for affected features (all passing)
205+
- [ ] Created/updated tests for new features or bug fixes
206+
- [ ] Updated `E2E_TEST_SCENARIOS.md` if added new tests
207+
208+
---
209+
210+
## Troubleshooting
211+
212+
### Debugging Failed Tests
213+
214+
1. **Check test videos**: `web/cypress/videos/`
215+
2. **Check screenshots**: `web/cypress/screenshots/`
216+
3. **Run with debug**:
217+
```bash
218+
export CYPRESS_DEBUG=true
219+
npm run cypress:run
220+
```
221+
4. **Run interactively**:
222+
```bash
223+
npm run cypress:open
224+
```
225+
226+
### Common Test Issues
227+
228+
| Issue | Solution |
229+
|-------|----------|
230+
| Test fails intermittently | Check for timing issues, add proper waits |
231+
| Element not found | Verify data-test attributes exist, check page object |
232+
| Assertion fails | Review expected vs actual values, update test |
233+
| Test hangs | Check for infinite loops or missing assertions |
234+
235+
### Setup & Configuration Issues
236+
237+
For environment variable issues, login problems, kubeconfig errors, and installation troubleshooting, see [README.md](README.md#troubleshooting-setup-issues)
238+
239+
### CI/CD Integration
240+
241+
Cypress tests run automatically in the CI pipeline:
242+
- **Pre-merge**: BVT tests run on every PR
243+
- **Post-merge**: Full regression suite runs on main branch
244+
- **Konflux Pipeline**: Automated testing for release candidates
245+
246+
---
247+
248+
## Additional Resources
249+
250+
- **Cypress Documentation**: https://docs.cypress.io/
251+
- **Test Scenarios Catalog**: `E2E_TEST_SCENARIOS.md`
252+
- **Setup Instructions**: `README.md`
253+
- **Main Guide**: `../../AGENTS.md`
254+

‎web/cypress/E2E_TEST_SCENARIOS.md‎

Lines changed: 7 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,12 @@ Located in `e2e/coo/`
2020
|------|------------|---------------|-------------|
2121
| `01.coo_bvt.cy.ts` | BVT: COO | 1. Admin perspective - Observe Menu | Verifies Observe menu navigation and submenus (Alerting, Silences, Alerting rules, Dashboards (Perses)) |
2222

23+
### ACM Alerting UI Tests
24+
25+
| File | Test Suite | Test Scenario | Description |
26+
|------|------------|---------------|-------------|
27+
| `02.acm_alerting_ui.cy.ts` | ACM Alerting UI | 1. Fleet Management perspective - ACM Alerting | Validates ACM integration with COO, Fleet Management perspective navigation, local-cluster access, and ACM alert visibility (Watchdog, Watchdog-spoke, ClusterCPUHealth) |
28+
2329
---
2430

2531
## Virtualization Tests
@@ -175,22 +181,11 @@ These test scenarios are reusable test suites called by the main E2E test files.
175181

176182
---
177183

178-
## Test Statistics Summary
179-
180-
| Category | Test Files | Direct it() Scenarios | Support Module Scenarios | Total Scenarios |
181-
|----------|------------|----------------------|-------------------------|-----------------|
182-
| **COO Tests** | 1 | 1 | 0 | 1 |
183-
| **Virtualization Tests** | 4 | 6 | ~30+ (via support modules) | ~36+ |
184-
| **Monitoring Tests** | 4 | 9 | ~30+ (via support modules) | ~39+ |
185-
| **Support Modules** | 8 | 0 | 39 | 39 |
186-
| **TOTAL** | **20** | **29** | **39** | **~128+** |
187-
188-
---
189-
190184
## Perspectives Tested
191185

192186
- **Administrator** - Standard admin perspective with full cluster access
193187
- **Virtualization** - Virtualization-specific perspective with integrated monitoring
188+
- **Fleet Management** - ACM multi-cluster management perspective with observability integration
194189

195190
## Namespace Scopes
196191

0 commit comments

Comments
 (0)