Add long-duration extension validation with screenshot evidence

Author: baboonzero

.gitignore

@@ -3,3 +3,4 @@ test-results/
 playwright-report/
 coverage/
 *.log
+artifacts/

README.md

@@ -43,10 +43,17 @@ For complete project history, architecture decisions, execution timeline, built/
   - `npm run test:all`
 - Run unpacked-extension smoke test in Chromium:
   - `npm run test:smoke:extension`
+- Run long-duration multi-window validation with artifact capture:
+  - `npm run test:validate:long`
+  - Optional headed mode:
+    - PowerShell: `$env:VALIDATION_HEADED='1'; npm run test:validate:long`
+  - Optional duration override:
+    - PowerShell: `$env:VALIDATION_DURATION_MINUTES='10'; npm run test:validate:long`
 
 Includes:
 - Unit tests for session engine transitions and IndexedDB retention/query boundaries.
 - Playwright dashboard tests for timeline filters, focus/open behavior, and settings action.
+- Timestamped screenshots and accessibility snapshots for long-run validation evidence.
 
 ## Notes
 

docs/testing/validation-evidence-2026-02-26.md

@@ -0,0 +1,67 @@
+# Validation Evidence - 2026-02-26
+
+This document records real extension-runtime validation runs with screenshot/snapshot evidence.
+
+## Tooling Decision
+
+- Intended skill for snapshots/screenshots: `agent-browser`.
+- Environment status: `agent-browser` CLI was not installed (`command not found`).
+- Fallback used: Playwright extension-runtime scripts:
+  - `scripts/extension-smoke-test.mjs`
+  - `scripts/long-duration-extension-validation.mjs`
+
+## Run 1: Long-Duration Multi-Window (Headless)
+
+- Run ID: `20260226-162944`
+- Duration: 8 minutes
+- Command: `npm run test:validate:long`
+- Artifact root:
+  - `artifacts/validation/20260226-162944/`
+- Evidence:
+  - 40+ screenshots across step checkpoints
+  - 8 structured ARIA snapshots
+  - JSON run log + markdown report
+
+Observed outcome:
+- Extension loaded and multi-window/tab switching actions executed.
+- Runtime status remained healthy (`ok=true`, `retentionDays=30`).
+- Final runtime idle state was `idle`, and final timeline count was `0`.
+
+Interpretation:
+- In headless automation, idle-state behavior can suppress effective session capture.
+
+## Run 2: Real Extension Runtime (Headed) Sanity Validation
+
+- Run ID: `20260226-163821`
+- Duration: 2 minutes
+- Command:
+  - `$env:VALIDATION_HEADED='1'; $env:VALIDATION_DURATION_MINUTES='2'; npm run test:validate:long`
+- Artifact root:
+  - `artifacts/validation/20260226-163821/`
+- Evidence:
+  - Screenshots at each checkpoint
+  - ARIA snapshots
+  - JSON run log + markdown report
+
+Observed outcome:
+- Extension loaded in headed Chromium with unpacked extension.
+- Timeline recorded at least one session (`timelineCount=1`).
+- Runtime status healthy (`ok=true`, `paused=false`, `retentionDays=30`).
+
+## Additional Runtime Proof
+
+- Smoke command:
+  - `npm run test:smoke:extension`
+- Evidence output (JSON):
+  - dashboard heading found
+  - timeline container present
+  - runtime message responded with retention=30
+  - settings page loaded
+
+## Conclusion
+
+- Automated quality gates are green (`test:unit`, `test:e2e`, `test:all`).
+- Real extension-runtime execution with screenshot evidence is complete.
+- For strict “real-user long-duration” sign-off, run headed long validation while actively using the machine:
+  - `$env:VALIDATION_HEADED='1'; $env:VALIDATION_DURATION_MINUTES='10'; npm run test:validate:long`
+  - then review artifacts in `artifacts/validation/<run-id>/`.

package.json

@@ -9,6 +9,7 @@
     "test:e2e:headed": "playwright test --headed",
     "test:e2e:ui": "playwright test --ui",
     "test:smoke:extension": "node scripts/extension-smoke-test.mjs",
+    "test:validate:long": "node scripts/long-duration-extension-validation.mjs",
     "test:all": "npm run test:unit && npm run test:e2e"
   },
   "keywords": [],