Skip to content

refactor: harden frontend workflows and reporting#1492

Open
Yunnglin wants to merge 15 commits into
mainfrom
agent/frontend-refactor-compliance
Open

refactor: harden frontend workflows and reporting#1492
Yunnglin wants to merge 15 commits into
mainfrom
agent/frontend-refactor-compliance

Conversation

@Yunnglin

@Yunnglin Yunnglin commented Jul 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • harden every Web evaluation, report, prediction, chart, and performance response with Zod runtime validation and schema-derived TypeScript contracts
  • restore the clear chronological Chat View for structured and agent traces, add adaptive 1D/2D/3D score visualizations, remove redundant Dashboard Quick Links, and make empty/error/retry states actionable
  • improve evaluation/performance history and comparison semantics, provider/protocol/workload context, low-sample warnings, responsive tables/cards, semantic forms, tabs, localization, and chart theme/fallback behavior
  • add deterministic unit/property/component tests, Storybook, critical Playwright flows, drift/decomposition/size gates, and a path-filtered frontend CI workflow
  • keep the existing React/TypeScript/Vite architecture and design identity; no state-library or framework rewrite
  • keep frontend unit/E2E tests inside the Web package, remove unused request/test helpers, and keep disposable Playwright/coverage artifacts out of Git
  • run frontend CI inside the pinned Playwright 1.61.1 Noble image on Node 24/npm 11, and normalize all lockfile tarballs to the public npm registry instead of an unreachable internal registry

Real-data verification

  • rescanned 51 real evaluation reports and the current performance archive through the local service
  • inspected real qwen-plus multi-dataset reports, structured predictions, Claw-Eval chronological traces, score/prediction comparison, and three DashScope performance archives in the in-app Browser
  • verified the original nullable single-sample std report failure now loads and renders unavailable dispersion values as em dashes

Tests

  • conda run -n eval pre-commit run --all-files
  • npm run lint
  • npm test — 51 files, 290 tests
  • npm run drift
  • npm run check:tofixed
  • npm run check:decoupling
  • npm run build
  • npm run size-budget — 118.71 KB / 150 KB initial gzip budget
  • npm run build-storybook
  • npm run e2e — 16/16 critical flows at 390 px and 1024 px
  • Playwright 1.61.1 Noble container — 16/16 critical flows; pixel visual regression and all screenshot goldens intentionally removed
  • conda run -n eval pytest tests/report/test_service_charts.py tests/perf/test_perf_archive.py -q — 11/11
  • npm audit --registry=https://registry.npmjs.org --audit-level=low — 0 vulnerabilities

Review notes

  • The separately owned P0 credential task is intentionally excluded.
  • Pixel visual regression was removed by product decision: no PNG goldens, screenshot helper, pixel threshold, npm script, or CI step remains. Storybook, semantic component assertions, real Browser verification, and fixed-viewport functional E2E remain.
  • Multi-configuration performance comparison currently preserves the specified non-blocking best-observed envelope behavior and warns about workload/configuration mismatch. A follow-up product choice can switch this to exact workload matching or user-selected rows without changing the API boundary.
  • Real performance runs are deliberately small and validate data plumbing/display, not service capacity.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the EvalScope Web Console frontend by decoupling data models from rendering, introducing Zod schemas for API validation, implementing list virtualization, and lazily loading heavy Markdown content. It also adds backend support for theme-aware Plotly charts and performance run metadata extraction. The review feedback highlights several critical issues, including a performance bottleneck in VirtualList due to a missing useEffect dependency array, invalid HTML nesting in ReportCard, potential crashes in the Python backend, and regressions across multiple components where removing score scaling breaks the rendering of 0-100 range scores.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread evalscope/perf/utils/report/perf_charts.py Outdated
Comment thread evalscope/web/src/components/common/VirtualList.tsx Outdated
Comment thread evalscope/web/src/components/reports/DetailsTab.tsx Outdated
Comment thread evalscope/web/src/components/reports/DetailsTab.tsx
Comment thread evalscope/web/src/components/reports/ReportHeader.tsx Outdated
Comment thread evalscope/web/src/components/reports/ReportSummaryStats.tsx Outdated
Comment thread evalscope/service/perf_archive.py Outdated
Comment thread evalscope/web/src/components/reports/ReportCard.tsx
Yunnglin added 7 commits July 16, 2026 11:49
- Strip 324 (Req X.Y) references across web source, scripts and DESIGN.md
- Remove size-budget guard: scripts/sizeBudget.ts, npm script, CI step, vite manifest
- Delete dead Combobox component + its test (no production usage)
- Remove one-off design-qa.md report and stray playwright test-results artifact
@Yunnglin
Yunnglin marked this pull request as ready for review July 16, 2026 09:43
Copilot AI review requested due to automatic review settings July 16, 2026 09:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the EvalScope Web Console and related service endpoints by introducing schema-validated API contracts, more deterministic frontend testing, improved accessibility/UX primitives, and enhanced performance-archive reporting metadata.

Changes:

  • Added Zod-backed runtime schemas + schema-derived TypeScript types, and migrated Web API clients to validated request/response handling (with typed domain errors and AbortSignal support).
  • Introduced deterministic frontend testing infrastructure (Vitest config + global deterministic setup + fixtures) and added focused unit/property/component tests.
  • Improved Web Console UI primitives (tabs/panels, tables, metric formatting consistency, accessibility/touch targets) and enhanced perf archive summaries (provider/protocol/api host, chart theming).

Reviewed changes

Copilot reviewed 162 out of 163 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
tests/report/test_service_charts.py Adds regression coverage for server-side chart theming behavior.
tests/perf/test_perf_archive.py Extends perf archive tests for extracted API host + concurrency metadata.
evalscope/web/vitest.config.ts Adds Vitest configuration merged with the existing Vite config.
evalscope/web/tsconfig.node.json Expands node TS project includes and adjusts module detection.
evalscope/web/tsconfig.app.json Excludes test-only sources from the app TS build.
evalscope/web/src/utils/formatUtils.ts Removes legacy score/percent formatting helpers in favor of centralized formatting.
evalscope/web/src/test/setup.ts Introduces deterministic test setup (fixed time/seed, network guard, fake timers).
evalscope/web/src/test/loadFixture.ts Adds a deterministic JSON fixture loader for Vitest tests.
evalscope/web/src/test/fixtures/report-single-dataset.json Adds a single-dataset report fixture for UI/schema testing.
evalscope/web/src/test/fixtures/report-real-single-sample.json Adds a real-shape fixture covering null std-dev single-sample behavior.
evalscope/web/src/test/fixtures/report-multi-dataset.json Adds a multi-dataset report fixture for visualization logic.
evalscope/web/src/test/fixtures/README.md Documents fixture rules and available fixture inventory.
evalscope/web/src/test/fixtures/perf-single-run.json Adds a perf single-run list fixture.
evalscope/web/src/test/fixtures/perf-multi-run.json Adds a perf multi-run comparison fixture.
evalscope/web/src/test/fixtures/perf-detail.json Adds a perf detail fixture for UI rendering and schema validation.
evalscope/web/src/pages/TasksPage.tsx Updates tabs to use explicit panel IDs and panel mapping.
evalscope/web/src/pages/BenchmarksPage.tsx Improves benchmark loading cancellation via AbortController and updates tab UI.
evalscope/web/src/index.css Removes eager KaTeX import and adds global focus/touch-target utilities.
evalscope/web/src/i18n/translations/types.ts Introduces shared locale dictionary typing.
evalscope/web/src/i18n/translations/trace.ts Adds new trace-related translation keys.
evalscope/web/src/i18n/translations/tasks.ts Adds tasks tab translations.
evalscope/web/src/i18n/translations/tabs.ts Adds translations for tab/panel validation messaging.
evalscope/web/src/i18n/translations/single.ts Adds/organizes translations for single-report views.
evalscope/web/src/i18n/translations/reports.ts Adds report list translations and labels.
evalscope/web/src/i18n/translations/reportDetail.ts Adds report detail view translations.
evalscope/web/src/i18n/translations/prediction.ts Adds prediction UI translations (including navigation/filter controls).
evalscope/web/src/i18n/translations/perf.ts Adds performance testing page translations.
evalscope/web/src/i18n/translations/nav.ts Adds navigation translation keys.
evalscope/web/src/i18n/translations/multi.ts Adds multi-report comparison translations.
evalscope/web/src/i18n/translations/metrics.ts Adds metric label translations and undefined-display messaging.
evalscope/web/src/i18n/translations/markdown.ts Adds markdown heavy-content gating and error translations.
evalscope/web/src/i18n/translations/index.ts Adds translation barrel with lookup + fallback logic and exposed dictionaries.
evalscope/web/src/i18n/translations/form.ts Adds localized validation message keys for forms.
evalscope/web/src/i18n/translations/eval.ts Adds evaluation page translations.
evalscope/web/src/i18n/translations/empty.ts Adds empty/error state translations and actions.
evalscope/web/src/i18n/translations/dashboard.ts Adds dashboard translations for new dashboard structure.
evalscope/web/src/i18n/translations/compare.ts Adds compare page translations and guidance messages.
evalscope/web/src/i18n/translations/common.ts Adds common UI strings (loading, errors, GitHub link, relative time).
evalscope/web/src/i18n/translations/charts.ts Adds chart loading/error/retry and fallback-table translations.
evalscope/web/src/i18n/translations/benchmarks.ts Adds benchmark catalog translations.
evalscope/web/src/domain/perf/perfWorkload.ts Adds pure workload normalization for perf runs (labels/placeholders).
evalscope/web/src/domain/perf/perfWorkload.test.ts Adds property tests for workload normalization invariants.
evalscope/web/src/domain/perf/compareModel.sampleTier.test.ts Adds property tests for low-sample tier boundary classification.
evalscope/web/src/domain/perf/compareModel.defaultBaseline.test.ts Adds property tests for default-baseline selection semantics.
evalscope/web/src/domain/metric/registry.test.ts Adds unit tests for metric key resolution and formatting behavior.
evalscope/web/src/domain/metric/metricFormat.prop3.test.ts Adds property tests for unbounded/no-spec formatting invariants.
evalscope/web/src/domain/metric/metricFormat.prop2.test.ts Adds property tests distinguishing missing vs legitimate zero values.
evalscope/web/src/domain/metric/MetricDisplaySpec.ts Introduces the canonical metric display spec + fallback contract.
evalscope/web/src/domain/form/validation.numeric.test.ts Adds property tests for numeric min/max validation rules.
evalscope/web/src/domain/form/validation.firstInvalid.test.ts Adds property tests for “first invalid field” focus ordering logic.
evalscope/web/src/domain/form/validation.datasetArgs.test.ts Adds property tests for Dataset Args JSON validation + round-trip behavior.
evalscope/web/src/domain/compare/compareModel.selectionCap.test.ts Adds property tests for compare-selection cap enforcement and dedup.
evalscope/web/src/domain/compare/compareModel.selection.test.ts Adds property tests for selection preservation across reorder/filter.
evalscope/web/src/domain/compare/compareModel.displayLabel.test.ts Adds property tests for run display label parsing/composition.
evalscope/web/src/contexts/ReportsContext.tsx Migrates config fetch to validated API + adds AbortSignal threading for report loads.
evalscope/web/src/components/ui/Table.tsx Adds per-column class hooks and localizes the empty-state text.
evalscope/web/src/components/ui/ScoreChip.tsx Routes score formatting through centralized metric registry + localization.
evalscope/web/src/components/ui/ScoreBadge.tsx Routes score formatting through centralized metric registry and updates threshold styling.
evalscope/web/src/components/ui/ModelGroupHeader.tsx Uses centralized metric formatting for best-score display.
evalscope/web/src/components/ui/KpiCard.tsx Improves long KPI value wrapping for responsive layouts.
evalscope/web/src/components/ui/Field.tsx Adds an accessible form-field primitive (label/aria/error association).
evalscope/web/src/components/ui/EvalRunCard.tsx Adjusts header text wrapping for narrow layouts.
evalscope/web/src/components/ui/Collapsible.tsx Adds headerAfter to avoid nesting interactive elements in a toggle button.
evalscope/web/src/components/single/ChatView.tsx Restores explicit chat presentation modes (traced/structured/legacy).
evalscope/web/src/components/single/ChatView.test.tsx Adds component tests for chat rendering modes.
evalscope/web/src/components/single/chat/MessageComponents.tsx Uses headerAfter to move MsgIdChip out of toggle button content.
evalscope/web/src/components/single/chat/EvalResultPanel.test.tsx Adds tests for labeling hierarchy and threshold styling.
evalscope/web/src/components/single/chat/AgentTraceView.tsx Introduces VirtualList-based windowing for long traces and timelines.
evalscope/web/src/components/reports/ReportSummaryStats.tsx Clamps ring math + centralizes score percentage formatting; improves responsiveness.
evalscope/web/src/components/reports/ReportHeader.tsx Switches header score formatting to metric registry output.
evalscope/web/src/components/reports/ReportFilters.tsx Improves touch targets and responsive search field sizing.
evalscope/web/src/components/reports/PredictionsTab.test.tsx Adds tests for localized controls, error states, and aborted request handling.
evalscope/web/src/components/reports/PerfMetricsPanel.test.tsx Adds regression test for missing std-dev placeholders and responsive KPI strip.
evalscope/web/src/components/reports/OverviewTab.test.tsx Adds tests for dataset score visualization gating (table vs radar).
evalscope/web/src/components/reports/metricConsistency.test.tsx Ensures metric display is identical across card/table surfaces.
evalscope/web/src/components/reports/DetailsTab.tsx Adds AbortController cancellation and centralizes score formatting.
evalscope/web/src/components/nav/TopNav.tsx Adds coarse-pointer touch target support to nav controls.
evalscope/web/src/components/nav/ThemeToggle.tsx Adds coarse-pointer touch target support to theme toggle.
evalscope/web/src/components/nav/LocaleToggle.tsx Adds coarse-pointer touch target support to locale toggle.
evalscope/web/src/components/common/virtualWindow.ts Adds pure virtualization window math used by list windowing components.
evalscope/web/src/components/common/MarkdownRenderer.test.tsx Adds tests for heavy-content gating behavior in markdown rendering.
evalscope/web/src/components/common/DataTableFallback.tsx Adds accessible chart fallback rendering as an authoritative data table.
evalscope/web/src/components/common/DataTable.tsx Centralizes score formatting via metric registry and adds localization.
evalscope/web/src/components/charts/PlotlyChart.tsx Refactors PlotlyChart to a wrapper around ChartFrame with table fallback support.
evalscope/web/src/api/types/reports.ts Introduces reports-domain type barrel derived from Zod schemas.
evalscope/web/src/api/types/perf.ts Introduces perf-domain type barrel derived from Zod schemas.
evalscope/web/src/api/types/index.ts Introduces a stable API types barrel split by domain.
evalscope/web/src/api/types/eval.ts Introduces eval-domain type barrel derived from Zod schemas.
evalscope/web/src/api/types/common.ts Introduces shared/common API response types derived from schemas.
evalscope/web/src/api/schemas/reports.schema.test.ts Adds schema compatibility tests for real report edge cases (null std).
evalscope/web/src/api/schemas/perf.schema.ts Adds perf-domain Zod schemas + inferred types.
evalscope/web/src/api/schemas/index.ts Adds schema barrel exports for stable import paths.
evalscope/web/src/api/schemas/eval.schema.ts Adds eval-domain Zod schemas + inferred types.
evalscope/web/src/api/schemas/endpointSchemas.test.ts Adds smoke tests ensuring each endpoint schema accepts valid shapes and rejects null.
evalscope/web/src/api/schemas/common.schema.ts Adds shared/common endpoint schemas + inferred types.
evalscope/web/src/api/reports.ts Migrates reports API to validated client calls + AbortSignal support.
evalscope/web/src/api/perf.ts Migrates perf API to validated client calls + AbortSignal support.
evalscope/web/src/api/eval.ts Migrates eval API to validated client calls + AbortSignal support.
evalscope/web/src/api/errors.ts Introduces typed API domain errors for discriminable failure handling.
evalscope/web/scripts/drift/structureCheck.ts Adds design structure drift checker for CI gating.
evalscope/web/scripts/drift/localeKeyCheck.ts Adds locale key drift checker plus CI/CLI runner.
evalscope/web/scripts/drift/index.ts Adds drift-check orchestrator (tokens/structure/locale) for CI.
evalscope/web/package.json Adds test/drift scripts and testing/schema dependencies (vitest, fast-check, zod, etc.).
evalscope/web/eslint.config.js Ignores test artifacts and adds restrictions to enforce formatting/domain-layer boundaries.
evalscope/web/.gitignore Ignores frontend coverage outputs.
evalscope/service/perf_archive.py Adds perf run identity metadata (provider/protocol/api_host) and concurrency aggregation.
evalscope/service/blueprints/reports.py Adds theme parameter support for chart rendering and applies Plotly templates.
evalscope/perf/utils/report/perf_charts.py Tweaks compare chart rendering for sparse series (markers-only).
.github/workflows/frontend.yml Adds path-filtered frontend CI workflow (lint/test/drift/build).

Comment thread evalscope/web/src/test/loadFixture.ts
Comment thread evalscope/web/src/domain/perf/perfWorkload.test.ts Outdated
Comment thread evalscope/web/src/domain/perf/perfWorkload.test.ts Outdated
Comment thread evalscope/web/src/domain/perf/compareModel.sampleTier.test.ts Outdated
Comment thread evalscope/web/src/domain/perf/compareModel.sampleTier.test.ts Outdated
Comment thread evalscope/web/src/domain/perf/compareModel.defaultBaseline.test.ts Outdated
Comment thread evalscope/web/src/domain/metric/metricFormat.prop2.test.ts Outdated
Comment thread evalscope/web/src/domain/metric/metricFormat.prop2.test.ts Outdated
Comment thread evalscope/web/src/domain/metric/metricFormat.prop3.test.ts Outdated
Comment thread evalscope/web/src/domain/metric/metricFormat.prop3.test.ts Outdated
Yunnglin added 7 commits July 17, 2026 09:40
Merge granular per-property test files into one file per module while
keeping every test case (221 total unchanged):
- metric/metricFormat: boundary + prop1/2/3 -> metricFormat.test.ts
- perf/compareModel: 6 property files -> compareModel.test.ts
- compare/compareModel: 3 property files -> compareModel.test.ts

Also remove two fixtures no test references (perf-multi-run.json,
report-single-dataset.json) and update the fixtures README table.

Net: test files 43 -> 33, ~170 fewer lines. tsc/eslint/vitest all green.
Remove 9 inferred type aliases that are declared in the zod schemas and
re-exported through the @/api/types barrel but consumed nowhere (verified
via knip + whole-repo reference scan): ConfigResponse, InvokeStatus,
SubsetData, CategoryData, MetricData, PerfMetricsSummary, SamplePerfMetrics,
ToolMessageError, AgentTraceEventType. The underlying schemas are kept
(still composed internally).

Also dedupe DashboardPage's local formatFull, which was a byte-for-byte
copy of @/utils/perf's formatFull; import the util instead.

tsc/eslint/vitest (221 tests) all green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants