feat(gui): add Frontier benchmark guide page by Wibias · Pull Request #144 · lidge-jun/opencodex

Wibias · 2026-07-17T23:56:17Z

Summary

Frontier workspace page for public coding-agent benchmarks (capability vs cost)
Audit rework kept as draft through P1s; this push closes the remaining open items

Addressed

Bundle: React.lazy Frontier route + tree-shaken ECharts
Provenance: per-board URL / capture date / license / citation
Cost honesty: per-row costKind; score/$ and Best value only when uniformly measured
AA Coding: Sol/Terra/Luna + Grok figures updated
ProgramBench: costs reconciled from extended leaderboard → all rows measured (GPT-5.5 xhigh/high/default + Opus 4.7 xhigh fixed)
i18n: board titles, axis labels, and source notes localized (en/de/ko/zh)
Harness vs effort: modeKind on boards; FrontierSWE / Terminal-Bench / SWE Marathon label filters + ranking column as Harness
Mobile: chart height capped on narrow viewports; CSS no longer forces a conflicting fixed height
Drift process: gui/src/data/FRONTIER.md refresh checklist
Tests: tests/frontier-rank.test.ts covers ranking + ProgramBench + modeKind

Test plan

Open #frontier — chunk load / full-bleed workspace
AA Coding: By value enabled
ProgramBench: By value enabled; costs match extended board ($8.85 / $3.65 / $10.96 / $1.21 for the GPT-5.5 / Opus xhigh rows)
Switch locale to de/ko/zh — board titles and axis labels translate
Terminal-Bench / FrontierSWE — filter + column say Harness
Narrow viewport — chart stays ≤ ~320px
bun test tests/frontier-rank.test.ts

Ship a snapshot-based model comparison guide with domain/board filters, ECharts charts (scatter, cost stack, score/value, by-reasoning), and en/de/ko/zh copy.

Satisfy @typescript-eslint/no-unused-vars in CI.

Local-only Playwright helper; not part of the feature.

Move Frontier off the stacked page/card layout into a full-bleed rail + detail shell (domain filters + board list on the left, chart/ranking on the right), matching Combos/Providers.

lidge-jun · 2026-07-18T09:35:18Z

Review verdict (independent audit, 2026-07-18): request changes — please keep in draft / rework before this can merge. The page concept is useful, but three blocking findings:

Data accuracy (P1). A full claim-ledger pass against the original leaderboards found contradicted values presented as rankings: AA Coding lists GPT-5.6 Sol/Terra/Luna as 66/64/63 where Artificial Analysis publishes 80/77/75 (published 2026-07-09, after the page's stated 2026-07-01 capture); Grok 4.5 as 61/$1.20 vs the official 76/$2.49–2.59; four ProgramBench costs contradict its extended leaderboard. Many cost fields (FrontierSWE, Arena, SWE-Marathon, Cybench, Terminal-Bench) are API-price blends or estimates, but the UI labels them "Cost / task" and awards "Best value". Estimates and measurements can't share a ranking axis — split costPerTask / apiPricePerMTok / estimatedRelativeCost and drop score-per-dollar until every row has a uniform source-backed cost.
Bundle impact (P1). The page, its 59KB JSON, and the full ECharts package are statically imported in App.tsx — initial JS goes from ~509KB to ~1.71MB (+236%, gzip +268%) for every GUI user. Minimum: React.lazy the route + modular ECharts imports. Honestly, this content may fit docs-site better than the operational GUI.
Attribution/licensing (P1). The schema has only a prose sourceNote — no source URLs, capture dates, benchmark versions, or licenses. Cybench explicitly requests citation. Each source needs a link + capture date + reuse terms before we republish its data.

Also noted (P2): board descriptions/axis labels/tooltips stay English under de/ko/zh; Terminal-Bench stores harnesses (codex, claude-code…) in the effort field; the dataset already drifted during review (DeepSWE added Kimi K3 on 07-17); no tests for the filtering/ranking logic; mobile CSS caps charts at 360px making 50-bar boards unreadable.

Happy to merge a reworked version — docs-site placement or lazy-loaded GUI route, with a provenance manifest (URL + date + version per row) and estimate/measurement separation. The verified rows (DeepSWE, FrontierCode, FrontierSWE, ProgramBench scores, Arena scores, Cybench scores) are solid work worth keeping.

Convert route to React.lazy + modular ECharts, add per-board provenance and per-row costKind, disable score/$ / Best value unless costs are uniformly measured, fix AA Coding GPT-5.6/Grok figures, and add ranking unit tests.

Wibias · 2026-07-18T16:37:20Z

Thanks for the independent audit - converting this to draft and starting the rework.

Done in the latest push

Bundle (P1): Frontier is React.lazy’d; ECharts is modular (frontier-echarts.ts). Local production build: main ~528KB / gzip ~157KB, Frontier chunk separate (~635KB) instead of ~1.71MB on every page.
Attribution (P1): Boards now carry provenance (URL + capture date + license/citation). UI shows source link + terms (Cybench citation included).
Cost axis (P1): Rows have costKind. Score/$ / “Best value” only when the board is uniformly measured. Estimate/API-blend boards rank by score only and label costs as estimated.
AA Coding contradictions (P1): Sol/Terra/Luna → 80/77/75 and Grok 4.5 → 76 / ~$2.54, capture date 2026-07-09, provenance → artificialanalysis.ai/agents/coding-agents.
Tests: ranking helpers covered in tests/frontier-rank.test.ts.

Still intentionally open

ProgramBench costs kept as estimated until we reconcile the extended board numbers row-by-row (scores retained).
P2 items (i18n of board strings, harness-vs-effort, mobile chart height, live drift process).

Happy to keep iterating in draft until you’re comfortable un-drafting.

Keep Frontier nav/lazy route alongside Combos full-bleed shell and retain both frontier.* and cws.* i18n keys after the combo stack landed.

Pull ProgramBench averages from the extended board as measured, localize board titles/axes/notes, label harness vs effort, cap mobile chart height, and document the snapshot refresh process.

Wibias · 2026-07-18T17:00:09Z

Closed the remaining open items on this draft:

ProgramBench — reconciled against https://programbench.com/extended/ (2026-06-23). GPT-5.5 xhigh/high/default and Opus 4.7 xhigh costs corrected; all rows now measured, so By value is enabled.
Board i18n — title / axis / sourceNote keys under frontier.board.<id>.* for en/de/ko/zh.
Harness vs effort — modeKind on boards; FrontierSWE, Terminal-Bench 2.1, and SWE Marathon use Harness labels.
Mobile chart height — JS cap at 320px on ≤720px; dropped the conflicting height: 360px !important.
Drift process — documented in gui/src/data/FRONTIER.md.

Happy to un-draft when you’re comfortable with the review.

Without it the action cannot list PR files, skips writing the changed-files list, then still passes --changed-files-from and the CLI exits 1 on ENOENT.

Build dynamic frontier.board.* TKeys via join() so the hardcoded-UI-string rule does not flag the lookup helper.

CLI 0.7.8 emits baseline reports with schemaVersion 3; the old v2.1.0 action only accepted 1–2, so ensure-json-report forced SCAN_STATUS=1 on every PR even with blocking: none.

Wibias · 2026-07-18T17:25:58Z

Should be done!

Wibias · 2026-07-19T18:32:32Z

Ready to merge @lidge-jun

…e docs site The GUI was the wrong home for 2.6k lines of hand-maintained leaderboard snapshots plus a new ECharts runtime. The Astro docs site (GitHub Pages, slow-update-tolerant) is the natural surface: - docs-site Benchmarks page in en/ko/zh-cn, sidebar entry under Guides - FrontierBoards.astro: per-board provenance (url/capturedAt/license), costKind honesty note, vanilla ECharts cost-vs-score scatter (no React), data table with score/$ only when every row is source-measured - data + i18n (104 keys x 3 locales) extracted from PR #144; German dropped (site ships en/ko/zh-CN); refresh checklist in src/data/README-frontier.md - echarts added to docs-site only Verified: bun run build (58 pages), headless Chrome render of en + ko pages (charts draw, honesty notes and best-value badge show), zh-cn markup check. Data and concept credited to Wibias (PR #144).

…per-domain pages User feedback: one long page with 10 boards was too much. Now a top-level Benchmarks category (collapsed) with an Overview index page plus five domain pages (coding 6 boards, frontend/terminal/security/intelligence 1 each) x en/ko/zh-cn = 18 pages. FrontierBoards.astro gains boards/intro props; the guides/benchmarks single pages are removed. Domain mapping follows PR #144's frontier-domains.ts. Verified: astro build (73 pages), headless Chrome render of ko overview (sidebar category + domain links) and ko coding page (6 charts, honesty notes, best-value badge), curl 200 + hreflang for en/ko/zh-cn.

lidge-jun · 2026-07-20T03:47:51Z

Direction update from the maintainer side: we ported the benchmark data + concept to the docs site instead of merging this into the GUI.

Rationale: half the diff is hand-maintained leaderboard snapshots, and the docs site (GitHub Pages, Astro Starlight) tolerates slow/manual data refreshes better than the live GUI runtime — no new ECharts/React bundle in the dashboard, and the data still ships.

What landed on main (62befa0, 3f04819), credited to this PR:

New Benchmarks sidebar category on https://lidge-jun.github.io/opencodex/benchmarks/ — Overview + per-domain pages (Coding / Frontend / Terminal / Security / Intelligence), in en/ko/zh-CN
Your frontier-benchmarks.json as-is, the frontier.* i18n strings (en/ko/zh extracted; German dropped since the site ships 3 locales), per-board provenance (url/capturedAt/license), the costKind honesty rule (score/$ + best-value only on uniformly measured boards), and the refresh checklist (docs-site/src/data/README-frontier.md)
Charts re-implemented with the vanilla ECharts API in an Astro component — no React island needed

What we deliberately did NOT take: the GUI page itself, the React/ECharts dashboard runtime, the domain/filter UI, and the react-doctor workflow change.

Thanks for the well-structured dataset and the provenance/costKind discipline — it made the port straightforward. Closing this as superseded-by-docs-site; if you want the GUI version later, the docs pages and your branch can coexist.

Wibias added 3 commits July 18, 2026 01:49

feat(gui): add Frontier benchmark guide page

1863351

Ship a snapshot-based model comparison guide with domain/board filters, ECharts charts (scatter, cost stack, score/value, by-reasoning), and en/de/ko/zh copy.

fix(gui): drop unused frontierChartHeight modelCount arg

e9b6191

Satisfy @typescript-eslint/no-unused-vars in CI.

chore(gui): remove Frontier screenshot helper from PR

a49c4f5

Local-only Playwright helper; not part of the feature.

Wibias marked this pull request as draft July 18, 2026 02:40

Wibias added 2 commits July 18, 2026 07:19

feat(gui): put Frontier on the workspace chrome

8a333cc

Move Frontier off the stacked page/card layout into a full-bleed rail + detail shell (domain filters + board list on the left, chart/ranking on the right), matching Combos/Providers.

Update styles-frontier-workspace.css

64322e9

Wibias marked this pull request as ready for review July 18, 2026 05:27

Wibias marked this pull request as draft July 18, 2026 16:29

Wibias added 2 commits July 18, 2026 18:43

merge: sync upstream/main into Frontier (Combos + providers)

6f9f438

Keep Frontier nav/lazy route alongside Combos full-bleed shell and retain both frontier.* and cws.* i18n keys after the combo stack landed.

fix(gui): close Frontier P2s and reconcile ProgramBench costs

f55a29b

Pull ProgramBench averages from the extended board as measured, localize board titles/axes/notes, label harness vs effort, cap mobile chart height, and document the snapshot refresh process.

Wibias added 3 commits July 18, 2026 19:05

fix(ci): grant react-doctor pull-requests: read for fork PRs

4ccf924

Without it the action cannot list PR files, skips writing the changed-files list, then still passes --changed-files-from and the CLI exits 1 on ENOENT.

fix(gui): satisfy local-i18n lint for Frontier board keys

8c272bd

Build dynamic frontier.board.* TKeys via join() so the hardcoded-UI-string rule does not flag the lookup helper.

fix(ci): bump react-doctor action to accept JSON schema v3

505e28e

CLI 0.7.8 emits baseline reports with schemaVersion 3; the old v2.1.0 action only accepted 1–2, so ensure-json-report forced SCAN_STATUS=1 on every PR even with blocking: none.

Wibias marked this pull request as ready for review July 18, 2026 17:25

This was referenced Jul 18, 2026

fix: keep configured port after update restart #152

Closed

feat(oauth): GitHub Copilot device-flow login #153

Closed

lidge-jun closed this Jul 20, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(gui): add Frontier benchmark guide page#144

feat(gui): add Frontier benchmark guide page#144
Wibias wants to merge 11 commits into
lidge-jun:mainfrom
Wibias:feat/gui-frontier-guide

Wibias commented Jul 17, 2026 •

edited

Loading

Uh oh!

lidge-jun commented Jul 18, 2026

Uh oh!

Wibias commented Jul 18, 2026 •

edited

Loading

Uh oh!

Wibias commented Jul 18, 2026

Uh oh!

Wibias commented Jul 18, 2026

Uh oh!

Wibias commented Jul 19, 2026

Uh oh!

lidge-jun commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Wibias commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Addressed

Test plan

Uh oh!

lidge-jun commented Jul 18, 2026

Uh oh!

Wibias commented Jul 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Done in the latest push

Still intentionally open

Uh oh!

Wibias commented Jul 18, 2026

Uh oh!

Wibias commented Jul 18, 2026

Uh oh!

Wibias commented Jul 19, 2026

Uh oh!

lidge-jun commented Jul 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Wibias commented Jul 17, 2026 •

edited

Loading

Wibias commented Jul 18, 2026 •

edited

Loading