feat(google_finance): add Google Finance mirror (site 17, port 40016)#73
Open
QiushiSun wants to merge 2 commits into
Open
feat(google_finance): add Google Finance mirror (site 17, port 40016)#73QiushiSun wants to merge 2 commits into
QiushiSun wants to merge 2 commits into
Conversation
Mirrors https://www.google.com/finance/ as site 17 on port 40016. Registers the site in the three required places (websyn_start.sh SITES, control_server.py SITES, Dockerfile EXPOSE 40000-40016) and pins MarkupSafe==3.0.3, which app.py imports directly for its server-rendered SVG charts. Covers the home page with seven region tabs, quote pages for stocks, ETFs, indexes, sector indexes, crypto, FX and futures across eight chart ranges and five tabs, eight market boards, scored search with autocomplete, news, a currency converter, a compare view, and auth-gated watchlists and portfolios. 10,275 seeded rows across 18 models; market data is synthetic and frozen at 2026-07-24, company identities and logos are real. 20 benchmark tasks in sites/google_finance/tasks.jsonl, contributor keys only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fetch_assets.sh otherwise resolves a revision that predates the paired assets PR, and websyn_start.sh copies instance_seed to instance for every site under set -e — so a missing directory aborts the whole entrypoint and no site comes up, not just this one. Verified from a clean clone of this branch: fetch_assets.sh, build.sh, and a container on 41000-41016 give /health ok with 17/17 alive and every port 200, with the seed DB byte-identical after POST /reset/google_finance. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.






Mirrors
https://www.google.com/finance/as site 17 on port 40016.Coverage
Home page with seven region tabs (US / Europe / Asia / Latin America /
Currencies / Crypto / Futures); quote pages for stocks, ETFs, indexes, sector
indexes, crypto, FX and futures across eight chart ranges and
Overview / Analysis / Earnings / Financials / Holdings tabs; eight market
boards (most active, gainers, losers, indexes, cryptocurrencies, currencies,
futures, climate leaders); scored token-overlap search with autocomplete; a
news index and article pages; a currency converter; a compare view; and
auth-gated watchlists and portfolios with live gain/loss maths. Sign-in works
through both an email/password form and an
accounts.google.com-style profilepicker.
Charts are rendered server-side as SVG — gradient area fill, volume band,
after-hours leg behind a session divider, previous-close line, key-moment
markers, 50/200-day moving averages, and a hover crosshair. JavaScript is
enhancement only; the toolbar menus are
<details>/<summary>so they workwith JS off.
Seeded rows — 10,275 total
The 182 instruments are 113 stocks across 11 GICS sectors (≥6 each), 12 ETFs,
20 indexes (five each for US / Europe / Asia / Latin America, matching the live
home-page tabs), 11 sector indexes, 10 cryptocurrencies, 10 FX pairs and 6
futures. All four benchmark accounts carry pre-existing auth-gated state, and
every portfolio holds at least one losing position so "which holding is down"
phrasings are answerable.
Data policy
Company identities, logos, About blocks and profile prose are real,
harvested from the live site. Every market number — price, OHLC series, key
statistics, financial statements, earnings, analyst targets — is synthetic,
generated from a fixed per-symbol seed and frozen at
MARKET_DATE = 2026-07-24,with magnitudes anchored to a real open price and share count so each
instrument lands in a believable range.
Two benchmark-driven reasons: scraped quotes are stale the day they are
captured, and a frontier model can approximately recall real fundamentals,
which is the "trivially answerable from prior knowledge" shortcut the guide
rejects. Every task therefore targets synthetic values only.
Assets
125 real company logos (inline SVG), 41 real publisher favicons, and Google
Finance's own favicon set — 4.3 MB packed. Google Finance ships almost no
bitmap imagery; its visual identity is typography, colour and charts, so
fidelity here comes from the SVG chart engine rather than photographs.
Tasks
20 in
sites/google_finance/tasks.jsonl, contributor keys only(
web_name, id, ques, web, upstream_url) — noanswer,verifier_pathorjudge_rubric. Every answer is a frozen synthetic value that has to be readoff a detail page.
Coverage spans the home region tabs, quote pages for stocks / ETFs / indexes /
crypto, all five quote tabs, the chart range switcher and key-moment markers,
two market boards, scored search, news, the currency converter, the compare
view, the Google account chooser, auth-gated watchlist and portfolio reads, and
one portfolio write (create + add position).
Audited with four Playwright passes: every answer is present on its target page
and absent from every upstream page; every answer page is reachable by clicking
only; every UI label and control a task's wording names actually exists; and
every "which one is highest/lowest" task has a genuine runner-up with no ties
(closest margin 10.4%).
One note on the hardening guide's ≤50% full-match density target:
/search?q=utilityreturns 8 rows of which 7 are Utilities-sector stocks. Thediscrimination in that task is the dividend-yield comparison across the
matches, not the sector filter — the two non-Utilities results (a sector index
and a Consumer Staples name) are the near-misses, and the QA suite asserts the
yield winner beats the runner-up by between 1% and 60%.
Two deliberate deviations from the current upstream UI
No Gemini "Research" rail.
google.com/financenow redirects to/finance/beta, whose right column is an LLM chat. Mirroring it would needa live model at request time, which breaks the determinism the project rests
on; a dead textbox is an explicit reject reason. The rail is kept and filled
with deterministic content instead: market movers and top news on most
pages, related assets and sector context on a quote page.
A real
/search?q=results page. In the beta UI the top search barroutes into that same AI panel. WebHarbor requires scored token-overlap
search and lists "non-functional search" as a reject reason, so the mirror
implements the classic Google Finance behaviour: an autocomplete dropdown
plus a results page over instruments and news.
Shared-file changes
The three registration points (
websyn_start.shSITES,control_server.pySITES,DockerfileEXPOSE 40000-40016), plus one dependency pin:MarkupSafe==3.0.3.sites/google_finance/app.pyis the only site module thatimports
markupsafedirectly (for the server-rendered SVG). pip already pullsit in transitively via Jinja2, but AGENTS.md asks for locked versions, and the
Dockerfile already pins Flask's other transitive deps (Werkzeug, Jinja2) the
same way.
Determinism
The market clock is frozen at 2026-07-24; nothing calls
date.today(). Allthree
seed_*()functions gate at function level.Pre-PR checks
Run against
webharbor:devbuilt from this branch (5.95 GB image), containeron
8201:8101and41000-41016:40000-40016:python3 -m py_compile sites/google_finance/app.py— cleanGET /health—ok: true, 17/17 sites alive41000–41016return 200(a row was inserted first, so the md5 genuinely diverged to
4eef20b25fa17283382abce7d2a5e28bbefore the reset):https://huggingface.co/datasets/ChilleD/WebHarbor/discussions/47
Assets pin
.assets-revisionis pinned torefs/pr/47— the paired HF PR — following thesame convention as #53 and #61, so this branch builds end-to-end before the
assets PR is merged. That ref is current main plus this one tarball, so all 17
sites are present. Maintainer: once HF PR #47 is merged, bump the pin to the
merged commit SHA on main; the file carries that note inline.
Verified from a clean
git cloneof this branch, not from my working tree:This matters more than it looks:
websyn_start.shcopiesinstance_seedtoinstancefor every site underset -e, so a missing directory aborts thewhole entrypoint — without the pin no site comes up, not just this one.
Port slot
The site is registered at index 16 / port 40016, appended after
merriam_webster. Several open PRs are queued on that same slot; happy torebase onto whichever lands first — the port appears only in the three
registration points and in
tasks.jsonl'swebfield.