fix(analytics): write one request line per request, closed or finished

docs/analytics.md

@@ -61,13 +61,15 @@ When `POSTHOG_KEY` and `POSTHOG_HOST` are set, `track()` also forwards each even
 
 A resolved part records what the question asked for, and counts the StatVars and places it expanded to: an aid query expands to hundreds of each, which would pass the 8 KB cap. A role asked for as a named group, such as the European Union, records the group, three of its members and the member count. A set-valued filter with more than three refs records three and the count. `sources` lists every source id while the whole event fits the cap, and the first three per part when it does not; `sourceCount` is always the full number, so a list shorter than the count is a sample. The run and settle fields stay on every form, and the floor drops only `byStep`. If the event still exceeds the cap, each resolved part keeps only `specId`, `measure`, `statVarCount`, `entityCount`, `sourceCount` and the per-role place `count`, and the event carries `truncated: true`. If that form is still too large, the event ends on a fixed-size floor: an empty `query`, no refused parts, and per resolved part only a clipped `specId` and the integer counts. The ladder runs on the complete props, `seeds` included, and measures the JSON length as the route does; the API's limit of six parts bounds the floor.
 
-In production the `dfo-prod-website-export` Cloud Run Job (`db/export-events.ts`) copies the table every hour into the BigQuery table `event` in the `website_export` dataset, one partition per UTC day of `created_at`. Each run replaces every day from the latest loaded day minus one through today, plus any earlier day whose row count differs from its partition's, so a day that failed to load is retried until it matches. The copy hashes `visitor_id`, `user_id` and the `shareId` prop, keeps `page` only as a site path without its query string and with share ids written `:id`, and keeps only the props `PROP_RULES` in `db/event-export.ts` allows, so it holds no search text. `deploy/README.md` ("Event export") has the schema. Questions about the text itself stay in Postgres, e.g. the most-requested measures:
+In production the `dfo-prod-website-export` Cloud Run Job (`db/export-events.ts`) copies the table every hour into the BigQuery table `event` in the `website_export` dataset, one partition per UTC day of `created_at`. Each run replaces every day from the latest loaded day minus one through today, plus any earlier day whose row count differs from its partition's, so a day that failed to load is retried until it matches, plus every day holding a row whose `export_rules` digest differs from the current one, so a change to the export rules or the hashing key reaches every partition already loaded. The copy hashes `visitor_id`, `user_id` and the `shareId` prop, keeps `page` only as a site path without its query string and with share ids written `:id`, and keeps only the props `PROP_RULES` in `db/event-export.ts` allows, so it holds no search text. `deploy/README.md` ("Event export") has the schema. Questions about the text itself stay in Postgres, e.g. the most-requested measures:
 
 ```sql
 select r->'measure' as measure, count(*) from event, jsonb_array_elements(props->'resolved') r
 where name = 'search.query' group by 1 order by 2 desc limit 20;
 ```
 
+The same Job first copies the Postgres view `account_daily` (`db/account-export.ts`) into the BigQuery table `account_daily` in the same dataset, replacing the whole table in one load job, so a failed run leaves the previous table. The view counts new accounts per UTC day and organisation, as `day`, `organisation` and `accounts`, from the first account's day through yesterday, so every day it holds is complete. An organisation is the lower-cased email domain once the domain holds at least 3 accounts, counting the accounts created through yesterday; `Personal email` for a webmail domain, whatever its count; and `Other organisations` for any other domain, so a domain with fewer than 3 accounts never shows by name. A day without new accounts is one row with a null `organisation` and 0 `accounts`. The table holds no email address and no user id. `deploy/README.md` ("Accounts") has the schema.
+
 ## Sentry
 
 `@sentry/nextjs` on both sides: `instrumentation-client.ts` (browser), `sentry.server.config.ts` / `sentry.edge.config.ts` loaded from `src/instrumentation.ts`, `src/app/global-error.tsx` for a crash in the root layout. Everything is off without `NEXT_PUBLIC_SENTRY_DSN`. `sendDefaultPii` is false and there is no session replay, so it needs no cookie consent.
@@ -92,13 +94,13 @@ To check it on production, open the network panel and filter on `google-analytic
 
 ## Request log
 
-The site writes one JSON line to stdout for each request whose response completes, which Cloud Logging stores as a structured entry. `src/instrumentation.ts` registers `registerRequestLogging()` in `src/lib/proxy/requestLog.ts` at server startup. It observes Node's `http.server.response.finish` diagnostic channel and writes the line when the response finishes, from the request's method, path and headers and the response's status code. A request accepted during startup is logged too, since its response finishes after registration. A page render the visitor abandons, an `/mcp` event stream the client closes, and any connection closed before the response ends write no line. Visitor counts come from this line, because Cloud Run's own request log records Cloudflare's edge address as the client.
+The site writes one JSON line to stdout for each logged request, which Cloud Logging stores as a structured entry. `src/instrumentation.ts` registers `registerRequestLogging()` in `src/lib/proxy/requestLog.ts` at server startup. It subscribes to Node's `http.server.request.start` diagnostic channel and attaches one `close` listener to each response. Node emits `close` after `finish`, so every request writes exactly one line, when its response closes, with `finished` true for a response that ended normally. A response that closes before it ends, such as a page render the visitor abandons, an aborted RSC fetch or an `/mcp` event stream the client closes, writes its line then, with the status code it had sent. A request accepted before the subscription starts has no listener, so its line is written when its response finishes. Visitor counts come from this line, because Cloud Run's own request log records Cloudflare's edge address as the client.
 
 ```json
-{"severity":"INFO","message":"request","caller":"ip:3f9c0a…","method":"GET","path":"/embed/abc123","host":"developmentfinance.org","kind":"page","status":200,"bot":false}
+{"severity":"INFO","message":"request","caller":"ip:3f9c0a…","method":"GET","path":"/embed/abc123","host":"developmentfinance.org","kind":"page","status":200,"finished":true,"bot":false,"prerender":false}
 ```
 
-`caller` is the same `ip:<digest>` the `/mcp` proxy sends upstream as `x-dfo-caller` (`src/lib/proxy/caller.ts`): an HMAC of the client address under `CALLER_HMAC_KEY`, with the address taken from `X-Forwarded-For` at the `TRUSTED_PROXY_HOPS` position. It is null when the key is unset or the request carries no usable address. `kind` is `page`, `api` (`/api` and `/mcp`), `rsc` (a client-side navigation) or `prefetch` (a Next.js or browser prefetch). Embeds under `/embed/` are logged as pages. `/_next/` files, images and fonts are not logged. `status` is the response's HTTP status code. `bot` is true when the `User-Agent` header is missing or empty, or when the `isbot` package flags it: crawlers such as Googlebot, bingbot and ClaudeBot, link previews such as Slack's and Discord's, headless browsers and tools such as curl. A client that sends a browser's user agent counts as a reader. The line holds no raw IP address, query string, cookie, `Authorization` header or user agent.
+`caller` is the same `ip:<digest>` the `/mcp` proxy sends upstream as `x-dfo-caller` (`src/lib/proxy/caller.ts`): an HMAC of the client address under `CALLER_HMAC_KEY`, with the address taken from `X-Forwarded-For` at the `TRUSTED_PROXY_HOPS` position. It is null when the key is unset or the request carries no usable address. `path` is the request target as it arrived, captured when the request starts so a later rewrite does not change it. It has no query string or fragment, and dot segments are resolved, so `/a/../mcp` is logged as `/mcp`. A target such as `//mcp` is a path and not a host, and is logged as `//mcp`. An absolute-form target such as `http://host/path` is logged as its pathname, and an absolute-form target whose URL does not parse is logged as the raw target cut at its first `?` or `#`. The asterisk-form target of `OPTIONS *` is logged as `*`, with kind `page`. `kind` is `api` for `/api`, `/mcp`, `/openapi.json` and `/codes` and every path below each of them (`/apiary` is not one). That covers the site's own endpoints under `/api`, such as auth, events and share, as well as the platform proxy's routes. `kind` is `prefetch` for a Next.js prefetch (`Next-Router-Prefetch`) or a browser prefetch (`Sec-Purpose: prefetch`), `rsc` for a client-side navigation (`RSC: 1`) and `page` for everything else. A `Sec-Purpose` header does not change the kind of an RSC fetch or a Next.js prefetch. Embeds under `/embed/` are logged as pages. `/_next/` files, images and fonts are not logged, and are dropped before the line's headers are read or its caller computed. `status` is the response's HTTP status code. `finished` is true when the response ended normally and false when it closed first, so a status of 200 with `finished` false is a visitor who left a stream. A response that closes before the app sent a status has none of its own, so the line carries 499 (client closed request) with `finished` false. The 499 means the app sent no status before the connection closed. It also covers a response Node answered itself, such as a 408 on a request timeout or a 400 on a client error, which never passes through the app. A response that called `writeHead` but wrote nothing to the socket before the close is logged with that status and `finished` false. Infra's parse keeps the line and counts it as a 4xx, outside the readers' 2xx count. Infra's `website_requests_daily` counts a line whose response did not finish like a finished one. `prerender` is true for a document request (no `RSC` and no `Next-Router-Prefetch` header) with `Sec-Purpose: prefetch;prerender`: the browser renders the page ahead of time and shows it on activation with no second request, so that one line is the page view. It has kind `page` and not `prefetch`, which keeps infra's `website_requests_daily` counting it as a parsed page request. The line is a page view whether or not the visitor opens the page, because the server cannot see activation. `bot` is true when the `User-Agent` header is missing or empty, or when the `isbot` package flags it: crawlers such as Googlebot, bingbot and ClaudeBot, link previews such as Slack's and Discord's, headless browsers and tools such as curl. A client that sends a browser's user agent counts as a reader. The line holds no raw IP address, query string, cookie, `Authorization` header or user agent.
 
 The line reads the request's raw headers, which keep the `RSC` and `Next-Router-Prefetch` headers that Next.js removes from `request.headers`. Logging leaves request bodies unread, so API and MCP payloads stream to their handlers in full. A Next.js proxy would buffer those bodies and truncate them at its default 10 MB limit.