Skip to content

Commit b795b9b

Browse files
jahoomaclaude
andcommitted
feat(ops): scheduled peak-hours scaling for the Render web service
The `web` service runs a flat 6 instances 24/7 with no autoscaling, so the predictable daily load peak (12-17Z) drives event-loop utilization to ~0.75 avg / 0.93 max and multi-second ingress queueing. Add a scheduled instance-floor scaler that scales the fleet out across the day. - scripts/ops/scale-web.ts: POSTs numInstances for the current UTC hour on a curve (peak 8), via Render's manual scale API. - scripts/ops/derive-scale-curve.ts: reproducibly re-derives the curve from Axiom (util↔load calibration × multi-day load shape; weekday/weekend split). - .github/workflows/web-peak-scale.yml: runs it hourly (idempotent). - scripts/logs/web-health.ts: reusable web saturation probe. - docs/web-scaling.md: strategy, curve derivation, caveats. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 03f49e9 commit b795b9b

5 files changed

Lines changed: 501 additions & 0 deletions

File tree

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
name: Web Peak-Hours Scaling
2+
3+
# Drives the Render `web` service instance count on a fixed daily curve so it can
4+
# absorb the 12-17Z load peak (see scripts/ops/scale-web.ts / docs/web-scaling.md).
5+
# The web tier is ingress/connection-bound and its load is clock-predictable, so a
6+
# time-based instance floor beats CPU-target autoscaling, which reacts late.
7+
#
8+
# Idempotent: runs hourly and just POSTs the curve's target for the current UTC
9+
# hour. A missed run self-corrects on the next tick. Requires native autoscaling
10+
# to be OFF on the service (the manual scale endpoint is ignored otherwise).
11+
#
12+
# Setup: add the `RENDER_API_KEY` secret (Render API key with access to the web
13+
# service). Optionally override the service id with the `RENDER_WEB_SERVICE_ID`
14+
# repo variable; the script defaults to the prod web service.
15+
16+
on:
17+
schedule:
18+
- cron: '5 * * * *' # hourly, at :05 to avoid the top-of-hour cron stampede
19+
workflow_dispatch:
20+
inputs:
21+
instances:
22+
description: 'Override: set an explicit instance count now (blank = use curve)'
23+
required: false
24+
type: string
25+
26+
jobs:
27+
scale:
28+
runs-on: ubuntu-latest
29+
timeout-minutes: 5
30+
steps:
31+
- uses: actions/checkout@v4
32+
- uses: oven-sh/setup-bun@v2
33+
- name: Scale web service
34+
env:
35+
RENDER_API_KEY: ${{ secrets.RENDER_API_KEY }}
36+
RENDER_WEB_SERVICE_ID: ${{ vars.RENDER_WEB_SERVICE_ID }}
37+
run: |
38+
set -euo pipefail
39+
if [ -n "${{ github.event.inputs.instances }}" ]; then
40+
bun scripts/ops/scale-web.ts --instances "${{ github.event.inputs.instances }}"
41+
else
42+
bun scripts/ops/scale-web.ts
43+
fi

docs/web-scaling.md

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# Web service scaling (Render)
2+
3+
The `web` service is **not** defined as IaC in this repo — there is no
4+
`render.yaml` blueprint. Its instance count, plan, env vars, build/start commands
5+
(`next start`), and health check (`/api/healthz`) are configured in the **Render
6+
dashboard**, and can be changed via the [Render API](https://api-docs.render.com).
7+
8+
- **Prod service id:** `srv-crm8estds78s73e7evd0` (derived from prod pod
9+
hostnames `srv-<id>-<replicaset>-<pod>` seen in Axiom `event_loop_lag.host`).
10+
- **Current state:** a **flat 6 instances, 24/7** (Axiom: `dcount(host)` per hour
11+
is exactly 6 every hour; the only variation is deploy pod-overlap).
12+
13+
## The problem
14+
15+
Load swings ~2x on a predictable daily curve, but capacity is fixed. From the
16+
2026-07 observability work (`event_loop_lag`, `chat_completion_concurrency`):
17+
18+
| | off-peak (02–07Z) | peak (12–17Z) |
19+
| --- | --- | --- |
20+
| event-loop `utilization` (avg / max) | ~0.40 / 0.62 | **0.73–0.76 / 0.93** |
21+
| chat-completion `queueMs` p95 | ~1.4s | **6–6.8s** |
22+
| max concurrent chat requests | ~43 | ~86 |
23+
24+
`queueMs` is ingress→handler wait (before any handler timer starts), and it
25+
tracks utilization almost exactly — so the peak wait is genuine instance
26+
backpressure, and the lever is to **scale out during the peak window**.
27+
28+
## Two strategies (mutually exclusive)
29+
30+
### A. Scheduled instance floor — recommended, and drafted here
31+
32+
Because the load is clock-predictable and the bottleneck (event-loop saturation →
33+
ingress queue) **leads** container CPU, a time-based floor is more precise than
34+
CPU-target autoscaling. Implemented as:
35+
36+
- `scripts/ops/scale-web.ts` — POSTs `numInstances` for the current UTC hour from
37+
a curve sized to hold peak per-instance utilization near ~0.4 (the healthy
38+
off-peak level).
39+
- `.github/workflows/web-peak-scale.yml` — runs it hourly (idempotent;
40+
self-correcting). `workflow_dispatch` accepts an explicit `instances` override.
41+
- `scripts/ops/derive-scale-curve.ts` — regenerates the curve from Axiom (util
42+
calibration × multi-day load shape, weekday/weekend split); paste its output
43+
back into `scale-web.ts`.
44+
45+
Curve (UTC hour → instances), re-derived over 14 days by
46+
`scripts/ops/derive-scale-curve.ts`:
47+
48+
```
49+
00–04: 6 05–07: 7 08: 8 09: 7 10: 8 11–14: 7
50+
15–17: 8 18–21: 7 22: 7 23: 6
51+
```
52+
53+
**How it was derived (and its caveats).** The utilization metric only went live
54+
~2026-07-03 23Z, so there is exactly one full util-day, **Sat 2026-07-04 — which
55+
ran ~65% above normal load.** Sizing straight off it over-fits that spike (it
56+
peaks at 14). Instead: calibrate `util ≈ 0.115 + 5.2e-5·load` from 07-04, then
57+
project onto the *normal* (spike-excluded) 14-day hourly load shape. A typical
58+
day peaks at **7–8** instances.
59+
60+
- **Weekday vs weekend: negligible.** Total load is within ~2% (weekday 153k /
61+
weekend 157k daily), and the per-hour instance counts differ by **≤1** — load
62+
is diurnal by UTC hour, not by day of week. So a single curve (the per-hour max
63+
of the two) is used rather than separate weekday/weekend curves.
64+
- **Trade-off — this sizes for NORMAL load.** On a normal day the flat-6 fleet
65+
already only reaches ~0.50 peak util / ~2.5s queue; the 6.8s crisis was the
66+
07-04 spike. Spike days will still see elevated queue at this floor. For
67+
spike-day protection, either raise the peak here or prefer **strategy B**
68+
(reactive autoscaling handles unpredictable spikes better than a fixed curve).
69+
- Re-run `derive-scale-curve.ts` once ≥2 full weekdays of real utilization exist
70+
to replace the load-projection with measured weekday util.
71+
72+
**Setup:** add repo secret `RENDER_API_KEY` (Render key with access to the
73+
service); optionally set repo variable `RENDER_WEB_SERVICE_ID`. Native
74+
autoscaling must be **OFF** — the manual scale endpoint
75+
(`POST /v1/services/{id}/scale`) is ignored when autoscaling is enabled.
76+
77+
Validate after rollout: re-run `bun scripts/logs/web-health.ts 24h` and confirm
78+
peak `queueMs` p95 drops toward the off-peak ~1.4s and peak `utilization` avg
79+
falls to ~0.4.
80+
81+
### B. Render native autoscaling — simpler ops, less precise
82+
83+
Dashboard → the service → Scaling: enable autoscaling, **min 6 / max 14**.
84+
Render only scales on **CPU and/or memory %**. Our bottleneck leads CPU (a
85+
single-threaded Node process can peg its event loop while multi-vCPU container
86+
CPU still looks moderate), so set a **low CPU target (~50%)** so it scales out
87+
*before* the queue builds; add a memory target (~70%) as a guard. Requires a Pro
88+
plan. This reacts to load (handles unexpected spikes) but lags the ~6s queue that
89+
builds before CPU crosses target — hence A is preferred for the known daily peak.
90+
91+
> Don't run A and B together: with autoscaling enabled, the scheduled `scale`
92+
> calls are silently ignored.
93+
94+
## Related
95+
96+
- `docs/logging.md` — the Axiom dataset and `scripts/logs/` query scripts.
97+
- The `freebuff-session-start-slowness-2026-07` investigation for the full
98+
saturation analysis and the other remediation levers.

scripts/logs/web-health.ts

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
/**
2+
* Ad-hoc web-backend health probe. Pulls the saturation signals added in the
3+
* 2026-07 observability work: event_loop_lag, chat_completion_concurrency
4+
* (queueMs), GC attribution, and error rates — bucketed hourly.
5+
*/
6+
import { axiom } from './lib'
7+
8+
const DATASET = process.env.AXIOM_DATASET || 'freebuff'
9+
const SINCE = process.argv[2] || '24h'
10+
11+
async function q(apl: string) {
12+
const res: any = await axiom().query(apl)
13+
if (res.buckets?.totals?.length) {
14+
return res.buckets.totals.map((g: any) => ({
15+
...(g.group ?? {}),
16+
...Object.fromEntries(
17+
(g.aggregations ?? []).map((a: any) => [a.alias ?? a.op, a.value]),
18+
),
19+
}))
20+
}
21+
return (res.matches ?? []).map((m: any) => ({ ...m.data, _time: m._time }))
22+
}
23+
24+
function table(title: string, rows: any[]) {
25+
console.log(`\n=== ${title} ===`)
26+
if (!rows.length) return console.log('(no rows)')
27+
console.table(rows)
28+
}
29+
30+
async function main() {
31+
const base = `['${DATASET}'] | where _time >= ago(${SINCE}) and service == "web"`
32+
33+
// 1. Event loop lag / utilization / GC, hourly
34+
table(
35+
'event_loop_lag (hourly): loop stalls, utilization, GC',
36+
await q(`${base}
37+
| where message startswith "[EventLoop]"
38+
| extend d = parse_json(data)
39+
| extend hr=tostring(bin(_time,1h))
40+
| extend p99=toreal(d.p99Ms), mx=toreal(d.maxMs), util=toreal(d.utilization),
41+
gcMax=toreal(d.gcMaxMs), gcTot=toreal(d.gcTotalMs), gcOld=toreal(d.gcOldGenMs)
42+
| summarize samples=count(), avgUtil=round(avg(util),3), maxUtil=round(max(util),3),
43+
avgP99ms=round(avg(p99),1), maxLagMs=round(max(mx),1),
44+
avgGcTotMs=round(avg(gcTot),1), maxGcMs=round(max(gcMax),1), maxGcOldMs=round(max(gcOld),1)
45+
by hr
46+
| sort by hr asc`),
47+
)
48+
49+
// 2. Number of distinct instances reporting (fleet size over time)
50+
table(
51+
'distinct web instances reporting event_loop_lag (hourly)',
52+
await q(`${base}
53+
| where message startswith "[EventLoop]"
54+
| extend d = parse_json(data)
55+
| summarize instances=dcount(tostring(d.host)) by bin(_time, 1h)
56+
| sort by _time asc`),
57+
)
58+
59+
// 3. Chat completion queue time (backpressure), hourly
60+
table(
61+
'chat_completion queueMs (ingress backpressure), hourly',
62+
await q(`${base}
63+
| extend d = parse_json(data)
64+
| where tostring(d.metric) == "chat_completion_concurrency"
65+
| where isnotnull(d.queueMs)
66+
| extend hr=tostring(bin(_time,1h))
67+
| extend qm=toreal(d.queueMs), cb=toreal(d.contentBytes), active=toreal(d.activeChatCompletionRequests)
68+
| summarize logged=count(), avgQueueMs=round(avg(qm),0), p95QueueMs=round(percentile(qm,95),0),
69+
maxQueueMs=round(max(qm),0), avgContentKB=round(avg(cb)/1024,1),
70+
maxActive=max(active)
71+
by hr
72+
| sort by hr asc`),
73+
)
74+
75+
// 4. Error/warn volume from web, hourly
76+
table(
77+
'web log volume by level (hourly)',
78+
await q(`${base}
79+
| extend hr=tostring(bin(_time,1h))
80+
| summarize n=count() by level, hr
81+
| sort by hr asc`),
82+
)
83+
84+
// 5. Top error messages last window
85+
table(
86+
'top web error messages',
87+
await q(`${base}
88+
| where level in ("error","fatal")
89+
| summarize n=count() by message
90+
| sort by n desc
91+
| limit 25`),
92+
)
93+
}
94+
95+
main().catch((e) => {
96+
console.error(e)
97+
process.exit(1)
98+
})

scripts/ops/derive-scale-curve.ts

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
/**
2+
* Derive the scale-web.ts hourly instance curve(s) from Axiom data.
3+
*
4+
* DATA CONSTRAINT (2026-07): the event-loop `utilization` metric (the capacity
5+
* signal) only went live ~2026-07-03 23:00Z, so there is exactly ONE full day of
6+
* it — Saturday 2026-07-04 — and that day ran ~65% above the normal daily load.
7+
* A straight multi-day utilization curve is therefore impossible yet. Request
8+
* LOAD, however, has months of history (chat_completion_concurrency count), and
9+
* the 14-day picture shows weekday vs weekend TOTAL load within ~2% — the real
10+
* variation is the hourly SHAPE, not weekday/weekend volume.
11+
*
12+
* METHOD (honest projection):
13+
* 1. Calibrate utilization-vs-load from the one util-day (07-04): least-squares
14+
* fit util ≈ a + b·load at the current flat BASELINE_INSTANCES.
15+
* 2. Pull the *normal* (07-04 spike excluded) 14-day hourly load shape, split
16+
* weekday vs weekend.
17+
* 3. Project util per hour per day-type, size instances to hold projected
18+
* per-instance util near TARGET_UTIL.
19+
*
20+
* Re-run once ≥2 full weekdays of real utilization exist to replace the
21+
* projection with measured weekday util.
22+
*/
23+
import { axiom } from '../logs/lib'
24+
25+
const DS = 'freebuff'
26+
const WINDOW = process.argv[2] || '14d'
27+
const CALIB_DAY = '2026-07-04' // the only full utilization day (also a load spike)
28+
const SKIP_DAYS = new Set([CALIB_DAY, '2026-07-05']) // spike day + partial today
29+
const BASELINE_INSTANCES = 6
30+
const TARGET_UTIL = 0.4
31+
const FLOOR = 6
32+
const CAP = 16
33+
34+
async function q(apl: string): Promise<Record<string, any>[]> {
35+
const res: any = await axiom().query(apl)
36+
if (res.buckets?.totals?.length)
37+
return res.buckets.totals.map((g: any) => ({
38+
...(g.group ?? {}),
39+
...Object.fromEntries(
40+
(g.aggregations ?? []).map((a: any) => [a.alias ?? a.op, a.value]),
41+
),
42+
}))
43+
return (res.matches ?? []).map((m: any) => ({ ...m.data, _time: m._time }))
44+
}
45+
46+
const hourOf = (iso: string) => new Date(iso).getUTCHours()
47+
const isWeekend = (iso: string) => [0, 6].includes(new Date(iso).getUTCDay())
48+
const sizeFor = (util: number) =>
49+
Math.min(CAP, Math.max(FLOOR, Math.round((BASELINE_INSTANCES * util) / TARGET_UTIL)))
50+
51+
async function main() {
52+
// --- 1. calibration day: util AND load per hour ---
53+
const calUtil = await q(`['${DS}']
54+
| where _time >= datetime(${CALIB_DAY}) and _time < datetime(${CALIB_DAY}) + 1d
55+
| where service == "web" and message startswith "[EventLoop]"
56+
| extend h = toint(bin(_time,1h) - startofday(_time)) / 3600000000000, u = toreal(parse_json(data).utilization)
57+
| summarize util = avg(u) by h | sort by h asc`)
58+
const calLoad = await q(`['${DS}']
59+
| where _time >= datetime(${CALIB_DAY}) and _time < datetime(${CALIB_DAY}) + 1d
60+
| where service == "web" and tostring(parse_json(data).metric) == "chat_completion_concurrency"
61+
| extend h = toint(bin(_time,1h) - startofday(_time)) / 3600000000000
62+
| summarize load = count() by h | sort by h asc`)
63+
const loadByH = new Map<number, number>(calLoad.map((r) => [Number(r.h), Number(r.load)]))
64+
const pts = calUtil
65+
.map((r) => ({ x: loadByH.get(Number(r.h)) ?? 0, y: Number(r.util) }))
66+
.filter((p) => p.x > 0 && p.y > 0)
67+
// least squares util = a + b*load
68+
const n = pts.length
69+
const sx = pts.reduce((s, p) => s + p.x, 0)
70+
const sy = pts.reduce((s, p) => s + p.y, 0)
71+
const sxy = pts.reduce((s, p) => s + p.x * p.y, 0)
72+
const sxx = pts.reduce((s, p) => s + p.x * p.x, 0)
73+
const b = (n * sxy - sx * sy) / (n * sxx - sx * sx)
74+
const a = (sy - b * sx) / n
75+
console.log(`Calibration (${CALIB_DAY}, ${n} hrs): util ≈ ${a.toFixed(4)} + ${b.toExponential(3)}·load`)
76+
77+
// --- 2. normal 14-day hourly load shape, weekday vs weekend ---
78+
const rows = await q(`['${DS}']
79+
| where _time >= ago(${WINDOW}) and service == "web"
80+
| where tostring(parse_json(data).metric) == "chat_completion_concurrency"
81+
| extend hr = tostring(bin(_time,1h)) | summarize n = count() by hr | sort by hr asc`)
82+
const wd: number[][] = Array.from({ length: 24 }, () => [])
83+
const we: number[][] = Array.from({ length: 24 }, () => [])
84+
for (const r of rows) {
85+
const iso = String(r.hr)
86+
if (SKIP_DAYS.has(iso.slice(0, 10))) continue
87+
;(isWeekend(iso) ? we : wd)[hourOf(iso)].push(Number(r.n) || 0)
88+
}
89+
const mean = (arr: number[]) => (arr.length ? arr.reduce((s, x) => s + x, 0) / arr.length : 0)
90+
91+
// --- 3. project util per hour per day-type, size instances ---
92+
const build = (buckets: number[][], label: string) => {
93+
const curve: number[] = []
94+
const table: any[] = []
95+
for (let h = 0; h < 24; h++) {
96+
const load = mean(buckets[h])
97+
const projUtil = a + b * load
98+
const inst = sizeFor(projUtil)
99+
curve.push(inst)
100+
table.push({ hourUTC: h, avgLoad: Math.round(load), projUtil: Math.round(projUtil * 1000) / 1000, instances: inst })
101+
}
102+
console.log(`\n--- ${label} (UTC) ---`)
103+
console.table(table)
104+
return curve
105+
}
106+
const wdCurve = build(wd, 'WEEKDAY (Mon–Fri)')
107+
const weCurve = build(we, 'WEEKEND (Sat–Sun)')
108+
const merged = wdCurve.map((v, i) => Math.max(v, weCurve[i])) // safe single curve
109+
110+
const fmt = (c: number[]) =>
111+
c.map((v, i) => (i % 12 === 0 ? `\n ${v}` : `${v}`)).join(', ')
112+
const delta = wdCurve.map((v, i) => Math.abs(v - weCurve[i]))
113+
console.log(`\nmax |weekday−weekend| across hours: ${Math.max(...delta)} instance(s)`)
114+
console.log(`WEEKDAY_CURVE = [${fmt(wdCurve)},\n]`)
115+
console.log(`WEEKEND_CURVE = [${fmt(weCurve)},\n]`)
116+
console.log(`MERGED (max) = [${fmt(merged)},\n]`)
117+
const peak = (c: number[]) => Math.max(...c)
118+
const avg = (c: number[]) => Math.round((c.reduce((s, v) => s + v, 0) / c.length) * 10) / 10
119+
console.log(
120+
`\nweekday peak ${peak(wdCurve)} avg ${avg(wdCurve)} · weekend peak ${peak(weCurve)} avg ${avg(weCurve)} · merged peak ${peak(merged)} avg ${avg(merged)}`,
121+
)
122+
}
123+
124+
main().catch((e) => {
125+
console.error(e)
126+
process.exit(1)
127+
})

0 commit comments

Comments
 (0)