docs: add release readiness checklist runbook and notes

Author: brianfunk

docs/OPERATIONS_RUNBOOK.md

@@ -2,13 +2,84 @@
 
 ## Scope
 
-Operational procedures for production incidents, recovery, and maintenance.
+Operational procedures for production incidents, recovery, and maintenance across Netlify + Supabase.
 
-## Runbook Areas
+## Ownership
 
-- Incident triage
-- Service degradation response
-- Data export job failures
-- Migration rollback steps
+- Incident Commander: on-call platform engineer.
+- Communications Lead: project owner or delegate.
+- Database Lead: engineer with Supabase migration permissions.
 
-Detailed runbook tasks are tracked in `/docs/plan/M08-quality-ci-cd-and-observability.md` and `/docs/plan/M09-release-readiness-and-pilot.md`.
+## Severity levels
+
+- `SEV-1`: Platform unavailable, cross-tenant risk, or data integrity risk.
+- `SEV-2`: Major feature degradation (submissions, moderation, exports, API unavailable).
+- `SEV-3`: Partial degradation with viable workaround.
+
+## First 15 minutes
+
+1. Declare incident severity and open incident channel.
+2. Confirm blast radius:
+   - Public browse/commenting
+   - Agency operations/moderation
+   - Public API exports
+3. Freeze deploys until incident is stabilized.
+4. Capture current signals:
+   - Netlify deploy health
+   - Supabase project status
+   - Recent function errors (`submit-comment`, `public-api`, `generate-export`)
+
+## Core playbooks
+
+### 1) Public submission failures
+
+- Check edge function logs for `submit-comment` errors.
+- Validate required environment variables: `HCAPTCHA_SECRET_KEY`, Supabase keys.
+- Check abuse-event volume spikes indicating CAPTCHA/rate-limit pressure.
+- Mitigation:
+  - Temporarily reduce traffic pressure using stricter edge throttles.
+  - If CAPTCHA provider outage occurs, switch to moderated maintenance banner for submissions.
+
+### 2) Public API degradation
+
+- Check `public-api` function logs and latency.
+- Verify `api_rate_limits` table growth and cleanup behavior.
+- Confirm RPC dependencies (`get_public_dockets`, `get_docket_public_detail`, `get_comment_detail`) return expected responses.
+- Mitigation:
+  - Increase edge function concurrency limits (where available).
+  - Apply temporary lower rate limits for abusive routes.
+
+### 3) Export pipeline failures
+
+- Inspect `exports` table rows in `failed`/stalled `processing`.
+- Check `generate-export` function errors and storage write failures.
+- Mitigation:
+  - Requeue failed jobs by creating replacement job records.
+  - Expire stale jobs and notify agency users.
+
+### 4) Data integrity or policy risk
+
+- Use `audit_events` for timeline reconstruction.
+- Use `abuse_events` to detect suspicious submission patterns.
+- If cross-tenant risk suspected: disable affected endpoints and enforce read-only mode for agency actions until triaged.
+
+## Rollback procedures
+
+### Application rollback (Netlify)
+
+1. Identify last known good deploy.
+2. Promote previous deploy in Netlify UI/CLI.
+3. Re-run smoke checks on public + agency flows.
+
+### Database rollback (Supabase)
+
+1. Stop deploy pipeline.
+2. Identify last migration applied before incident.
+3. Execute approved rollback playbook for affected migration set.
+4. Validate RLS policies and key RPCs before re-opening write traffic.
+
+## Post-incident actions
+
+1. Publish incident summary with root cause and customer impact.
+2. Create follow-up tasks for preventive fixes.
+3. Update this runbook if a playbook gap was discovered.

docs/RELEASE_NOTES.md

@@ -0,0 +1,45 @@
+# Release Notes
+
+## v1.0.0-rc1 (milestone-driven candidate)
+
+### Platform foundation
+
+- Added formal milestone tracking under `docs/plan` with append-only progress logs.
+- Reconciled schema contracts with application usage and added missing domain tables/RPCs.
+- Added CI pipeline checks for lint, typecheck, tests, and build.
+
+### Agency operations
+
+- Replaced mock docket list/dashboard/detail behavior with real Supabase-backed workflows.
+- Added docket edit-mode support in wizard route (`/agency/dockets/:id/edit`).
+- Canonicalized moderation queue behavior for `under_review` and `published` status families.
+
+### Public workflow
+
+- Hardened public comment submission with server-side CAPTCHA verification and rate limits.
+- Enforced per-docket identity policy and deterministic initial moderation status.
+- Added shared submission policy module and unit tests for identity/status rule coverage.
+
+### Open data and exports
+
+- Delivered public API v1 routes for dockets, comments, agencies, and exports.
+- Added API route rate limiting primitives and metadata behavior.
+- Completed export generation paths for CSV/ZIP/combined outputs.
+
+### Security and compliance
+
+- Added immutable `audit_events` and `abuse_events` telemetry.
+- Added PII-safe export masking controls.
+- Added control mapping and observability documentation.
+
+### Verification additions
+
+- Added schema contract tests ensuring `.from()`/`.rpc()` usage aligns with migrations.
+- Added tenant isolation policy tests for RLS assumptions.
+- Added submission policy tests for identity modes and status assignment.
+
+## Known open items before final GA tag
+
+- Fresh local migration replay validation is blocked without Docker daemon.
+- Pilot execution evidence and pilot-driven hardening are still required.
+- Final WCAG 2.1 AA evidence pack for required pages remains open.

docs/RELEASE_READINESS_CHECKLIST.md

@@ -0,0 +1,34 @@
+# Release Readiness Checklist
+
+This checklist is the operational gate for OpenComments release promotion.
+
+## Current execution snapshot (2026-02-11)
+
+- [x] `npm run lint` passes with warnings only (no lint errors).
+- [x] `npm run typecheck` passes.
+- [x] `npm run test:ci` passes.
+- [x] `npm run build` passes.
+- [ ] Local clean-database migration replay (`supabase db reset --local`) verified in this environment.
+- [x] Public API v1 route contracts documented.
+- [x] Submission workflow identity/rate/captcha controls implemented and tested.
+- [x] Tenant isolation policy assumptions are validated by tests.
+- [x] Export pipeline supports CSV/ZIP/combined outputs and status tracking.
+- [x] Immutable audit and abuse event logging are active in schema/migrations.
+- [ ] Pilot agency workflow sign-off completed.
+- [ ] Production cutover approval recorded.
+
+## Gate criteria
+
+A release candidate can be promoted only when:
+
+1. All mandatory checks above are complete.
+2. Any unchecked items are explicitly accepted as risk by release owner.
+3. Rollback path is verified for both app deploy and database migration.
+
+## Required evidence
+
+- CI run URL or captured command output for lint/typecheck/test/build.
+- Migration report for target environment.
+- API smoke test results for `/v1/dockets`, `/v1/comments`, `/v1/agencies`, `/v1/exports`.
+- Security checklist sign-off (abuse controls, audit logging, PII handling).
+- Rollback dry run notes.

docs/plan/M09-release-readiness-and-pilot.md

@@ -14,10 +14,10 @@ Complete production readiness verification and pilot execution.
 
 ## Implementation checklist
 
-- [ ] Define and execute readiness checklist.
+- [x] Define and execute readiness checklist.
 - [ ] Execute pilot and capture findings.
 - [ ] Apply pilot-driven fixes.
-- [ ] Publish runbook and release notes.
+- [x] Publish runbook and release notes.
 
 ## Acceptance criteria
 
@@ -35,3 +35,6 @@ Complete production readiness verification and pilot execution.
 ## Progress log (append-only)
 
 - 2026-02-11: Milestone initialized.
+- 2026-02-11: Published `docs/RELEASE_READINESS_CHECKLIST.md` and executed current engineering quality gates (lint/typecheck/test/build) as readiness evidence.
+- 2026-02-11: Expanded `docs/OPERATIONS_RUNBOOK.md` from placeholder to actionable incident and rollback playbooks for production operations.
+- 2026-02-11: Published `docs/RELEASE_NOTES.md` with milestone-driven v1.0.0-rc1 candidate notes and open GA blockers.

docs/plan/README.md

@@ -40,4 +40,4 @@ This folder tracks implementation progress for the OpenComments production roadm
 - `M06`: Completed
 - `M07`: In progress
 - `M08`: Completed
-- `M09`: Planned
+- `M09`: In progress