Problem Statement
Gateway-minted sandbox JWTs include a kid, but the gateway currently loads one signing key and one matching public verification key at startup. Rotation today requires replacing or deleting the JWT key Secret, letting certgen recreate key material, and restarting gateway pods. That invalidates outstanding supervisor tokens and gives operators no online rotation window.
Proposed Design
Add first-class online rotation support for gateway sandbox JWT keys:
- Load multiple verification keys keyed by
kid.
- Mark one key as the active signing key.
- Accept still-valid sandbox JWTs signed by configured verify-only keys during a bounded rotation window.
- Provide an operator-facing rotation flow, such as a
rotate-jwt task, command, or Helm hook, that creates a new active key without deleting the old verification key immediately.
- Document the rotation procedure, rollback behavior, and expected impact on active supervisors.
Alternatives Considered
The current delete-Secret-and-restart approach is simple, but it is disruptive and invalidates all active supervisor tokens at once. Keeping only short TTLs limits exposure but does not provide a safe operational rotation path. A multi-key verifier keyed by kid matches the existing JWT header shape and allows deliberate overlap between old and new keys.
Agent Investigation
PR #1404 added gateway-minted per-sandbox JWTs. Follow-up review noted that crates/openshell-server/src/auth/sandbox_jwt.rs and certgen currently support only one active keypair despite emitting kid. architecture/gateway.md now documents the current offline rotation behavior; this issue tracks the long-term online rotation implementation.
Problem Statement
Gateway-minted sandbox JWTs include a
kid, but the gateway currently loads one signing key and one matching public verification key at startup. Rotation today requires replacing or deleting the JWT key Secret, letting certgen recreate key material, and restarting gateway pods. That invalidates outstanding supervisor tokens and gives operators no online rotation window.Proposed Design
Add first-class online rotation support for gateway sandbox JWT keys:
kid.rotate-jwttask, command, or Helm hook, that creates a new active key without deleting the old verification key immediately.Alternatives Considered
The current delete-Secret-and-restart approach is simple, but it is disruptive and invalidates all active supervisor tokens at once. Keeping only short TTLs limits exposure but does not provide a safe operational rotation path. A multi-key verifier keyed by
kidmatches the existing JWT header shape and allows deliberate overlap between old and new keys.Agent Investigation
PR #1404 added gateway-minted per-sandbox JWTs. Follow-up review noted that
crates/openshell-server/src/auth/sandbox_jwt.rsand certgen currently support only one active keypair despite emittingkid.architecture/gateway.mdnow documents the current offline rotation behavior; this issue tracks the long-term online rotation implementation.