feat(gateway): behavioral anomaly detection for adaptive rate limiting (#615)

[Itodo-S]

Static per-IP / per-key limits are trivially bypassed by distributing requests across many IPs or rotating API keys. This adds unsupervised behavioral anomaly scoring that learns normal per-key usage and adaptively tightens limits when traffic looks anomalous.

Backend (TypeScript — jest-tested)

• backend/gateway/featureExtraction.ts — features: request rate, endpoint-distribution entropy, time-of-day, payload size, user-agent entropy, geographic (distinct-IP) spread.
• backend/gateway/isolationForest.ts — dependency-free Isolation Forest (unsupervised), deterministic via a seeded PRNG, anomaly score in [0, 1].
• backend/gateway/anomalyDetector.ts — trains on normal-traffic windows, scores new ones.
• backend/gateway/adaptiveRateLimit.ts — reduces the limit 50% past the threshold (default 0.8) and 90% past 0.95; allow-list (webhooks/health) + per-key manual override for false positives.
• backend/gateway/middleware/adaptiveRateLimitMiddleware.ts — Express-compatible (sliding window per key, scoring, enforcement, headers).
• backend/monitoring/anomalyMetrics.ts — per-key anomaly-score Prometheus gauge.

ml-service (Python/FastAPI — mirrors the model, no new deps)

• ml-service/anomaly/{isolation_forest,features,detector}.py — pure-Python port (matches the existing no-sklearn dependency set).
• ml-service/routers/anomaly.py — /v1/anomaly train / score / status, registered in main.py. The gateway can score in-process or delegate here.

Acceptance criteria

• ✅ Feature extraction (rate, endpoint distribution, time-of-day, payload size, UA entropy, geo spread)
• ✅ ML model: Isolation Forest (unsupervised) with configurable threshold
• ✅ Adaptive limiting: -50% past threshold (0.8), -90% past 0.95
• ✅ False-positive handling: allow-listed patterns + per-key manual override
• ✅ Prometheus anomaly-score metric per key

Testing

npx jest --config jest.backend.config.js backend/gateway/__tests__/anomalyRateLimit.test.ts   # 15 passing
cd ml-service && python -m pytest tests/test_anomaly.py                                          # 6 passing

Scope note

This delivers the working, tested core of a 200-point issue. The remaining infra-heavy criteria are implemented as documented follow-ups (see backend/gateway/README.md): real-time Slack/PagerDuty alerting on high-confidence attacks (the detector already surfaces high_confidence), the admin RateLimitDashboardScreen (mobile/app/screens/), and the seasonal model + weekly auto-retrain / drift alerting (ml-service/jobs/, retrain.py). Happy to follow up on any of these.

Closes #615

[drips-wave]

@Itodo-S Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits