feat(exception-capture): implement thread-safe fixed-window rate limiting with post-limit sampling#655
Conversation
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Introduces client-side exception capture throttling to prevent flooding PostHog with repeated exception events.
Changes:
- Added a thread-safe fixed-window
ExceptionRateLimiterwith post-limit “heartbeat” sampling. - Integrated the rate limiter (and an additional sample-rate gate) into exception capture flow.
- Added unit tests validating rate limiter behavior, window resets, and parameter validation.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| posthog/rate_limiter.py | Adds the ExceptionRateLimiter implementation used to throttle exception capture. |
| posthog/test/test_rate_limiter.py | Adds unit coverage for rate limit behavior and configuration validation. |
| posthog/exception_capture.py | Wires the limiter into exception capture and adds probabilistic sampling. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Refactor test for invalid parameters to use parameterization for cleaner code and better coverage.
Removed the internal state retrieval method for testing.
Remove debug log for exception capture rate limit.
| if max_exceptions < 0: | ||
| raise ValueError("max_exceptions must be >= 0") |
| # Pull configurations dynamically from user-facing Client setups | ||
| max_exceptions = getattr(client, "exception_capture_max_per_window", 100) | ||
| window_seconds = getattr(client, "exception_capture_window_seconds", 60.0) | ||
| post_limit_every = getattr(client, "exception_capture_post_limit_every", 10) | ||
|
|
||
| self._sample_rate = getattr(client, "exception_capture_sample_rate", 1.0) | ||
|
|
||
| # Initialize the rate limiter engine | ||
| self._rate_limiter = ExceptionRateLimiter( | ||
| max_exceptions=max_exceptions, | ||
| window_seconds=window_seconds, | ||
| post_limit_every=post_limit_every, | ||
| ) |
| self._max = int(max_exceptions) | ||
| self._window = float(window_seconds) | ||
| self._post_every = int(post_limit_every) |
| Parameters | ||
| - max_exceptions: non-negative int, number of allowed events per window. | ||
| - window_seconds: positive float, window length in seconds. | ||
| - post_limit_every: positive int, after the limit, allow 1 in ``post_limit_every``. |
|
@PostHog/team-error-tracking do we need this? we dont have that in any other SDK i think (for exceptions at least) |
🚀 Description
Implements client-side exception rate limiting to resolve the open
TODOin
posthog/exception_capture.py. Introduces a dedicatedExceptionRateLimiterthat clamps exception traffic in local memory before any network transmission.
🛠️ Design Architecture
Dual-stage filtering pipeline:
[ Unhandled Exception Caught ]
│
▼
Stage 1: ExceptionRateLimiter (Fixed Window + Heartbeat)
├─► Acquires thread lock
├─► Checks window expiry, resets if elapsed
├─► count <= max ──► Allow
└─► count > max ──► Allow every Nth (heartbeat signal, count % post_limit_every == 0)
│
▼ (passed Stage 1)
Stage 2: Sampling (lock-free)
└─► random.random() > sample_rate ──► Drop
│
▼ (passed Stage 2)
[ Client Outbound Ingestion Queue ]
⚙️ New Client Configuration Parameters
exception_capture_max_per_window100exception_capture_window_seconds60.0exception_capture_sample_rate1.0🧪 Tests Added
test_allows_within_limit_and_handles_heartbeat— verifies first N pass, then 1-in-N heartbeattest_window_resets_counters_cleanly— verifies counter resets after window expirytest_invalid_parameters_raise_value_errors— verifies bad config raisesValueErrortest_post_every_one_allows_all_events_after_limit— verifiespost_limit_every=1bypasses throttle📝 Checklist
sampo addNone at runtime — new client config params use
getattrwith safe defaults.Existing
Clientinstances without these attributes will behave identically to before.