Skip to content

fix(analytics): thread-safe polling start + surface fetch errors (SDK-81, SDK-78)#155

Merged
tylerjroach merged 3 commits into
masterfrom
fix/sdk-85-thread-safe-polling-start
Jul 21, 2026
Merged

fix(analytics): thread-safe polling start + surface fetch errors (SDK-81, SDK-78)#155
tylerjroach merged 3 commits into
masterfrom
fix/sdk-85-thread-safe-polling-start

Conversation

@tylerjroach

@tylerjroach tylerjroach commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Bundles two small audit fixes in the polling loop — both local_flags_provider.rb, both touching the same begin/rescue block.

SDK-81 — thread-safe polling start

start_polling_for_definitions! had a check-then-act race: if @config[:enable_polling] && !@polling_thread ran unsynchronized, so two concurrent callers could both see @polling_thread == nil, both assign Thread.new { ... }, and the earlier thread would become orphaned but keep polling forever.

Add a @polling_mutex guarding the polling-thread lifecycle:

  • start_polling_for_definitions!: under the mutex, bail early if @polling_thread already exists, otherwise spawn the thread.
  • stop_polling_for_definitions!: snapshot @polling_thread and @stop_polling under the mutex, then join outside the mutex so a long-running join can't block a concurrent start! for a full poll interval.

SDK-78 — surface polling-loop errors

The loop caught StandardError and dispatched only to @error_handler, whose default Mixpanel::ErrorHandler#handle is a no-op. Schema drift (NoMethodError, JSON::ParserError, TypeError, …) looped forever undetected unless the user had configured a custom handler.

Add a log_polling_error helper that always warns to $stderr before dispatching to @error_handler. Visibility no longer depends on configuration. Matches the de facto convention across every other Mixpanel SDK polling loop (mixpanel-python logger.exception, mixpanel-java logger.log(WARNING), mixpanel-go log.Printf, mixpanel-node this.logger?.error) — all log unconditionally and keep polling. Crashing the polling thread on the first bad payload would freeze local evaluation at whatever variants were last cached — strictly worse than continuing.

Context

Linear: SDK-81, SDK-78. Same audit-driven cross-SDK push; for SDK-81, Ruby is one of three affected (also Java #91, Node #278). SDK-78 was Ruby-only — the other SDKs already log unconditionally.

Test plan

  • Full flags spec suite (47 examples) passes
  • "is idempotent under concurrent start calls and does not leak threads" (SDK-81) spins up 8 contender threads behind a Queue gate; asserts only 1 new background thread exists after they all return. Before the fix the count would be 8.
  • "warns to stderr when fetch raises and no error_handler is configured" (SDK-78) builds the provider with nil error_handler + 500 response, asserts the warn lands on stderr.
  • "surfaces unexpected errors (schema drift) instead of swallowing them silently" (SDK-78) injects NoMethodError via allow(...).to receive(:fetch_flag_definitions), asserts both the error_handler dispatch AND the stderr warning fire.

🤖 Generated with Claude Code

@tylerjroach
tylerjroach requested review from a team and tdumitrescu June 29, 2026 17:42
@linear-code

linear-code Bot commented Jun 29, 2026

Copy link
Copy Markdown

SDK-85

SDK-81

SDK-78

@codecov

codecov Bot commented Jun 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.98%. Comparing base (961ee20) to head (343f8d4).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #155      +/-   ##
==========================================
- Coverage   97.11%   96.98%   -0.13%     
==========================================
  Files          15       15              
  Lines         762      764       +2     
==========================================
+ Hits          740      741       +1     
- Misses         22       23       +1     
Flag Coverage Δ
openfeature 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tylerjroach tylerjroach changed the title fix(flags): guard polling start so concurrent callers don't leak threads (SDK-85) fix(analytics): guard polling start so concurrent callers don't leak threads (SDK-85) Jun 29, 2026
@tylerjroach tylerjroach changed the title fix(analytics): guard polling start so concurrent callers don't leak threads (SDK-85) fix(analytics): guard polling start so concurrent callers don't leak threads (SDK-81) Jun 29, 2026
@tylerjroach tylerjroach changed the title fix(analytics): guard polling start so concurrent callers don't leak threads (SDK-81) fix(analytics): thread-safe polling start + surface fetch errors (SDK-81, SDK-78) Jun 29, 2026
@tylerjroach

Copy link
Copy Markdown
Contributor Author

@greptileai

@greptile-apps

greptile-apps Bot commented Jul 2, 2026

Copy link
Copy Markdown

Confidence Score: 5/5

Safe to merge — both changes are narrowly scoped, the polling lifecycle and mutex structure are unchanged, and two new regression tests cover the fixed paths.

The diff is minimal: one inner rescue block added around the initial fetch, and one warn prepended in safe_handle_error. Neither alters the threading model, mutex ordering, or stop/start lifecycle. The new specs exercise both code paths directly.

No files require special attention.

Important Files Changed

Filename Overview
lib/mixpanel-ruby/flags/local_flags_provider.rb Wraps the initial fetch in a begin/rescue so a transient failure no longer aborts thread spawning; adds unconditional stderr warn to safe_handle_error so schema-drift errors are visible without a custom handler.
spec/mixpanel-ruby/flags/local_flags_spec.rb Adds two targeted regression tests for SDK-78: one verifying stderr output when no error_handler is configured, one confirming the polling thread still spawns after an initial-fetch 500.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Caller
    participant LocalFlagsProvider
    participant HTTP as HTTP / fetch_flag_definitions
    participant Thread as Polling Thread
    participant Stderr as $stderr

    Caller->>LocalFlagsProvider: start_polling_for_definitions!
    LocalFlagsProvider->>LocalFlagsProvider: "@polling_mutex: @stop_polling = false"

    LocalFlagsProvider->>HTTP: "fetch_flag_definitions (initial, outside @lifecycle_mutex)"
    alt fetch succeeds
        HTTP-->>LocalFlagsProvider: flag definitions
    else fetch fails (SDK-78 fix)
        HTTP-->>LocalFlagsProvider: StandardError
        LocalFlagsProvider->>Stderr: warn "[Mixpanel] Failed to fetch..."
        LocalFlagsProvider->>LocalFlagsProvider: "@error_handler.handle(e) if present"
    end

    LocalFlagsProvider->>LocalFlagsProvider: "@lifecycle_mutex: check stop + alive?"
    LocalFlagsProvider->>Thread: "Thread.new (if enable_polling && !alive)"

    loop every polling_interval_in_seconds
        Thread->>Thread: "@polling_mutex: wait on ConditionVariable or check @stop_polling"
        Thread->>HTTP: fetch_flag_definitions
        alt fetch fails
            HTTP-->>Thread: StandardError
            Thread->>Stderr: warn "[Mixpanel] Failed to fetch..."
        end
    end

    Caller->>LocalFlagsProvider: stop_polling_for_definitions!
    LocalFlagsProvider->>LocalFlagsProvider: "@lifecycle_mutex: @stop_polling=true + broadcast"
    LocalFlagsProvider->>Thread: join
    Thread-->>LocalFlagsProvider: exits
    LocalFlagsProvider->>LocalFlagsProvider: "@polling_thread = nil"
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Caller
    participant LocalFlagsProvider
    participant HTTP as HTTP / fetch_flag_definitions
    participant Thread as Polling Thread
    participant Stderr as $stderr

    Caller->>LocalFlagsProvider: start_polling_for_definitions!
    LocalFlagsProvider->>LocalFlagsProvider: "@polling_mutex: @stop_polling = false"

    LocalFlagsProvider->>HTTP: "fetch_flag_definitions (initial, outside @lifecycle_mutex)"
    alt fetch succeeds
        HTTP-->>LocalFlagsProvider: flag definitions
    else fetch fails (SDK-78 fix)
        HTTP-->>LocalFlagsProvider: StandardError
        LocalFlagsProvider->>Stderr: warn "[Mixpanel] Failed to fetch..."
        LocalFlagsProvider->>LocalFlagsProvider: "@error_handler.handle(e) if present"
    end

    LocalFlagsProvider->>LocalFlagsProvider: "@lifecycle_mutex: check stop + alive?"
    LocalFlagsProvider->>Thread: "Thread.new (if enable_polling && !alive)"

    loop every polling_interval_in_seconds
        Thread->>Thread: "@polling_mutex: wait on ConditionVariable or check @stop_polling"
        Thread->>HTTP: fetch_flag_definitions
        alt fetch fails
            HTTP-->>Thread: StandardError
            Thread->>Stderr: warn "[Mixpanel] Failed to fetch..."
        end
    end

    Caller->>LocalFlagsProvider: stop_polling_for_definitions!
    LocalFlagsProvider->>LocalFlagsProvider: "@lifecycle_mutex: @stop_polling=true + broadcast"
    LocalFlagsProvider->>Thread: join
    Thread-->>LocalFlagsProvider: exits
    LocalFlagsProvider->>LocalFlagsProvider: "@polling_thread = nil"
Loading

Reviews (6): Last reviewed commit: "Merge branch 'master' into fix/sdk-85-th..." | Re-trigger Greptile

Comment thread lib/mixpanel-ruby/flags/local_flags_provider.rb Outdated
Comment thread spec/mixpanel-ruby/flags/local_flags_spec.rb Outdated
@tylerjroach

Copy link
Copy Markdown
Contributor Author

Pushed d8682c5 addressing both Greptile threads.

P1 — concurrent stop+start race (local_flags_provider.rb:65): replaced the shared @stop_polling boolean with a per-thread [false] cell captured by closure. Each polling thread checks its own cell, so a subsequent start_polling_for_definitions! allocating a fresh cell cannot clear the previous thread's stop signal. Also named the thread mixpanel-flags-poller for debuggability. Added a new spec stop+start does not leave the previous poller running that would fail on the old shared-flag design.

P2 — fragile Thread.list.size baseline (local_flags_spec.rb:790): the concurrent-start test now filters by t.name == 'mixpanel-flags-poller' instead of diffing Thread.list.size, so unrelated background threads (RSpec workers, GC finalizers) can't perturb the assertion.

All 48 specs pass locally.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens Mixpanel::Flags::LocalFlagsProvider’s polling loop by making polling thread startup/shutdown thread-safe and by ensuring polling fetch failures are visible even when no custom error_handler is configured.

Changes:

  • Add a mutex + per-thread stop signal to prevent duplicate/orphaned polling threads under concurrent start_polling_for_definitions! calls.
  • Add log_polling_error to always warn to stderr before delegating to @error_handler.
  • Add specs covering concurrent start idempotency, stop+start behavior, and stderr warning behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
lib/mixpanel-ruby/flags/local_flags_provider.rb Adds mutex-guarded polling lifecycle + unconditional stderr logging for polling errors.
spec/mixpanel-ruby/flags/local_flags_spec.rb Adds regression tests for concurrency/idempotency and for surfacing polling errors to stderr.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lib/mixpanel-ruby/flags/local_flags_provider.rb Outdated
Comment thread lib/mixpanel-ruby/flags/local_flags_provider.rb Outdated
Comment thread lib/mixpanel-ruby/flags/local_flags_provider.rb Outdated
ketanmixpanel
ketanmixpanel previously approved these changes Jul 17, 2026

@ketanmixpanel ketanmixpanel left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

review copilot comments.

…tch failure

Two SDK-78 concerns addressed against master's current polling architecture
(PR #149's ConditionVariable + lifecycle-mutex design already covers the
original SDK-81 concurrent-start concerns, so those commits from an earlier
iteration of this PR are dropped as superseded).

- safe_handle_error now warns to stderr unconditionally in addition to
  dispatching to @error_handler. Prior to this, callers without an
  explicit error_handler saw no signal for schema drift (NoMethodError,
  JSON::ParserError, etc.) — the default Mixpanel::ErrorHandler#handle is
  a no-op, so the polling loop would swallow every failure forever.
  Matches the convention across mixpanel-python / mixpanel-java /
  mixpanel-go / mixpanel-node.

- start_polling_for_definitions! wraps just the initial fetch in an
  inner begin/rescue. A transient initial-fetch failure (network blip,
  HTTP 500) previously fell through to the method-level rescue and
  returned before the @lifecycle_mutex block ever spawned the poller —
  a single startup blip left the SDK permanently without polling until
  the caller manually retried. Now the loop still spawns and retries on
  the configured interval.
@tylerjroach
tylerjroach force-pushed the fix/sdk-85-thread-safe-polling-start branch from 7804025 to ffd1980 Compare July 21, 2026 03:04
@tylerjroach

Copy link
Copy Markdown
Contributor Author

Salvaged this PR against current master. Since PR #149 refactored the polling architecture entirely (ConditionVariable#wait + @lifecycle_mutex), the SDK-81 concurrent-start commits from the earlier iteration of this branch were architecturally superseded, so I've reset the branch to master and reapplied only the two concerns that master doesn't cover.

Force-pushed: ffd1980 — replaces the previous 4 commits with a single focused change. If Ketan's earlier approval was significant, worth a quick re-look.

What survives:

  1. SDK-78 — surface polling errors when no error_handler is configured
    safe_handle_error now also warns to stderr unconditionally. The default Mixpanel::ErrorHandler#handle is a no-op, so master's existing dispatch-only path silently swallowed schema drift (NoMethodError, JSON::ParserError, etc.) — the polling loop would run forever undetected. Matches the convention across mixpanel-python / mixpanel-java / mixpanel-go / mixpanel-node.

  2. Initial-fetch failure no longer kills polling
    start_polling_for_definitions! now wraps only the initial fetch_flag_definitions call in begin/rescue. Previously the outer method-level rescue caught a transient startup failure (network blip, HTTP 500) and returned before the @lifecycle_mutex block ever spawned the poller thread — a single blip left the SDK permanently without polling until the caller manually retried. The poller now still spawns and retries on the configured interval.

What was dropped (superseded by master PR #149):

  • 205e4be guard concurrent start — master's @lifecycle_mutex + .alive? check already handles this
  • d8682c5 per-thread stop signal — master's ConditionVariable#broadcast + @polling_thread&.join under lifecycle mutex covers this

Tests: 52/52 in local_flags_spec.rb pass locally (one pre-existing failure in tracker_spec.rb:23CGI.parse API change on Ruby 4.0 — is unrelated). CI will re-run.

The PR title still says "thread-safe polling start + surface fetch errors (SDK-81, SDK-78)" but the SDK-81 half is no longer in this PR since master already fixes it. Happy to retitle if you'd prefer.

@tylerjroach
tylerjroach enabled auto-merge (squash) July 21, 2026 14:11
@tylerjroach
tylerjroach merged commit 022417b into master Jul 21, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants