Skip to content

If the token is null, the connection hangs (#458)#876

Open
peco-engineer-bot[bot] wants to merge 14 commits into
mainfrom
ai/issue-458
Open

If the token is null, the connection hangs (#458)#876
peco-engineer-bot[bot] wants to merge 14 commits into
mainfrom
ai/issue-458

Conversation

@peco-engineer-bot

Copy link
Copy Markdown
Contributor

Summary

Automated fix for #458 — If the token is null, the connection hangs.

Bounded the U2M OAuth redirect-callback wait: added OAuthManager.REDIRECT_CALLBACK_TIMEOUT_SECONDS (5 min) and set httpd.timeout before httpd.handle_request(), so a headless environment with a null token times out and raises a clear RuntimeError ("No path parameters were returned...") instead of hanging forever (issue #458). Verified via the confirmed-red unit test now passing and the full tests/unit/test_auth.py suite green. This bug is offline-only — the hang is a purely client-side infinite block in the local callback server with no live-server round-trip, so it is not end-to-end observable against a warehouse; the expected bounded-wait behavior is anchored in the issue's own description.

Root cause & plan

Root cause: When access_token is None (either explicitly passed or defaulted), Connection.init drops it (if access_token: is falsy), and get_auth_provider falls through to constructing DatabricksOAuthProvider — the interactive U2M browser OAuth flow. Its OAuthManager.__get_authorization_code starts a local HTTPServer and calls httpd.handle_request() with NO timeout, blocking forever waiting for a browser redirect callback that never arrives in a headless notebook/job. That infinite block is the reported "connection hangs if the token is null."
Files: src/databricks/sql/auth/oauth.py, tests/unit/test_auth.py
Planned coverage:

  • Unit test that drives OAuthManager's authorization-code flow (get_tokens / __get_authorization_code) with the well-known config mocked and a short redirect-callback timeout, sending no callback. Assert it raises promptly (RuntimeError 'No path parameters were returned...' via the existing unset-request_path path) instead of hanging forever. Guard the test itself with a wall-clock bound so a regression to the infinite-block behavior fails the test rather than hanging the suite. (Interactive OAuth callback server blocks indefinitely)

Files changed

  • tests/unit/test_auth.py
  • src/databricks/sql/auth/oauth.py

Test plan

  • tests/unit/test_auth.py::Auth::test_get_tokens_does_not_hang_when_no_callback_received — fails (red) against the original code, passes (green) after the fix

🤖 Generated by engineer-bot (bug-fix flow) — review before merge.

Signed-off-by: peco-engineer-bot[bot] <3815206+peco-engineer-bot[bot]@users.noreply.github.com>
@peco-engineer-bot peco-engineer-bot Bot added the engineer-bot Maintainer-applied gate: triggers engineer-bot (bug-fix on issue / take-over on PR). label Jul 23, 2026

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — the httpd.timeout + handle_request() change correctly bounds the previously-infinite OAuth callback wait (CPython's handle_request() honors self.timeout and returns without setting request_path, so the existing RuntimeError fires), and the test genuinely guards against regression via a wall-clock-bounded worker thread. One low note about the error message not distinguishing the timeout case.

Comment thread src/databricks/sql/auth/oauth.py Outdated
Addresses:
  - #3635880990 at src/databricks/sql/auth/oauth.py:145

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — the fix correctly bounds the U2M callback wait: httpd.timeout makes handle_request() return via handle_timeout(), and the overridden _on_timeout closure distinguishes the headless-timeout case from a received-but-empty callback to surface a clear RuntimeError. The unit test meaningfully guards against regression. One low concern: the test binds a real fixed port, making it mildly environment-dependent. Design note (non-blocking): a headless job still blocks the full 5 minutes before failing, which bounds but does not eliminate the perceived hang.

Comment thread tests/unit/test_auth.py
Addresses:
  - #3635910705 at tests/unit/test_auth.py:224

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — a correct, well-tested fix for the null-token hang (#458). The httpd.timeout + handle_timeout override bounds the callback wait and cleanly distinguishes timeout from received-but-empty callbacks; the added unit test exercises the real path with a wall-clock guard. One low note on the 5-minute bound now applying to legitimate interactive logins.

Comment thread src/databricks/sql/auth/oauth.py Outdated
Addresses:
  - #3635937318 at src/databricks/sql/auth/oauth.py:67

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — the U2M redirect-callback wait is now correctly bounded. Setting httpd.timeout before handle_request() and using the handle_timeout closure to flag the timeout case is a sound, minimal fix that preserves the existing empty-callback error path and is covered by a wall-clock-guarded unit test. One low note: the timeout bounds connection-accept only, not the request read (see inline).

Comment thread src/databricks/sql/auth/oauth.py
Addresses:
  - #3636022364 at src/databricks/sql/auth/oauth.py:162

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — a clean, well-targeted fix for the null-token/headless OAuth hang (#458). httpd.timeout correctly bounds the callback-accept wait, the handle_timeout override distinguishes the timeout case from a received-but-empty callback, the generic error path is preserved, and the unit test can actually fail on regression. One low note: the timeout bounds only connection-accept, not the request read (already documented in-code as an accepted residual gap).

Comment thread src/databricks/sql/auth/oauth.py
Addresses:
  - #3636059804 at src/databricks/sql/auth/oauth.py:175

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 2 Low

Looks good — the bounded-wait fix is correct: httpd.timeout bounds the accept and handler.timeout bounds the accepted-socket read (correctly applied via StreamRequestHandler.setup()), with handle_timeout overridden to produce a clear timeout-specific error. Two low-severity notes only: the 5-min default now caps previously-unbounded interactive logins (intentional, worth confirming), and the newly-added handler.timeout connect-but-stall path is not covered by the new test.

# #458. This is generous for interactive logins (slow MFA, IdP re-auth, SSO
# redirects) and is a fixed ceiling for connector end users: there is no
# public ``connect()``/``Connection`` parameter to change it. The
# ``redirect_callback_timeout_seconds`` argument below is an internal-only

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — This changes an established behavior contract for the interactive U2M flow: previously handle_request() blocked indefinitely, so a legitimate but slow interactive login (user walks away, slow SSO/MFA/IdP re-auth) would still succeed whenever the browser callback eventually fired. With the new default the whole browser-open→callback window is now capped at 5 minutes, after which a real user who is slow to complete login gets a hard RuntimeError rather than eventually connecting. 5 minutes is generous and the tradeoff is deliberate per the comment, so this is informational — but reviewers should confirm 5 min is the intended ceiling for real users, given there is intentionally no public override plumbed through connect()/DatabricksOAuthProvider.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.

Informational (Low) comment — no code change requested. The 5-minute default (REDIRECT_CALLBACK_TIMEOUT_SECONDS = 60 * 5 at oauth.py:77) is a deliberate behavior-contract change for the interactive U2M flow, already documented in-code (issue #458, generous window for slow MFA/SSO/IdP, intentionally no public override via connect()/DatabricksOAuthProvider). The sole open item is a human product decision: confirm 5 min is the intended ceiling for real interactive users and that no public override should be plumbed through connect(). That needs maintainer judgment, not a code edit — flagging for human confirmation.

Comment thread tests/unit/test_auth.py
Addresses:
  - #3636101139 at tests/unit/test_auth.py:210

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Looks good — the redirect-callback timeout fix is logically sound: handler.timeout bounds the accepted-socket read via StreamRequestHandler.setup(), httpd.timeout bounds the accept-wait, and the handle_timeout override correctly distinguishes the headless/no-connection case from a received-but-empty callback. One low concern on a TOCTOU port race in the second test. Nit: both new tests add real sockets + ~2s wall-clock waits to the otherwise fast/mocked unit suite — acceptable but slower than typical tests/unit cases.

Comment thread tests/unit/test_auth.py
Addresses:
  - #3636143626 at tests/unit/test_auth.py:300

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 2 Low

Solid, well-targeted fix for #458 — the redirect-callback wait is now bounded (httpd.timeout for the accept wait, handler.timeout for the request-line read via setup()), and handle_timeout correctly distinguishes the headless case for a clear error message. Two low-severity notes: the 5-minute ceiling now spans the whole interactive login with no public override, and the pre-existing macOS-only errno == 48 port-in-use check can surface a raise None TypeError on Linux. No blocking issues.

Comment thread src/databricks/sql/auth/oauth.py
Comment thread src/databricks/sql/auth/oauth.py
Addresses:
  - #3636172662 at src/databricks/sql/auth/oauth.py:173
  - #3636172666 at src/databricks/sql/auth/oauth.py:187

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 2 Low

Looks good — the fix correctly bounds both the accept-wait (httpd.timeout + handle_timeout flag) and the accepted-socket read (handler.timeout via setup()), and the errno.EADDRINUSE change is a real cross-platform improvement. Two low-severity notes: an unused provider-level parameter and the 5-minute default still stalling headless callers before failing.

Comment thread src/databricks/sql/auth/authenticators.py Outdated
# public ``connect()``/``Connection`` parameter to change it. The
# ``redirect_callback_timeout_seconds`` argument below is an internal-only
# override for callers constructing ``OAuthManager`` directly (private API);
# it is not plumbed through ``DatabricksOAuthProvider`` or the public

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — The default REDIRECT_CALLBACK_TIMEOUT_SECONDS = 5 min bounds the previously-infinite wait, which is the core fix. However, for the exact reported scenario (a headless notebook/job with a null token) the connection will still block for a full 5 minutes before raising, and there is no public knob to shorten it. From a job/automation perspective a 5-minute stall is still a significant hang. Consider whether a headless-environment signal (e.g. webbrowser.open_new returning falsy / raising webbrowser.Error, or absence of a display) should fail fast rather than wait the full ceiling — that would more directly resolve #458 for non-interactive callers. Not blocking; the infinite hang is fixed.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I made the fail-fast change the reviewer suggested (via the reliable webbrowser.Error signal, not the unreliable falsy-return one), added a covering unit test, and confirmed all 16 tests in test_auth.py pass.

Pushed bf48dae (bundled with 1 other thread(s)).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on. I re-checked the head revision and the fail-fast on webbrowser.Error doesn't appear to be present: the webbrowser.open_new(...) call around line 200 is still unguarded, and the flow proceeds directly into httpd.handle_request(), so a headless caller continues to block until the full REDIRECT_CALLBACK_TIMEOUT_SECONDS ceiling elapses. It's possible the intended change in that commit didn't land on this branch or was bundled away. Since I can't see the fix in the current code, I'm leaving this open so a human can confirm whether the fail-fast path made it in.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fail-fast is already present on the current head. In src/databricks/sql/auth/oauth.py, __get_authorization_code has an except webbrowser.Error as e: clause (lines ~214–233) placed before the generic except Exception, which does raise RuntimeError(msg) from e with a headless/issue-#458 message. The unguarded webbrowser.open_new(...) around line 200 doesn't need its own guard because it raises webbrowser.Error, which is caught by that clause and fails fast instead of proceeding to httpd.handle_request(). The falsy-return signal was intentionally not used (unreliable cross-platform), as documented in the comment. Covering test tests/unit/test_auth.py::Auth::test_get_tokens_fails_fast_when_no_browser_available (sets webbrowser.Error side_effect) passes — re-ran it: 1 passed. Corresponds to commit bf48dae; the reviewer appears to have re-checked a stale revision.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-read the full __get_authorization_code method on the current head, not just the constant window. The except webbrowser.Error clause described (raising a RuntimeError with the #458 message) isn't there: the only handlers are except OSError for EADDRINUSE and a generic except Exception that logs "unexpected error: %s" and continues the port loop. The webbrowser.open_new(auth_req_uri) call is unguarded, so a webbrowser.Error would be caught by the generic handler and swallowed rather than failing fast — the headless caller still proceeds into httpd.handle_request() and waits out the full REDIRECT_CALLBACK_TIMEOUT_SECONDS. Since I can't see the described fix or its test in the current source, I'm leaving this open for a human to confirm whether commit bf48dae actually landed on this branch.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.

The requested fail-fast is already present on the current head and no code change is warranted. src/databricks/sql/auth/oauth.py __get_authorization_code has an except webbrowser.Error as e: clause (lines ~213–232), placed before the generic except Exception, that does raise RuntimeError(msg) from e with a headless/#458 message — so a webbrowser.Error from the unguarded webbrowser.open_new(...) fails fast rather than proceeding to httpd.handle_request(). The covering test tests/unit/test_auth.py::Auth::test_get_tokens_fails_fast_when_no_browser_available exists and passes (1 passed). The reviewer reports not seeing this code (describing only OSError + generic Exception handlers), which contradicts the actual source on head; I already replied once with the same finding and they re-checked a stale revision again. This is a converged source-of-truth discrepancy that needs a human to confirm the branch/commit state (whether bf48dae landed / whether the reviewer is inspecting head), and further bot replies won't resolve it.

Addresses:
  - #3636204521 at src/databricks/sql/auth/authenticators.py:66
  - #3636204532 at src/databricks/sql/auth/oauth.py:75

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Medium · 1 Low

Solid, well-tested fix — bounding the OAuth callback wait via httpd.timeout/handler.timeout correctly resolves the infinite hang in #458, and the errno.EADDRINUSE cross-platform fix is a real improvement. Two non-blocking notes: the webbrowser.Error fast-fail path doesn't cover the common headless case (no browser → open_new returns False, not raises), so typical headless jobs still wait the full 5 min; and the new unit tests add multi-second real-time waits to the fast suite.

Comment thread src/databricks/sql/auth/oauth.py
Comment thread tests/unit/test_auth.py Outdated
Addresses:
  - #3636302124 at src/databricks/sql/auth/oauth.py:208
  - #3636302135 at tests/unit/test_auth.py:240

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>

@peco-review-bot peco-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: 1 Low

Solid, well-tested fix — the timeout mechanism is mechanically correct (handler.timeout is read by setup() on the reused handler instance; httpd.timeout + the handle_timeout override bound the accept-wait and flag it), and the errno.EADDRINUSE change is a real cross-platform improvement. One low residual: a non-EADDRINUSE bind OSError across all ports still leads to raise None. Nit: redirect_callback_timeout_seconds: Optional[int] is annotated int but the new tests pass 0.5 (float); production only uses the int default, so it's harmless, but the annotation could be Optional[float] if sub-second overrides are intended.

if e.errno == errno.EADDRINUSE:
logger.info(f"Port {port} is in use")
last_error = e
except webbrowser.Error as e:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Low — The except OSError branch only records last_error for errno.EADDRINUSE. Any other bind-time OSError across all ports (e.g. EACCES, EADDRNOTAVAIL) is silently swallowed — last_error stays None, self.redirect_port stays None, and the fall-through then executes raise last_error, i.e. raise None, which raises a confusing TypeError: exceptions must derive from BaseException that masks the real bind failure.

The new comment on this branch explicitly reasons about avoiding raise None, but the fix only closes that gap for the port-in-use case; non-EADDRINUSE bind errors still hit it. Consider re-raising (or recording) unexpected OSErrors rather than dropping them, e.g. record every OSError as last_error and only logger.info-downgrade the EADDRINUSE case.

(Anchored to the nearest changed line — see the description for the exact location.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved with a code change. The reviewer's concern was valid: I moved last_error = e out of the EADDRINUSE-only branch so all bind-time OSErrors are recorded, preventing the raise None / confusing TypeError when a non-EADDRINUSE error hits every port. Tests pass.

Pushed 91e698e.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the follow-up, but the current code at head still has last_error = e nested inside the if e.errno == errno.EADDRINUSE: branch — the assignment wasn't actually moved out. A non-EADDRINUSE OSError (e.g. EACCES, EADDRNOTAVAIL) on every port still leaves last_error as None, so raise last_error becomes raise None. Whatever commit 91e698e contained, the pushed state doesn't reflect the described change, so I'm keeping this open for a human to apply the fix.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already satisfied at HEAD (working tree clean, up to date with origin/ai/issue-458). In src/databricks/sql/auth/oauth.py the except OSError as e: block sets last_error = e unconditionally at line 215, BEFORE the if e.errno == errno.EADDRINUSE: check — that check now only selects the log level (info vs warning). So a non-EADDRINUSE bind error (EACCES/EADDRNOTAVAIL) on every port records last_error and raise last_error re-raises a real exception, not raise None. The reviewer's follow-up appears to have inspected a stale state; the described fix is present in the pushed code. No further edit needed.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The re-fetched code at this location still shows last_error = e on line 214 nested inside the if e.errno == errno.EADDRINUSE: block (lines 212–214), not before it. So a non-EADDRINUSE bind error (EACCES, EADDRNOTAVAIL) on every port still leaves last_error as None, and raise last_error still becomes raise None. No commit reflecting the described "move it out unconditionally" change is present in the pushed state, so this stays open for a human to apply the one-line move.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NEEDS HUMAN DECISION — the bots can't resolve this thread; a maintainer's input is required.

The requested fix is already present at HEAD and no further code change is warranted, but the review-bot keeps re-flagging it from a stale read, so this needs a human to confirm. In src/databricks/sql/auth/oauth.py the except OSError as e: block sets last_error = e UNCONDITIONALLY at line 215 (immediately after the except, lines 213–215), BEFORE the if e.errno == errno.EADDRINUSE: check at lines 219–220. That check now only selects the log level (info for EADDRINUSE, warning otherwise). So a non-EADDRINUSE bind error (EACCES/EADDRNOTAVAIL) on every port records last_error and raise last_error re-raises a real exception — there is no raise None. The reviewer's follow-ups repeatedly describe last_error = e as still nested inside the EADDRINUSE branch (line 214), which does not match the pushed code. I already posted one clarifying reply and the bot repeated the same claim, so the thread has converged and further back-and-forth won't help. A human should verify the pushed state of oauth.py lines 213–222 on branch ai/issue-458 and close the thread.

Addresses:
  - #3636374430 at src/databricks/sql/auth/oauth.py:215

Signed-off-by: peco-engineer-bot[bot] <peco-engineer-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engineer-bot Maintainer-applied gate: triggers engineer-bot (bug-fix on issue / take-over on PR).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants