Skip to content

fix(automate): mark every Automate session in describe/it suites that…#52

Merged
shubhamkd merged 1 commit into
nightwatchjs:mainfrom
AakashHotchandani:SDK-5943
May 8, 2026
Merged

fix(automate): mark every Automate session in describe/it suites that…#52
shubhamkd merged 1 commit into
nightwatchjs:mainfrom
AakashHotchandani:SDK-5943

Conversation

@AakashHotchandani
Copy link
Copy Markdown
Contributor

… call browser.end() between tests

When a Nightwatch test file uses BDD describe/it with browser.end() in afterEach, only the last session of the file was being marked on the Automate dashboard. Earlier sessions stayed at status=done (untested).

Two distinct issues were fixed:

  1. Session marking on the Automate dashboard

    • Nightwatch's built-in transport only PUTs the session status via sendReasonToBrowserstack once, from testSuiteFinished. When tests call browser.end() between tests, every session except the last one ends without ever being marked.
    • Fix: in the plugin's TestRunFinished event handler, call browserstack_executor: setSessionStatus against the live session before the user's afterEach (and any browser.end()) runs. Nightwatch core's later GET-then-PUT respects the existing status, so the last session is not double-marked.
  2. Stale session_id in observability TestRun events

    • reporter.testResults.sessionId is populated only once per file via setSessionInfo() in startTestSuite(). When subsequent tests get a new BrowserStack session (after browser.end()), the metadata still points at the first session, so observability events for test 2+ were tagged with the wrong integrations.browserstack.session_id.
    • The async TestRunFinished event handler can also race the user's afterEach: in some envs browser.sessionId is already null by the time the handler runs.
    • Fix: snapshot (sessionId, capabilities) at TestRunStarted time (when the session is guaranteed live), store on TestMap.activeTestRuns, and resolve in sendTestRunEvent with priority:
      1. snapshot (always set, reliable)
      2. live browser.sessionId 3. metadata fallback

Verified against npm run sample-test (2 envs * 2 tests each in one describe):

Before: env1/env2 Login -> done, Locked -> passed
After : all 4 sessions -> passed, each with its own correct session_id

Also: surface the actual response body when build start fails (was printing [object Object] because makeRequest rejects with a plain object on non-2xx).

… call browser.end() between tests

When a Nightwatch test file uses BDD `describe`/`it` with `browser.end()` in
`afterEach`, only the *last* session of the file was being marked on the
Automate dashboard. Earlier sessions stayed at status=`done` (untested).

Two distinct issues were fixed:

1. Session marking on the Automate dashboard
   - Nightwatch's built-in transport only PUTs the session status via
     `sendReasonToBrowserstack` once, from `testSuiteFinished`. When tests
     call `browser.end()` between tests, every session except the last one
     ends without ever being marked.
   - Fix: in the plugin's `TestRunFinished` event handler, call
     `browserstack_executor: setSessionStatus` against the live session
     before the user's `afterEach` (and any `browser.end()`) runs.
     Nightwatch core's later GET-then-PUT respects the existing status, so
     the last session is not double-marked.

2. Stale `session_id` in observability TestRun events
   - `reporter.testResults.sessionId` is populated only once per file via
     `setSessionInfo()` in `startTestSuite()`. When subsequent tests get a
     new BrowserStack session (after `browser.end()`), the metadata still
     points at the first session, so observability events for test 2+ were
     tagged with the wrong `integrations.browserstack.session_id`.
   - The async `TestRunFinished` event handler can also race the user's
     `afterEach`: in some envs `browser.sessionId` is already null by the
     time the handler runs.
   - Fix: snapshot `(sessionId, capabilities)` at `TestRunStarted` time
     (when the session is guaranteed live), store on `TestMap.activeTestRuns`,
     and resolve in `sendTestRunEvent` with priority:
       1. snapshot (always set, reliable)
       2. live `browser.sessionId`
       3. metadata fallback

Verified against `npm run sample-test` (2 envs * 2 tests each in one
`describe`):

  Before: env1/env2 Login -> done, Locked -> passed
  After : all 4 sessions -> passed, each with its own correct session_id

Also: surface the actual response body when build start fails (was
printing `[object Object]` because `makeRequest` rejects with a plain
object on non-2xx).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@shubhamkd shubhamkd merged commit 2dbe174 into nightwatchjs:main May 8, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants