fix(automate): mark every Automate session in describe/it suites that…#52
Merged
Merged
Conversation
… call browser.end() between tests
When a Nightwatch test file uses BDD `describe`/`it` with `browser.end()` in
`afterEach`, only the *last* session of the file was being marked on the
Automate dashboard. Earlier sessions stayed at status=`done` (untested).
Two distinct issues were fixed:
1. Session marking on the Automate dashboard
- Nightwatch's built-in transport only PUTs the session status via
`sendReasonToBrowserstack` once, from `testSuiteFinished`. When tests
call `browser.end()` between tests, every session except the last one
ends without ever being marked.
- Fix: in the plugin's `TestRunFinished` event handler, call
`browserstack_executor: setSessionStatus` against the live session
before the user's `afterEach` (and any `browser.end()`) runs.
Nightwatch core's later GET-then-PUT respects the existing status, so
the last session is not double-marked.
2. Stale `session_id` in observability TestRun events
- `reporter.testResults.sessionId` is populated only once per file via
`setSessionInfo()` in `startTestSuite()`. When subsequent tests get a
new BrowserStack session (after `browser.end()`), the metadata still
points at the first session, so observability events for test 2+ were
tagged with the wrong `integrations.browserstack.session_id`.
- The async `TestRunFinished` event handler can also race the user's
`afterEach`: in some envs `browser.sessionId` is already null by the
time the handler runs.
- Fix: snapshot `(sessionId, capabilities)` at `TestRunStarted` time
(when the session is guaranteed live), store on `TestMap.activeTestRuns`,
and resolve in `sendTestRunEvent` with priority:
1. snapshot (always set, reliable)
2. live `browser.sessionId`
3. metadata fallback
Verified against `npm run sample-test` (2 envs * 2 tests each in one
`describe`):
Before: env1/env2 Login -> done, Locked -> passed
After : all 4 sessions -> passed, each with its own correct session_id
Also: surface the actual response body when build start fails (was
printing `[object Object]` because `makeRequest` rejects with a plain
object on non-2xx).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
osho-20
approved these changes
May 7, 2026
anish353
approved these changes
May 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
… call browser.end() between tests
When a Nightwatch test file uses BDD
describe/itwithbrowser.end()inafterEach, only the last session of the file was being marked on the Automate dashboard. Earlier sessions stayed at status=done(untested).Two distinct issues were fixed:
Session marking on the Automate dashboard
sendReasonToBrowserstackonce, fromtestSuiteFinished. When tests callbrowser.end()between tests, every session except the last one ends without ever being marked.TestRunFinishedevent handler, callbrowserstack_executor: setSessionStatusagainst the live session before the user'safterEach(and anybrowser.end()) runs. Nightwatch core's later GET-then-PUT respects the existing status, so the last session is not double-marked.Stale
session_idin observability TestRun eventsreporter.testResults.sessionIdis populated only once per file viasetSessionInfo()instartTestSuite(). When subsequent tests get a new BrowserStack session (afterbrowser.end()), the metadata still points at the first session, so observability events for test 2+ were tagged with the wrongintegrations.browserstack.session_id.TestRunFinishedevent handler can also race the user'safterEach: in some envsbrowser.sessionIdis already null by the time the handler runs.(sessionId, capabilities)atTestRunStartedtime (when the session is guaranteed live), store onTestMap.activeTestRuns, and resolve insendTestRunEventwith priority:browser.sessionId3. metadata fallbackVerified against
npm run sample-test(2 envs * 2 tests each in onedescribe):Before: env1/env2 Login -> done, Locked -> passed
After : all 4 sessions -> passed, each with its own correct session_id
Also: surface the actual response body when build start fails (was printing
[object Object]becausemakeRequestrejects with a plain object on non-2xx).