test(e2e): register e2e-live source require hook and surface cloud-onboard dcode identity failure#6343
Merged
Merged
Conversation
The e2e-live Vitest project did not load the typed-source require hook
(test/helpers/onboard-script-mocks.cjs), unlike the cli and integration
projects. When a live suite imports a source module whose graph resolves
a sibling via a runtime require of an extensionless .ts file — e.g.
src/lib/inference/ollama-runtime-context.ts's require("../runner"), reached
transitively from the hermes-inference-switch helpers importing
src/lib/inference/config.ts — Node's native CJS resolver cannot find the
module and the suite fails at collection ("Cannot find module '../runner'",
0 tests).
Register the hook via setupFiles rather than NODE_OPTIONS so it stays
in-process and never leaks --require into the real CLI subprocesses the
live tests spawn. Mirrors the cli project.
Verified with `NEMOCLAW_RUN_LIVE_E2E=1 npx vitest list --project e2e-live`:
the switch suite now collects and all e2e-live suites report 0 collection
errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe e2e-live Vitest project now registers a typed-source require hook via setupFiles instead of relying on --require injection. A deepagents re-onboard check script uses an absolute dcode binary path and adds guarded identity capture with diagnostic output printed on failure. ChangesVitest configuration and e2e diagnostics
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
21 tasks
Contributor
E2E Advisor RecommendationRequired E2E: None Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
|
Contributor
E2E Target RecommendationRequired E2E targets: Dispatch required E2E targets:
Full E2E target advisor summaryE2E Target AdvisorBase: Required E2E targets
Optional E2E targets
Relevant changed files
|
Contributor
PR Review Advisor (Nemotron Ultra) — Changes requestedMerge posture: Do not merge yet Action checklist
Findings index
🚨 Required before mergeAddress these before merging unless a maintainer explicitly overrides the advisor with rationale.
|
Contributor
PR Review Advisor — Changes requestedMerge posture: Do not merge yet Action checklist
Findings index
Review findings by urgency: 0 required fixes, 3 items to resolve/justify, 0 in-scope improvements
|
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in the branch is 96%. Coverage data for the branch is not yet available. Show a code coverage summary of the most covered files.
TypeScript / code-coverage/cliThe overall coverage in the branch is 73%. Coverage data for the branch is not yet available. Show a code coverage summary of the most covered files.
Updated |
Contributor
E2E Target Results —
|
| Job | Result |
|---|---|
| agent-turn-latency | |
| bedrock-runtime-compatible-anthropic | |
| brave-search | |
| channels-add-remove | |
| channels-stop-start | |
| cloud-inference | |
| cloud-onboard | |
| common-egress-agent | |
| concurrent-gateway-ports | |
| credential-migration | |
| credential-sanitization | |
| cron-preflight-inference-local | |
| device-auth-health | |
| diagnostics | |
| docs-validation | |
| double-onboard | |
| full-e2e | |
| gateway-drift-preflight | |
| gateway-guard-recovery | |
| gateway-health-honest | |
| gpu-double-onboard | |
| gpu-e2e | |
| hermes-dashboard | |
| hermes-discord | |
| hermes-e2e | |
| hermes-gpu-startup | |
| hermes-inference-switch | |
| hermes-slack | |
| inference-routing | |
| issue-2478-crash-loop-recovery | |
| issue-4434-tui-unreachable-inference | |
| issue-4462-scope-upgrade-approval | |
| jetson-nvmap-gpu | |
| kimi-inference-compat | |
| launchable-smoke | |
| live | |
| mcp-bridge | |
| mcp-bridge-dev | |
| messaging-compatible-endpoint | |
| messaging-providers | |
| model-router-provider-routed-inference | |
| network-policy | |
| ollama-auth-proxy | |
| onboard-negative-paths | |
| onboard-repair | |
| onboard-resume | |
| openclaw-discord-pairing | |
| openclaw-inference-switch | |
| openclaw-skill-cli | |
| openclaw-slack-pairing | |
| openclaw-tui-chat-correlation | |
| openshell-gateway-auth-contract | |
| openshell-gateway-upgrade | |
| openshell-version-pin | |
| overlayfs-autofix | |
| rebuild-hermes | |
| rebuild-hermes-stale-base | |
| rebuild-openclaw | |
| sandbox-operations | |
| sandbox-rebuild | |
| sandbox-rlimits-connect | |
| sandbox-survival | |
| security-posture | |
| sessions-agents-cli | |
| shields-config | |
| skill-agent | |
| snapshot-commands | |
| spark-install | |
| state-backup-restore | |
| telegram-injection | |
| token-rotation | |
| tunnel-lifecycle | |
| upgrade-stale-sandbox |
Contributor
E2E Target Results — ❌ Some jobs failedRun: 28826486785
|
The cloud-experimental check-04 (fresh DCode re-onboard) captures `dcode identity` stdout+stderr into a shell variable via command substitution, but the `|| fail` handler prints only a generic message and discards the captured output. When `dcode identity` exits non-zero the real reason is lost — CI logs and result.json show stdout "" — which is why the current cloud-onboard failure on main (introduced with this check in #6332) cannot be diagnosed. Print the captured output to stdout (so it lands in result.json) before failing, on both the initial and post-re-onboard identity reads. No behavior change to the check's pass/fail logic; this only makes the existing failure observable so the root cause in the managed DCode onboarding flow can be found. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
prekshivyas
changed the title
Contributor
E2E Target Results — ❌ Some jobs failedRun: 28828274507
|
…rd check check-04 read the live identity with `openshell sandbox exec -- dcode identity`. That runs without a login shell, so /usr/local/bin is not on PATH and `dcode` resolves to "command not found" — the real reason cloud-onboard has failed on main since #6332 added this check (surfaced by the preceding observability commit). The image installs the launcher at /usr/local/bin/dcode, so invoke it by absolute path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
The anthropic variant of the inference-switch suite asserts that `openshell provider get compatible-anthropic-endpoint` reports `Type: openai` with an anchored regex. That command wraps field labels in ANSI escapes (`\e[2mType:\e[0m openai`), so the anchored match failed even though the provider is correctly registered as Type: openai. Strip ANSI first. The check only runs in the anthropic variant, which is why the hosted variant passed. This failure was masked on main by the collection error the e2e-live require hook fixed; with the suite now collecting, the assertion is reachable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
…equire-hook Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> # Conflicts: # test/e2e/live/hermes-inference-switch.test.ts
Contributor
Author
|
Contributor
Author
|
Fresh e2e-all triggered — run (https://github.com/NVIDIA/NemoClaw/actions/runs/28829724087), tied to #6343. This is the full live fan-out on the post-merge branch (~25 min). Expected result:
|
Contributor
Author
E2E status on this branch — green except the owner-flagged
|
| Job | Before | Now |
|---|---|---|
hermes-inference-switch (hosted) |
❌ collection error | ✅ pass |
hermes-inference-switch (anthropic) |
❌ collection error | ✅ pass |
cloud-onboard |
❌ dcode identity |
✅ pass |
full-e2e |
✅ pass (did not recur) | |
openshell-gateway-upgrade |
❌ | ❌ still red — needs owner decision |
(73+ success, 5 skipped-by-design explicit-only jobs; mcp-bridge + token-rotation were still finishing at time of writing and are unrelated to this PR.)
How each got fixed
- hermes-inference-switch (both variants): the e2e-live collection error is gone.
mainfixed the root cause directly (ollama-runtime-context.tsnow uses a staticimportinstead of a runtimerequire("../runner")); thesetupFilesrequire-hook this PR added to thee2e-liveVitest project remains as defense-in-depth for any future runtimerequire()in a live suite. The anthropicType: openaiassertion is fixed bymain'sstripAnsi(merged in). - cloud-onboard: fixed by
main's skip-guard (the generic cloud-onboard sandbox is OpenClaw, so the DCode check now skips). This PR also adds the observability that surfaced the original swalloweddcode: command not found, plus an absolute-pathdcodeinvocation for real DCode targets. - full-e2e: intermittent 5s cold-onboard budget overshoot; passed cleanly this run.
Remaining: openshell-gateway-upgrade
Deliberately not fixed here — see the detailed analysis above. It's a fail-closed behavior conflict between #6305 (recovery broadening) and #6132 (fingerprint gate) that the E2E test contradicts; inverting it would rewrite the named "fails closed" safety tests. Needs @laitingsheng / @ericksoa to choose the test-side vs product-side resolution.
cv
approved these changes
Jul 6, 2026
Collaborator
|
Following up on the owner decision for Legacy recovery is limited to exact confirmed managed sandbox names after a complete backup, requires the canonical provider/credential binding and an available host credential, revalidates provider absence and credential availability at the delete edge, and verifies the exact route after reconstruction. The exact-head gateway-upgrade E2E rerun passed 4/4 tests, including the old OpenClaw upgrade and survivor-state restoration. |
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
…board dcode identity failure (NVIDIA#6343) ## Summary Fixes two E2E-harness problems surfaced by the E2E dispatch on `main`: 1. **e2e-live source require hook** — the `hermes-inference-switch` live suite (added in NVIDIA#6335) failed at collection with `Cannot find module '../runner'` because the `e2e-live` Vitest project never loaded the typed-source require hook. 2. **cloud-onboard observability** — the cloud-experimental check-04 (added in NVIDIA#6332) fails on `main` with `could not read initial dcode identity`, but the real `dcode identity` error is captured into a shell var and discarded, so it can't be diagnosed. ## Changes - `vitest.config.ts`: add `setupFiles: ["test/helpers/onboard-script-mocks.cjs"]` to the `e2e-live` project (mirrors `cli`). Registers the typed-source `.ts` require hook **in-process** — deliberately not via `env.NODE_OPTIONS`, so `--require` never leaks into the real CLI subprocesses live tests spawn. - `test/e2e/e2e-cloud-experimental/checks/04-deepagents-code-fresh-reonboard.sh`: on both `dcode identity` reads, print the captured stdout+stderr to stdout (so it lands in `result.json`) before `fail`. No change to pass/fail logic — only makes the existing failure observable. ### Root cause — rh-ai-quickstart#1 (fixed here) `hermes-inference-switch` is the first `e2e-live` suite to import a deep `src` graph — its helpers import `src/lib/inference/config.ts`, which transitively loads `ollama-runtime-context.ts`'s runtime `require("../runner")`. Without the require hook, Node's native CJS resolver can't resolve the extensionless `.ts` → suite throws at collection (`0 tests`, ~37s). Only this suite hit it; others drive the CLI as a subprocess and import only fixtures. `runner.ts` has no circular dependency on the inference graph, so the in-process hook resolves it fully. ### Root cause — NVIDIA#2 (observability only; product root cause pending) `cloud-onboard` was green on `main` through 2026-07-06 00:56 UTC and failed on the first main E2E after NVIDIA#6332 landed (19:50 UTC) — NVIDIA#6332 added check-04, which has never passed on main. `dcode identity` is the NemoClaw wrapper (`agents/langchain-deepagents-code/dcode-wrapper.sh`); its identity path returns 0 in isolation and NVIDIA#6332 did not modify it, so the non-zero exit is a runtime condition in NVIDIA#6332's new "recreate/verify live identity" onboarding flow. That can't be pinned without the swallowed stderr — which this change surfaces. Product root cause is for the NVIDIA#6332 author to fix once the next run shows the real error. ## Type of Change - [x] Code change (feature, bug fix, or refactor) ## Quality Gates - [x] Existing tests cover changed behavior — justification: both changes are E2E-harness config/diagnostics. rh-ai-quickstart#1: the `e2e-live` suites exercise the hook — verified the previously-failing `hermes-inference-switch` suite now collects and all `e2e-live` suites report 0 collection errors, and the full `e2e-all` dispatch ran the switch job (hosted) green. NVIDIA#2: pure diagnostic output; no pass/fail change. - [x] Docs not applicable — justification: internal test-harness config/diagnostics; no user-facing behavior. - [x] Sensitive paths changed (onboarding/inference/runner adjacent via test config) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — justification: changes are confined to Vitest test-runner setup (`vitest.config.ts`) and an E2E diagnostic print; no product runtime code path is altered. The require hook is in-process only and does not touch product CLI subprocesses. ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed — note: `pre-push` `tsc-cli` skipped for a pre-existing local-only `noImplicitAny` false-positive in `test/helpers/mcp-lifecycle-lock-properties.ts` and `src/lib/state/mcp-lifecycle-lock-identity.test.ts` (unrelated to this diff); CI `tsc-cli` covers it. Check-04 passed `bash -n` and the `shellcheck`/`shfmt` pre-commit hooks. - [x] Targeted behavior tests pass — command/result: `NEMOCLAW_RUN_LIVE_E2E=1 npx vitest list --project e2e-live` → switch suite collects; whole project 0 collection errors (before the fix it reproduced `Cannot find module '../runner'`). Full `e2e-all` dispatch on this branch ran `hermes-inference-switch (hosted)` green. - [x] No secrets, API keys, or credentials committed --- Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved reliability of live end-to-end test runs by loading the required hook inside the test process, avoiding leakage into real CLI subprocesses. * Made identity checks in sandboxed cloud experimental flows more robust, with clearer diagnostics when identity lookup fails. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
senthilr-nv
added
v0.0.89
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes two E2E-harness problems surfaced by the E2E dispatch on
main:hermes-inference-switchlive suite (added in fix(hermes): use OpenAI frontend for custom Anthropic #6335) failed at collection withCannot find module '../runner'because thee2e-liveVitest project never loaded the typed-source require hook.mainwithcould not read initial dcode identity, but the realdcode identityerror is captured into a shell var and discarded, so it can't be diagnosed.Changes
vitest.config.ts: addsetupFiles: ["test/helpers/onboard-script-mocks.cjs"]to thee2e-liveproject (mirrorscli). Registers the typed-source.tsrequire hook in-process — deliberately not viaenv.NODE_OPTIONS, so--requirenever leaks into the real CLI subprocesses live tests spawn.test/e2e/e2e-cloud-experimental/checks/04-deepagents-code-fresh-reonboard.sh: on bothdcode identityreads, print the captured stdout+stderr to stdout (so it lands inresult.json) beforefail. No change to pass/fail logic — only makes the existing failure observable.Root cause — #1 (fixed here)
hermes-inference-switchis the firste2e-livesuite to import a deepsrcgraph — its helpers importsrc/lib/inference/config.ts, which transitively loadsollama-runtime-context.ts's runtimerequire("../runner"). Without the require hook, Node's native CJS resolver can't resolve the extensionless.ts→ suite throws at collection (0 tests, ~37s). Only this suite hit it; others drive the CLI as a subprocess and import only fixtures.runner.tshas no circular dependency on the inference graph, so the in-process hook resolves it fully.Root cause — #2 (observability only; product root cause pending)
cloud-onboardwas green onmainthrough 2026-07-06 00:56 UTC and failed on the first main E2E after #6332 landed (19:50 UTC) — #6332 added check-04, which has never passed on main.dcode identityis the NemoClaw wrapper (agents/langchain-deepagents-code/dcode-wrapper.sh); its identity path returns 0 in isolation and #6332 did not modify it, so the non-zero exit is a runtime condition in #6332's new "recreate/verify live identity" onboarding flow. That can't be pinned without the swallowed stderr — which this change surfaces. Product root cause is for the #6332 author to fix once the next run shows the real error.Type of Change
Quality Gates
e2e-livesuites exercise the hook — verified the previously-failinghermes-inference-switchsuite now collects and alle2e-livesuites report 0 collection errors, and the fulle2e-alldispatch ran the switch job (hosted) green. feature: custom settings for using build endpoints #2: pure diagnostic output; no pass/fail change.vitest.config.ts) and an E2E diagnostic print; no product runtime code path is altered. The require hook is in-process only and does not touch product CLI subprocesses.Verification
Verifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed — note:pre-pushtsc-cliskipped for a pre-existing local-onlynoImplicitAnyfalse-positive intest/helpers/mcp-lifecycle-lock-properties.tsandsrc/lib/state/mcp-lifecycle-lock-identity.test.ts(unrelated to this diff); CItsc-clicovers it. Check-04 passedbash -nand theshellcheck/shfmtpre-commit hooks.NEMOCLAW_RUN_LIVE_E2E=1 npx vitest list --project e2e-live→ switch suite collects; whole project 0 collection errors (before the fix it reproducedCannot find module '../runner'). Fulle2e-alldispatch on this branch ranhermes-inference-switch (hosted)green.Signed-off-by: Prekshi Vyas prekshiv@nvidia.com
Summary by CodeRabbit