fix(inference): route local compatible endpoints through gateway#6331
Closed
chengjiew wants to merge 10 commits into
Closed
fix(inference): route local compatible endpoints through gateway#6331chengjiew wants to merge 10 commits into
chengjiew wants to merge 10 commits into
Conversation
Contributor
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage remains at 96%, unchanged from the branch. TypeScript / code-coverage/cliThe overall coverage in the branch remains at 77%, unchanged from the branch. Show a code coverage summary of the most impacted files.
Updated |
Contributor
|
🌿 Preview your docs: https://nvidia-preview-pr-6331.docs.buildwithfern.com/nemoclaw |
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds gateway-reachable rewriting for eligible loopback compatible endpoints, tests rewrite and credential-reuse flows, and updates local inference setup documentation and examples. ChangesGateway-reachable endpoint rewrite
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant setupRemoteProviderInference
participant gatewayReachableCompatibleEndpointUrl
participant upsertProvider
participant runOpenshell
setupRemoteProviderInference->>gatewayReachableCompatibleEndpointUrl: Rewrite resolved endpoint URL
gatewayReachableCompatibleEndpointUrl-->>setupRemoteProviderInference: Return gateway endpoint URL
setupRemoteProviderInference->>upsertProvider: Register gateway endpoint
setupRemoteProviderInference->>runOpenshell: Run inference set with verification flags
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
Contributor
E2E Target RecommendationRequired E2E targets: Dispatch required E2E targets:
Full E2E target advisor summaryE2E Target AdvisorBase: Required E2E targets
Optional E2E targets
Relevant changed files
|
Contributor
PR Review Advisor (Nemotron Ultra) — Changes requestedMerge posture: Do not merge yet Action checklist
Findings index
🚨 Required before mergeAddress these before merging unless a maintainer explicitly overrides the advisor with rationale.
|
Contributor
PR Review Advisor — Changes requestedMerge posture: Do not merge yet Action checklist
Findings index
Review findings by urgency: 0 required fixes, 2 items to resolve/justify, 0 in-scope improvements
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/lib/onboard/inference-providers/remote.ts (1)
104-165: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winCompute
gatewayEndpointUrlbefore the reuse branch.
The reuse path leaves it equal toresolvedEndpointUrl, soinference setwon’t add--no-verifyfor a provider that was already rewritten tohost.openshell.internal. That can bring back the loopback verify failure on resume/reuse; add a reuse-path test inremote.test.ts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/onboard/inference-providers/remote.ts` around lines 104 - 165, Compute gatewayEndpointUrl before entering the reuseGatewayCredentialWithoutLocalKey branch so it reflects the gateway-rewritten endpoint in both paths. Right now the reuse path leaves gatewayEndpointUrl at resolvedEndpointUrl, which prevents inference set from applying the same no-verify behavior for reused providers; update the remote flow in remote.ts around gatewayEndpointUrl/providerResult handling and add a reuse-path test in remote.test.ts to cover the resumed provider case.Source: Path instructions
🧹 Nitpick comments (2)
src/lib/onboard/inference-providers/remote.ts (1)
29-30: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueTrailing-slash normalization is hard to follow.
parsed.toString().replace(/\/$/, parsed.pathname === "/" ? "/" : "")is a no-op whenpathname === "/"(replaces trailing/with/) and only has an effect when the path is non-root and ends with/(strips it). Worth a short comment explaining intent, or simplifying, since this is on the path that computes the URL registered with the gateway.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/onboard/inference-providers/remote.ts` around lines 29 - 30, The URL normalization in the `remote.ts` host rewrite is hard to read because `parsed.toString().replace(...)` hides the intent of stripping only a non-root trailing slash. Simplify the logic in the `parsed.hostname` / return path, or add a short comment near the URL-building code to make it clear that root paths keep their slash while non-root paths have the trailing slash removed before registering the gateway URL.src/lib/onboard/inference-providers/remote.test.ts (1)
53-92: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winGood happy-path coverage; missing the credential-reuse branch.
This test correctly exercises the fresh-provision path and asserts on the real (non-mocked)
gatewayReachableCompatibleEndpointUrlbehavior via the injectedupsertProvider/runOpenshellmocks — that's an appropriate boundary. Consider adding a case withreuseGatewayCredentialWithoutLocalKey: truefor a loopbackcompatible-endpoint, which would surface thegatewayEndpointUrlrecomputation gap flagged inremote.ts(Lines 104-165).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/onboard/inference-providers/remote.test.ts` around lines 53 - 92, Add a second test in setupRemoteProviderInference that covers the reuseGatewayCredentialWithoutLocalKey branch for a loopback compatible-endpoint. Use the existing makeDeps fixture and verify that when reuseGatewayCredentialWithoutLocalKey is true, the code recomputes gatewayEndpointUrl correctly before calling upsertProvider and runOpenshell. Keep the current happy-path test, but extend coverage with this case so the credential-reuse behavior is asserted through the same setupRemoteProviderInference entry point.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib/onboard/inference-providers/remote.ts`:
- Around line 11-32: The loopback rewrite logic in
gatewayReachableCompatibleEndpointUrl misses default-port HTTP URLs and has a
dead IPv6 hostname check because URL.hostname preserves brackets for IPv6
literals. Update the loopback detection to rely on the normalized hostname
handling already in this helper, remove the ineffective "::1" comparison, and
allow http://localhost/v1-style URLs to be rewritten even when parsed.port is
empty so the compatible-endpoint path is still routed through
host.openshell.internal.
---
Outside diff comments:
In `@src/lib/onboard/inference-providers/remote.ts`:
- Around line 104-165: Compute gatewayEndpointUrl before entering the
reuseGatewayCredentialWithoutLocalKey branch so it reflects the
gateway-rewritten endpoint in both paths. Right now the reuse path leaves
gatewayEndpointUrl at resolvedEndpointUrl, which prevents inference set from
applying the same no-verify behavior for reused providers; update the remote
flow in remote.ts around gatewayEndpointUrl/providerResult handling and add a
reuse-path test in remote.test.ts to cover the resumed provider case.
---
Nitpick comments:
In `@src/lib/onboard/inference-providers/remote.test.ts`:
- Around line 53-92: Add a second test in setupRemoteProviderInference that
covers the reuseGatewayCredentialWithoutLocalKey branch for a loopback
compatible-endpoint. Use the existing makeDeps fixture and verify that when
reuseGatewayCredentialWithoutLocalKey is true, the code recomputes
gatewayEndpointUrl correctly before calling upsertProvider and runOpenshell.
Keep the current happy-path test, but extend coverage with this case so the
credential-reuse behavior is asserted through the same
setupRemoteProviderInference entry point.
In `@src/lib/onboard/inference-providers/remote.ts`:
- Around line 29-30: The URL normalization in the `remote.ts` host rewrite is
hard to read because `parsed.toString().replace(...)` hides the intent of
stripping only a non-root trailing slash. Simplify the logic in the
`parsed.hostname` / return path, or add a short comment near the URL-building
code to make it clear that root paths keep their slash while non-root paths have
the trailing slash removed before registering the gateway URL.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 1220f9a7-0668-44d1-ac0d-d868232051af
📒 Files selected for processing (3)
docs/inference/use-local-inference.mdxsrc/lib/onboard/inference-providers/remote.test.tssrc/lib/onboard/inference-providers/remote.ts
Signed-off-by: Chengjie Wang <chengjiew@nvidia.com>
chengjiew
changed the title
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/lib/onboard/inference-providers/remote.test.ts (1)
63-146: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winSolid coverage of the rewrite contract, but IPv6 loopback success path is untested.
Both success-path tests use
localhostand127.0.0.1; the helper'sisLoopbackcheck also treats::1as loopback (src/lib/onboard/inference-providers/remote.ts:11-39), but no test exercises the bracketed-IPv6 rewrite branch (bracket-stripping regex +::1match). Consider adding one more case to lock in that behavior, since it's a distinct code path in the helper.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/lib/onboard/inference-providers/remote.test.ts` around lines 63 - 146, Add a dedicated test in setupRemoteProviderInference to cover the IPv6 loopback branch, since the current cases only exercise localhost and 127.0.0.1. Use the existing remote.test.ts setup with makeDeps and assert that a bracketed ::1 endpoint is treated as loopback and rewritten to the sandbox/gateway host the same way as the other success paths. Reference setupRemoteProviderInference and the isLoopback/endpoint rewrite behavior so the bracket-stripping path stays covered even if the implementation changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/lib/onboard/inference-providers/remote.test.ts`:
- Around line 63-146: Add a dedicated test in setupRemoteProviderInference to
cover the IPv6 loopback branch, since the current cases only exercise localhost
and 127.0.0.1. Use the existing remote.test.ts setup with makeDeps and assert
that a bracketed ::1 endpoint is treated as loopback and rewritten to the
sandbox/gateway host the same way as the other success paths. Reference
setupRemoteProviderInference and the isLoopback/endpoint rewrite behavior so the
bracket-stripping path stays covered even if the implementation changes.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: a28aca25-17cd-494f-a35e-498922cda13c
📒 Files selected for processing (2)
src/lib/onboard/inference-providers/remote.test.tssrc/lib/onboard/inference-providers/remote.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- src/lib/onboard/inference-providers/remote.ts
Signed-off-by: Chengjie Wang <chengjiew@nvidia.com>
Signed-off-by: Chengjie Wang <chengjiew@nvidia.com>
This was referenced Jul 6, 2026
wscurran
added
area: inference
Contributor
|
✨ Thanks for the PR. This fixes the sandbox inference.local 503s for local vLLM-compatible endpoints by routing loopback addresses through the OpenShell host alias. Ready for maintainer review. Related open issues: Related open issues: |
This was referenced Jul 7, 2026
Signed-off-by: San Dang <sdang@nvidia.com>
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (2)
docs/inference/local-compatible-inference-setup.mdx (2)
351-351: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMinor wording nit: "one time" → "once".
Static analysis flags this as more idiomatic phrasing.
✏️ Suggested wording
-The prompt masks the key during input and retries one time on a bad key before failing. +The prompt masks the key during input and retries once on a bad key before failing.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/inference/local-compatible-inference-setup.mdx` at line 351, In the retry behavior sentence, replace the phrase “one time” with the more idiomatic “once,” preserving the rest of the wording.Source: Linters/SAST tools
58-63: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider mentioning
127.0.0.1/::1alongsidelocalhost.The rewrite behavior applies to all loopback forms (
localhost,127.0.0.1,::1) per the PR objectives, but the doc's examples only showlocalhost. Worth a brief mention so users entering127.0.0.1or::1know the same alias rewrite applies.Also applies to: 86-89
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/inference/local-compatible-inference-setup.mdx` around lines 58 - 63, Update the local inference setup documentation to explicitly mention that loopback endpoint handling applies to localhost, 127.0.0.1, and ::1. Extend the examples or surrounding explanation in the relevant onboarding guidance so users know all three forms receive the same host.openshell.internal alias rewrite.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@docs/inference/local-compatible-inference-setup.mdx`:
- Line 351: In the retry behavior sentence, replace the phrase “one time” with
the more idiomatic “once,” preserving the rest of the wording.
- Around line 58-63: Update the local inference setup documentation to
explicitly mention that loopback endpoint handling applies to localhost,
127.0.0.1, and ::1. Extend the examples or surrounding explanation in the
relevant onboarding guidance so users know all three forms receive the same
host.openshell.internal alias rewrite.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d4b10a0c-915f-4f8e-ad2b-49b757f7ea7a
📒 Files selected for processing (1)
docs/inference/local-compatible-inference-setup.mdx
Signed-off-by: San Dang <sdang@nvidia.com>
Signed-off-by: San Dang <sdang@nvidia.com>
Signed-off-by: San Dang <sdang@nvidia.com>
This reverts commit 3cbe47e. Signed-off-by: San Dang <sdang@nvidia.com>
Signed-off-by: San Dang <sdang@nvidia.com>
Contributor
E2E Target Results — ✅ All requested jobs passedRun: 29091003888
|
Collaborator
|
Advisor follow-up for #6331 (comment) I agree the in-scope risk is runtime proof for the bridge route: this PR verifies the user-entered local endpoint on the host, but registers the gateway provider URL through That is addressed with selective E2E on this head:
I am not expanding this PR with topology detection or a large direct URL-helper negative-test matrix. The code path remains narrow: only The broader items in the Nemotron comment are either covered by the E2E evidence above or are out of scope for this focused #5744 fix:
Keeping the checked-in regression simple: one integration-style command construction test plus the live E2E evidence. |
cv
requested changes
Jul 10, 2026
cv
left a comment
cv
left a comment
Collaborator
There was a problem hiding this comment.
Exact-head maintainer sweep at dc666e5759: ordinary CI and the five-target live run are green, but contributor compliance is a hard blocker. GitHub reports these commits as unsigned and Unverified: 030c98d6250e26954fbba11f0bc711b199305fcb, 14f49ff89c2aec4b8c8aaa9987dd7ee3eb835125, and 9506281950235a7a30628400ee7195f75e20c332.
Every commit retained in a contributor PR must appear as GitHub Verified. Maintainers cannot amend, sign, or replace contributor history on the contributor behalf. Please publish a clean, fully Verified history; if this already-published branch cannot be safely rewritten, open a fresh branch and PR with compliant commits. The replacement head will need fresh CI and trusted advisor evidence.
Also disposition the remaining trusted-advisor security warning about recommending an unauthenticated 0.0.0.0 inference bind by adding an adjacent exposure/mitigation warning or documenting an explicit accepted-risk rationale.
21 tasks
Collaborator
cv
added a commit
that referenced
this pull request
Jul 11, 2026
<!-- markdownlint-disable MD041 --> ## Summary Replaces PR #6331 with a clean, GitHub-verified branch for the local compatible-endpoint gateway fix. The change keeps host-side validation on the user-entered loopback URL, registers eligible sandbox routes through `host.openshell.internal`, and documents the exposure risk of `0.0.0.0` binds. ## Related Issue Fixes #5744. Replaces #6331. ## Changes - Rewrite only `compatible-endpoint` HTTP URLs with exact loopback hosts and explicit bundled policy ports (`11434`, `11435`, or `8000`) when registering the OpenShell provider route. - Keep host validation on the user-entered URL; on credential reuse, update an existing provider to the gateway alias without exporting or replacing its credential, then configure inference with `--no-verify`. - Extract the focused transformation and recovery logic from `remote.ts`, and document the exact supported URL shapes, bundled ports, bridge reachability, and `0.0.0.0` exposure mitigations. - Add unit, recovery-integration, docs, fast E2E-support, and hermetic live inference-routing coverage for the gateway registration contract. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates <!-- Check exactly one tests line and one docs line. Check other lines when applicable. Add every requested justification or approval reference. --> - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: This is the clean-history replacement for #6331 after the maintainer sweep at #6331 (review). The replacement preserves the reviewed gateway-routing change and adds the requested adjacent `0.0.0.0` exposure warning instead of accepting the risk silently. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification <!-- Check each applicable item only when supported by the requested evidence. Run targeted tests once per relevant change set and rerun after later edits or hook autofixes that can affect the tested behavior. Do not rerun hook-covered checks. --> - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: focused CLI, integration, docs, E2E-support, mock-parity, project-membership, title, size, and build checks passed; `npm run docs` reported 0 errors and 2 existing warnings; live run [29129492764](https://github.com/NVIDIA/NemoClaw/actions/runs/29129492764) passed `inference-routing`, `network-policy`, `onboard-repair`, `onboard-resume`, and `cloud-onboard`. - [x] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: `env -u SSH_CONNECTION -u SSH_CLIENT -u SSH_TTY npm test` passed 1,383 test files and 15,864 tests (39 skipped, 1 todo). - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- <!-- DCO sign-off is required in this PR description, and every commit must appear as Verified in GitHub. Run: git config user.name && git config user.email --> Signed-off-by: San Dang <sdang@nvidia.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Compatible inference endpoints using supported loopback addresses and ports can now be automatically routed through the sandbox gateway. - Rebuilds can restore compatible endpoint routing without requiring stored host credentials. - Endpoint paths, query strings, and fragments are preserved during automatic routing. - **Documentation** - Expanded guidance explains supported endpoints, routing behavior, validation limits, and runtime status checks. - Clarified credential handling during gateway recovery. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: San Dang <sdang@nvidia.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com> Signed-off-by: Charan Jagwani <cjagwani@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com> Co-authored-by: Charan Jagwani <cjagwani@nvidia.com>
Hadar301
pushed a commit
to Hadar301/NemoClaw-OpenShift
that referenced
this pull request
Jul 12, 2026
…DIA#6649) <!-- markdownlint-disable MD041 --> ## Summary Replaces PR NVIDIA#6331 with a clean, GitHub-verified branch for the local compatible-endpoint gateway fix. The change keeps host-side validation on the user-entered loopback URL, registers eligible sandbox routes through `host.openshell.internal`, and documents the exposure risk of `0.0.0.0` binds. ## Related Issue Fixes NVIDIA#5744. Replaces NVIDIA#6331. ## Changes - Rewrite only `compatible-endpoint` HTTP URLs with exact loopback hosts and explicit bundled policy ports (`11434`, `11435`, or `8000`) when registering the OpenShell provider route. - Keep host validation on the user-entered URL; on credential reuse, update an existing provider to the gateway alias without exporting or replacing its credential, then configure inference with `--no-verify`. - Extract the focused transformation and recovery logic from `remote.ts`, and document the exact supported URL shapes, bundled ports, bridge reachability, and `0.0.0.0` exposure mitigations. - Add unit, recovery-integration, docs, fast E2E-support, and hermetic live inference-routing coverage for the gateway registration contract. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [x] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates <!-- Check exactly one tests line and one docs line. Check other lines when applicable. Add every requested justification or approval reference. --> - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: This is the clean-history replacement for NVIDIA#6331 after the maintainer sweep at NVIDIA#6331 (review). The replacement preserves the reviewed gateway-routing change and adds the requested adjacent `0.0.0.0` exposure warning instead of accepting the risk silently. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification <!-- Check each applicable item only when supported by the requested evidence. Run targeted tests once per relevant change set and rerun after later edits or hook autofixes that can affect the tested behavior. Do not rerun hook-covered checks. --> - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: focused CLI, integration, docs, E2E-support, mock-parity, project-membership, title, size, and build checks passed; `npm run docs` reported 0 errors and 2 existing warnings; live run [29129492764](https://github.com/NVIDIA/NemoClaw/actions/runs/29129492764) passed `inference-routing`, `network-policy`, `onboard-repair`, `onboard-resume`, and `cloud-onboard`. - [x] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: `env -u SSH_CONNECTION -u SSH_CLIENT -u SSH_TTY npm test` passed 1,383 test files and 15,864 tests (39 skipped, 1 todo). - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- <!-- DCO sign-off is required in this PR description, and every commit must appear as Verified in GitHub. Run: git config user.name && git config user.email --> Signed-off-by: San Dang <sdang@nvidia.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Compatible inference endpoints using supported loopback addresses and ports can now be automatically routed through the sandbox gateway. - Rebuilds can restore compatible endpoint routing without requiring stored host credentials. - Endpoint paths, query strings, and fragments are preserved during automatic routing. - **Documentation** - Expanded guidance explains supported endpoints, routing behavior, validation limits, and runtime status checks. - Clarified credential handling during gateway recovery. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: San Dang <sdang@nvidia.com> Signed-off-by: Carlos Villela <cvillela@nvidia.com> Signed-off-by: Charan Jagwani <cjagwani@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com> Co-authored-by: Charan Jagwani <cjagwani@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Routes loopback OpenAI-compatible endpoints through the OpenShell host alias when registering gateway inference, while keeping host-side endpoint validation against the user-provided
localhostURL. This fixes sandboxinference.local503s for local vLLM-style endpoints.Related Issue
Fixes #5744
Changes
http://localhost:<port>/...,127.0.0.1, and::1compatible endpoint URLs tohost.openshell.internal:<port>for gateway registration.Type of Change
Quality Gates
aits-log-worker-6.Verification
Verifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailablenpm_config_cache=/Users/wangchengjie/Project/NemoClaw-issue-5744/.npm-cache npx vitest run src/lib/onboard/inference-providers/remote.test.ts src/lib/onboard/inference-selection-validation.test.ts src/lib/onboard/local-inference-route.test.ts src/lib/inference/local.test.ts→ 4 files, 71 tests passednpm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result:npm run docsbuilds without warnings (doc changes only)Additional verification:
npm run typecheck:clipassednpm run format:checkpassednpm run docspassed with 0 errors and 2 existing Fern warningsgit diff --checkpassedaits@aits-log-worker-6:http://localhost:8000/v1/modelsreturning 200 produced sandboxhttps://inference.local/v1/models→HTTP/1.1 503 Service Unavailablehttps://inference.local/v1/models→HTTP/1.1 200 OKSigned-off-by: Chengjie Wang chengjiew@nvidia.com
Summary by CodeRabbit
--host 0.0.0.0).