fix(installer): refresh packaged Station CDI#7158
Merged
Merged
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe Station host-preparation flow now repairs missing CDI exposure for AI Developer Tools through packaged refresh units, preserves forced-runtime failure behavior, adds focused validation tests, and updates platform, preparation, and troubleshooting documentation. ChangesDGX Station CDI validation
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant RuntimeValidation
participant NVIDIACtk
participant CDIRefreshService
participant AcceptanceProbes
RuntimeValidation->>NVIDIACtk: Check nvidia.com/gpu=all
NVIDIACtk-->>RuntimeValidation: CDI device present or missing
RuntimeValidation->>CDIRefreshService: Repair packaged CDI refresh lifecycle
CDIRefreshService-->>RuntimeValidation: Advertise refreshed CDI device
RuntimeValidation->>AcceptanceProbes: Run CDI and GPU validation
Possibly related issues
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ci/platform-matrix.json`:
- Line 69: Add the repository-standard SPDX license identifier to the top-level
$comment metadata in platform-matrix.json, preserving the existing JSON
structure and content. Ensure the file remains valid JSON and the identifier
appears before the current platform notes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: bf1e3ccf-a20c-4475-a6e2-dd411552c26c
📒 Files selected for processing (6)
ci/platform-matrix.jsondocs/get-started/dgx-station-preparation.mdxdocs/reference/platform-support.mdxdocs/reference/troubleshooting.mdxscripts/prepare-dgx-station-host.shtest/install-station-dgx-os.test.ts
Contributor
PR Review Advisor — InformationalAdvisor assessment: Informational / medium confidence Model lanes
Nemotron output stays in workflow artifacts and does not change the assessment above. E2E guidanceAdvisory only. E2E / PR Gate selects and runs jobs independently. Recommended E2E: 1 optional E2E recommendation
This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
21 tasks
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/install-station-dgx-os.test.ts`:
- Around line 920-970: Strengthen the parameterized test around
verify_dgx_os_runtime_sudo so the “missing device after restart” scenario proves
the packaged CDI repair sequence runs: have the sudo stub emit distinct markers
for enable, start, and restart, record their order, and assert each marker and
the expected ordering. Keep the existing failure and diagnostic assertions,
ensuring the test cannot pass solely because the CDI advertisement check fails
while restart was skipped.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: b296d629-afb2-4c82-905b-13954255da09
📒 Files selected for processing (3)
docs/get-started/dgx-station-preparation.mdxscripts/prepare-dgx-station-host.shtest/install-station-dgx-os.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- scripts/prepare-dgx-station-host.sh
- docs/get-started/dgx-station-preparation.mdx
Comment on lines
+920
to
+970
| it.each([ | ||
| ["restart failure", "return 1", "Packaged CDI refresh failed"], | ||
| ["missing device after restart", "return 0", "did not advertise nvidia.com/gpu=all"], | ||
| ] as const)("fails closed on AI Developer Tools CDI %s", (_scenario, restartResult, error) => { | ||
| const { result, output } = runSourced( | ||
| STATION_PREPARE, | ||
| ` | ||
| require_command() { :; } | ||
| check_dgx_os_runtime_commands() { printf 'FACTORY_GATES_OK\n'; } | ||
| systemctl() { | ||
| case "$*" in | ||
| 'is-active --quiet containerd.service'|'is-active --quiet docker.service') return 0 ;; | ||
| *) printf 'UNEXPECTED_SYSTEMCTL %s\n' "$*"; return 1 ;; | ||
| esac | ||
| } | ||
| station_sudo_local_default_docker() { | ||
| case "$*" in | ||
| 'info'|'buildx version') return 0 ;; | ||
| *) printf 'UNEXPECTED_DOCKER %s\n' "$*"; return 1 ;; | ||
| esac | ||
| } | ||
| require_docker_mutation_quiescence() { printf 'WORKLOAD_GATE_OK %s\n' "$1"; } | ||
| sudo() { | ||
| case "$*" in | ||
| 'nvidia-ctk cdi list') return 0 ;; | ||
| 'systemctl enable nvidia-cdi-refresh.path nvidia-cdi-refresh.service'|'systemctl start nvidia-cdi-refresh.path') return 0 ;; | ||
| 'systemctl restart nvidia-cdi-refresh.service') ${restartResult} ;; | ||
| 'systemctl status nvidia-cdi-refresh.service --no-pager'|'journalctl -u nvidia-cdi-refresh.service --no-pager -n 50') | ||
| printf 'SERVICE_DIAGNOSTICS %s\n' "$*" | ||
| ;; | ||
| *) printf 'UNEXPECTED_SUDO %s\n' "$*"; return 1 ;; | ||
| esac | ||
| } | ||
| nvidia-ctk() { return 0; } | ||
| ensure_dgx_os_acceptance_image() { printf 'UNEXPECTED_ACCEPTANCE_IMAGE\n'; return 1; } | ||
| run_dgx_os_cdi_test_sudo() { printf 'UNEXPECTED_CDI_PROBE\n'; return 1; } | ||
| run_dgx_os_gpus_test_sudo() { printf 'UNEXPECTED_GPUS_PROBE\n'; return 1; } | ||
| STATION_HOST_PROFILE=ai-developer-tools | ||
| verify_dgx_os_runtime_sudo | ||
| `, | ||
| ); | ||
|
|
||
| expect(result.status, output).not.toBe(0); | ||
| expect(output).toContain("FACTORY_GATES_OK"); | ||
| expect(output).toContain("WORKLOAD_GATE_OK"); | ||
| expect(output).toContain("SERVICE_DIAGNOSTICS"); | ||
| expect(output).toContain(error); | ||
| expect(output).not.toContain("UNEXPECTED_"); | ||
| expect(output).not.toContain("systemctl restart docker.service"); | ||
| expect(output).not.toContain("systemctl restart containerd.service"); | ||
| }); |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Prove the missing-device case actually performs the restart.
The "missing device after restart" row can pass if restart is skipped: the empty CDI-list stub still reaches the final advertisement error. Emit/assert markers for enable, start, and restart (and their order) so this test protects the packaged repair contract.
As per path instructions, flag “conditionals that make a test pass without exercising its claim.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@test/install-station-dgx-os.test.ts` around lines 920 - 970, Strengthen the
parameterized test around verify_dgx_os_runtime_sudo so the “missing device
after restart” scenario proves the packaged CDI repair sequence runs: have the
sudo stub emit distinct markers for enable, start, and restart, record their
order, and assert each marker and the expected ordering. Keep the existing
failure and diagnostic assertions, ensuring the test cannot pass solely because
the CDI advertisement check fails while restart was skipped.
Source: Path instructions
cv
approved these changes
Jul 18, 2026
cv
left a comment
cv
left a comment
Collaborator
There was a problem hiding this comment.
Approved. Security review: PASS. The accepted #7154 Station profile is narrowly enforced: workload quiescence precedes repair, only the packaged CDI service is restarted, and restart failure or missing post-refresh device fails closed before probes. The implementation does not install packages, generate CDI directly, or mutate Docker/containerd configuration. The full 67-test Station suite, failure-path coverage, docs build/audit, both advisor lanes, conventional CI, and the protected fork E2E decision pass.
senthilr-nv
added
bug-fix
21 tasks
ericksoa
added a commit
that referenced
this pull request
Jul 18, 2026
<!-- markdownlint-disable MD041 --> ## Summary Adds the canonical `docs/changelog/2026-07-18.mdx` release-prep entry with the exact `## v0.0.88` heading. The entry summarizes every user-visible change on `main` since v0.0.87 and links each release theme to the focused user documentation. ## Changes - Add one parser-safe dated changelog entry for v0.0.88 covering DGX Station preparation, inference health, multi-gateway sandbox operations and recovery, onboarding policy defaults, and rebuild credential reuse. - Reconcile the changelog against the merged v0.0.88-labeled PRs and the complete `v0.0.87..origin/main` commit range. - Source mapping: - [#7152](#7152) -> `docs/changelog/2026-07-18.mdx`: Document RDMA-aware OpenIB service remediation during DGX Station preparation. - [#7155](#7155) -> `docs/changelog/2026-07-18.mdx`: Document stopped-container preservation and fail-closed restart-policy boundaries. - [#7158](#7158) -> `docs/changelog/2026-07-18.mdx`: Document bounded packaged CDI refresh for the exact AI Developer Tools Station profile. - [#7074](#7074) -> `docs/changelog/2026-07-18.mdx`: Document authenticated upstream model probes and precise route-reachability claims. - [#7007](#7007) -> `docs/changelog/2026-07-18.mdx`: Document the explicit serving-process health gap in `status` and `doctor`. - [#7113](#7113) -> `docs/changelog/2026-07-18.mdx`: Document owning-gateway selection for sandbox-scoped status and exec operations. - [#7092](#7092) -> `docs/changelog/2026-07-18.mdx`: Document idempotent recovery for target-owned active port forwards. - [#7133](#7133) -> `docs/changelog/2026-07-18.mdx`: Document web-search-aware policy preset defaults during onboarding. - [#7129](#7129) -> `docs/changelog/2026-07-18.mdx`: Document gateway-registered web-search credential reuse during rebuild preflight. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates <!-- Check one tests line and one docs line. Check other lines when applicable. Add every requested justification or approval reference. --> - [ ] Tests added or updated for changed behavior - [x] Existing tests cover changed behavior — justification: `test/changelog-docs.test.ts` validates the dated changelog contract, exact release heading, and parser-safe MDX structure. - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification <!-- Check each applicable item only when supported by the requested evidence. Run targeted tests once per relevant change set and rerun after later edits or hook autofixes that can affect the tested behavior. Do not rerun hook-covered checks. --> - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: `npx vitest run test/changelog-docs.test.ts` passed 6 tests. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — completed successfully with 0 errors and 2 existing Fern warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) — not applicable because native changelog entries use the required parser-safe MDX SPDX comment without frontmatter. --- Signed-off-by: Aaron Erickson <aerickson@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added improved DGX Station preparation workflows. * Enhanced sandbox status and diagnostic reporting for inference health. * Improved state selection and recovery across multiple gateways. * Added safer onboarding defaults for web search policies. * Improved rebuild preflight handling for credential reuse and fail-closed behavior. * **Documentation** * Added release notes for version 0.0.88. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The exact AI Developer Tools Station profile previously stopped when its packaged CDI refresh units were installed but disabled.
This change conditionally runs that packaged lifecycle after the existing driver, GB300, Docker, Buildx, service, and workload gates, while preserving validation-only behavior when
nvidia.com/gpu=allis already present.Related Issue
Tracks #7154.
The product decision and bounded mutation scope were accepted in issue comment 5012459490.
Physical Station Express acceptance remains required before issue resolution.
Changes
nvidia-cdi-refresh.pathandnvidia-cdi-refresh.servicelifecycle only for the exactai-developer-toolsprofile when CDI inventory is missing.Type of Change
Quality Gates
Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailablenpx vitest run --project installer-integration test/install-station-dgx-os.test.ts test/install-station-host-preparation.test.ts(120 passed);npm run docscompleted with 0 errors and 2 existing Fern warnings; platform-doc synchronization, JSON parsing, Biome, and test-title checks passed.npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result:npm run docsbuilds without warnings (doc changes only)Signed-off-by: Senthil Ravichandran senthilr@nvidia.com
Summary by CodeRabbit
nvidia.com/gpu=allis missing.