test(e2e): prove managed exec egress denial by apurvvkumaria · Pull Request #6513 · NVIDIA/NemoClaw

apurvvkumaria · 2026-07-08T23:26:46Z

Summary

Add exact live-policy evidence that directly invoking DCode's public managed-exec helper with managed Python cannot bypass the default Tavily denial. This closes the remaining security-review evidence gap from #6506 without changing runtime behavior.

Related Issue

Follow-up to #6506 and completed issue #6504.

Changes

Extend the existing DCode Python-egress live check to run the managed helper directly with /opt/venv/bin/python3 and require confirmed Tavily policy-denial evidence before opt-in.
Preserve command-shape coverage for both ordinary Python and helper-prefixed Python, with static image-contract assertions for the new boundary proof.
Rename the internal DCode launcher constant to match the dcode-managed-exec entrypoint it identifies.

Type of Change

Code change (feature, bug fix, or refactor)
Code change with doc updates
Doc only (prose changes, no code sample modifications)
Doc only (includes code sample changes)

Quality Gates

Tests added or updated for changed behavior
Existing tests cover changed behavior — justification:
Tests not applicable — justification:
Docs updated for user-facing behavior changes
Docs not applicable — justification: this adds regression evidence for an existing network-policy contract and makes an internal constant rename; no user-facing command, configuration, output, schema, or runtime behavior changes.
Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent reviews confirmed that direct live-policy evidence is safer and less brittle than hard-coding the generated probe argv; command construction uses quoted argv assembly, and the existing positive connect --probe-only live check remains the allowed-route counterpart.
Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: 97 focused CLI, integration/image-contract, and E2E-support tests passed; Bash syntax, CLI build/typecheck, repository ratchets, and scoped prek also passed.
Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: not applicable; this is one additional assertion in an existing typed live check plus static/command-shape coverage.
Quality Gates section completed with required justifications or waivers
No secrets, API keys, or credentials committed
npm run docs builds without warnings (doc changes only)
Doc pages follow the style guide (doc changes only)
New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Summary by CodeRabbit

Bug Fixes
- Improved Python egress handling so direct execution through the managed wrapper is correctly blocked when policy requires it.
- Updated probe execution so command formatting stays single-line and consistent across supported launch paths.
Tests
- Expanded end-to-end coverage for Python reachability checks, including managed-exec invocation.
- Updated policy-behavior checks to reflect the latest command execution flow.

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>

coderabbitai · 2026-07-08T23:27:00Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 23514348-855f-4f58-99a1-813ccbf6fb54

📥 Commits

Reviewing files that changed from the base of the PR and between 6f45d84 and 814db95.

📒 Files selected for processing (4)

src/lib/actions/sandbox/connect-inference-route-probe.ts
test/e2e/e2e-cloud-experimental/checks/06-deepagents-code-python-egress.sh
test/e2e/support/platform-parity-cloud-experimental.test.ts
test/langchain-deepagents-code-image.test.ts

📝 Walkthrough

Walkthrough

The sandbox inference route probe now injects a DCODE_MANAGED_EXEC_LAUNCHER constant instead of DCODE_MANAGED_RUNTIME_LAUNCHER. Corresponding e2e shell checks and TypeScript tests are updated to verify direct managed-exec Python invocations remain blocked and produce single-line commands.

Changes

Managed-exec launcher switch and verification

Layer / File(s)	Summary
Sandbox probe launcher constant update `src/lib/actions/sandbox/connect-inference-route-probe.ts`	Replaces the `DCODE_MANAGED_RUNTIME_LAUNCHER` constant with `DCODE_MANAGED_EXEC_LAUNCHER` in the probe command array construction.
E2E egress check for direct managed-exec Python `test/e2e/e2e-cloud-experimental/checks/06-deepagents-code-python-egress.sh`	Adds a `DCODE_MANAGED_EXEC` path constant, extends `python_probe`/`expect_blocked` to accept a command-prefix, adjusts probe-command-shape test-mode output, and adds a Tavily denial check for direct managed-exec Python invocation.
Test expectations for managed-exec single-line commands `test/e2e/support/platform-parity-cloud-experimental.test.ts`, `test/langchain-deepagents-code-image.test.ts`	Updates the OpenShell single-line command test to assert two `SINGLE_LINE_COMMAND:` outputs, and updates `pythonEgressCheck` expectations to reference the managed-exec Python command segment.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Test as e2e-egress-check
  participant Probe as python_probe
  participant Sandbox as sandbox_exec
  participant Tavily as Tavily endpoint

  Test->>Probe: python_probe(url, DCODE_MANAGED_EXEC, /opt/venv/bin/python3)
  Probe->>Sandbox: remote_cmd (managed-exec prefix + python3 -c payload)
  Sandbox->>Tavily: attempt egress
  Tavily-->>Sandbox: BLOCKED
  Sandbox-->>Probe: BLOCKED
  Probe-->>Test: expect_blocked assertion

Suggested labels: area: e2e

Suggested reviewers: ericksoa, brandonpelfrey

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: an e2e test proving managed-exec egress denial.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch codex/6506-managed-exec-egress-proof

_{Comment @coderabbitai help to get the list of available commands.}

github-code-quality · 2026-07-08T23:27:24Z

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage remains at 96%, unchanged from the main branch.

TypeScript / code-coverage/cli

The overall coverage in the codex/6506-managed-e... branch remains at 76%, unchanged from the main branch.

Show a code coverage summary of the most impacted files.

File	main `6f45d84`	codex/6506-managed-e... `814db95`	+/-
`src/lib/actions...e-validation.ts`	90%	81%	-9%
`src/lib/actions...x/mcp-bridge.ts`	44%	35%	-9%
`src/lib/runner.ts`	80%	73%	-7%
`src/lib/actions...lution-probe.ts`	94%	88%	-6%
`src/lib/actions...ridge-policy.ts`	64%	62%	-2%
`src/lib/security/redact.ts`	97%	95%	-2%
`src/lib/shields/index.ts`	60%	60%	0%
`src/lib/policy/index.ts`	65%	66%	+1%
`src/lib/actions...r-inspection.ts`	77%	78%	+1%
`src/lib/actions...-add-restart.ts`	14%	19%	+5%

_{Updated July 08, 2026 23:35 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.}

github-actions · 2026-07-08T23:28:41Z

PR Review Advisor (Nemotron Ultra) — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: No advisor follow-up required beyond maintainer review.
Open items: 0 required · 0 warnings · 0 suggestions · 0 test follow-ups
Since last review: 0 prior items resolved · 0 still apply · 0 new items found

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

github-actions · 2026-07-08T23:29:16Z

PR Review Advisor — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: Add or justify PRA-T1 and any related test follow-ups.
Open items: 0 required · 0 warnings · 0 suggestions · 1 test follow-up

Action checklist

PRA-T1 Add or justify test follow-up: Runtime validation

Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

PRA-T1 Runtime validation — Direct managed-exec Python can reach an approved host such as https://pypi.org/ or https://api.github.com/ while still denying https://api.tavily.com/, to prove the helper preserves allowed policy rather than merely breaking all egress.. The changed behavior is a sandbox/network-policy contract, so meaningful validation is runtime/integration validation. The PR adds that validation in the existing cloud-experimental e2e shell check and adds static command-shape coverage, but this read-only review did not execute the live sandbox path.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

apurvvkumaria · 2026-07-08T23:29:28Z

@coderabbitai review

coderabbitai · 2026-07-08T23:29:37Z

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

github-actions · 2026-07-08T23:31:19Z

E2E Advisor Recommendation

Required E2E: ubuntu-repo-cloud-langchain-deepagents-code, inference-routing
Optional E2E: issue-4434-tui-unreachable-inference, cloud-onboard

Dispatch hint: targets=ubuntu-repo-cloud-langchain-deepagents-code,inference-routing

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

ubuntu-repo-cloud-langchain-deepagents-code (high): Required because the PR changes a required Deep Agents Code live egress/security check and the DCode managed-exec boundary. This typed E2E target onboards a real langchain-deepagents-code sandbox and runs the cloud-experimental check list including 06-deepagents-code-python-egress.sh.
inference-routing (medium): Required because the runtime source change is in the inference.local route probe path used by connect/status/doctor. The inference-routing E2E is the existing free-standing live coverage for inference routing classification and cleanup boundaries.

Optional E2E

issue-4434-tui-unreachable-inference (very high): Optional adjacent confidence for the connect/status inference.local probe path under an unreachable-route scenario. It exercises connect --probe-only and status diagnostics, but it is OpenClaw-focused rather than the Deep Agents Code managed-exec path.
cloud-onboard (high): Optional packaging confidence that the public installer path carries the updated source and cloud-experimental check assets into a hosted onboarding run. It is less targeted because the default cloud-onboard sandbox is not a Deep Agents Code sandbox.

New E2E recommendations

deepagents-code-connect-route-probe (high): Existing live coverage onboards Deep Agents Code and runs egress/headless-inference checks, but there does not appear to be a dedicated live E2E that runs nemoclaw <dcode-sandbox> connect --probe-only and status against a Deep Agents Code sandbox to prove the CLI uses /usr/local/lib/nemoclaw/dcode-managed-exec for the inference.local route probe while ignoring attacker-writable shell startup output.
- Suggested test: Add a Deep Agents Code live connect/status route-probe check, either as a new cloud-experimental check or as an assertion in the ubuntu-repo-cloud-langchain-deepagents-code target, that creates hostile profile output and verifies connect/status classify inference.local only from the managed-exec probe.

Dispatch hint

Workflow: E2E
jobs input: targets=ubuntu-repo-cloud-langchain-deepagents-code,inference-routing

github-actions · 2026-07-08T23:31:20Z

E2E Target Recommendation

Required E2E targets: ubuntu-repo-cloud-langchain-deepagents-code
Optional E2E targets: None

Dispatch required E2E targets:

gh workflow run e2e.yaml --ref <pr-head-ref> --field targets=ubuntu-repo-cloud-langchain-deepagents-code

Workflow run

Full E2E target advisor summary

E2E Target Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E targets

ubuntu-repo-cloud-langchain-deepagents-code: The PR changes the Deep Agents Code cloud-experimental Python egress live check and its E2E support guardrail, and adjusts the Deep Agents Code managed-exec inference route probe path. The live-supported typed target ubuntu-repo-cloud-langchain-deepagents-code is the smallest target that onboards a Deep Agents Code sandbox and runs the cloud-experimental Deep Agents Code policy checks including 06-deepagents-code-python-egress.sh.
- Dispatch: gh workflow run e2e.yaml --ref <pr-head-ref> --field targets=ubuntu-repo-cloud-langchain-deepagents-code

Optional E2E targets

None.

Relevant changed files

src/lib/actions/sandbox/connect-inference-route-probe.ts
test/e2e/e2e-cloud-experimental/checks/06-deepagents-code-python-egress.sh
test/e2e/support/platform-parity-cloud-experimental.test.ts

apurvvkumaria · 2026-07-08T23:32:37Z

@coderabbitai review

coderabbitai · 2026-07-08T23:32:42Z

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

github-actions · 2026-07-08T23:33:54Z

E2E Target Results — ✅ All requested jobs passed

Run: 28982989053
Workflow ref: codex/6506-managed-exec-egress-proof
Requested targets: (default — all supported)
Requested jobs: inference-routing
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped

Job	Result
inference-routing	✅ success

## Summary  Add exact live-policy evidence that directly invoking DCode's public managed-exec helper with managed Python cannot bypass the default Tavily denial. This closes the remaining security-review evidence gap from NVIDIA#6506 without changing runtime behavior. ## Related Issue  Follow-up to NVIDIA#6506 and completed issue NVIDIA#6504. ## Changes  - Extend the existing DCode Python-egress live check to run the managed helper directly with `/opt/venv/bin/python3` and require confirmed Tavily policy-denial evidence before opt-in. - Preserve command-shape coverage for both ordinary Python and helper-prefixed Python, with static image-contract assertions for the new boundary proof. - Rename the internal DCode launcher constant to match the `dcode-managed-exec` entrypoint it identifies. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates  - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: this adds regression evidence for an existing network-policy contract and makes an internal constant rename; no user-facing command, configuration, output, schema, or runtime behavior changes. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: independent reviews confirmed that direct live-policy evidence is safer and less brittle than hard-coding the generated probe argv; command construction uses quoted argv assembly, and the existing positive `connect --probe-only` live check remains the allowed-route counterpart. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification  - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: 97 focused CLI, integration/image-contract, and E2E-support tests passed; Bash syntax, CLI build/typecheck, repository ratchets, and scoped prek also passed. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: not applicable; this is one additional assertion in an existing typed live check plus static/command-shape coverage. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) ---  Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>  ## Summary by CodeRabbit * **Bug Fixes** * Improved Python egress handling so direct execution through the managed wrapper is correctly blocked when policy requires it. * Updated probe execution so command formatting stays single-line and consistent across supported launch paths. * **Tests** * Expanded end-to-end coverage for Python reachability checks, including managed-exec invocation. * Updated policy-behavior checks to reflect the latest command execution flow.  Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>

test(e2e): prove managed exec egress denial

814db95

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>

apurvvkumaria self-assigned this Jul 8, 2026

apurvvkumaria requested a review from cv July 8, 2026 23:34

cv approved these changes Jul 8, 2026

View reviewed changes

cv merged commit abc9aa5 into main Jul 8, 2026
202 checks passed

cv deleted the codex/6506-managed-exec-egress-proof branch July 8, 2026 23:37

Uh oh!

Conversation

apurvvkumaria commented Jul 8, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issue

Changes

Type of Change

Quality Gates

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

github-code-quality Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Overview

TypeScript / code-coverage/plugin

TypeScript / code-coverage/cli

Uh oh!

github-actions Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor (Nemotron Ultra) — No blocking findings

Uh oh!

github-actions Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review Advisor — No blocking findings

Action checklist

Uh oh!

apurvvkumaria commented Jul 8, 2026

Uh oh!

coderabbitai Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 8, 2026

E2E Advisor Recommendation

E2E Recommendation Advisor

Required E2E

Optional E2E

New E2E recommendations

Dispatch hint

Uh oh!

github-actions Bot commented Jul 8, 2026

E2E Target Recommendation

E2E Target Advisor

Required E2E targets

Optional E2E targets

Relevant changed files

Uh oh!

apurvvkumaria commented Jul 8, 2026

Uh oh!

coderabbitai Bot commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 8, 2026

E2E Target Results — ✅ All requested jobs passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

apurvvkumaria commented Jul 8, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 8, 2026 •

edited

Loading

github-code-quality Bot commented Jul 8, 2026 •

edited

Loading

github-actions Bot commented Jul 8, 2026 •

edited

Loading

github-actions Bot commented Jul 8, 2026 •

edited

Loading

coderabbitai Bot commented Jul 8, 2026 •

edited

Loading

coderabbitai Bot commented Jul 8, 2026 •

edited

Loading