bug: vm driver denies allowed network egress due to /init.krun ancestor integrity failure

[IamMrandrew]

bug: vm driver denies allowed network egress due to /init.krun ancestor integrity failure

Agent Diagnostic

• I used an agent to investigate the OpenShell policy behavior and compare VM-driver vs Docker-driver behavior.
• I reviewed the relevant OpenShell docs:
  • https://docs.nvidia.com/openshell/reference/sandbox-compute-drivers
  • https://docs.nvidia.com/openshell/reference/policy-schema
• I reproduced the failure on the OpenShell VM driver.
• I ran a Docker-driver A/B test with the same target endpoint and same network policy shape. Docker succeeds; VM fails.
• I found related prior art in OpenShell PR fix(sandbox): strip " (deleted)" suffix from unlinked /proc/<pid>/exe paths #844, which appears to be a similar class of ancestor/integrity attribution issue:
  • fix(sandbox): strip " (deleted)" suffix from unlinked /proc/<pid>/exe paths #844

Description

When a sandbox runs with the OpenShell VM driver, an explicitly allowed outbound request is denied because the process ancestry integrity check tries to stat /init.krun.

The same endpoint and policy work from a Docker-driver sandbox on the same host.

Expected:

• A VM-driver sandbox should allow policy-authorized network access when the endpoint and binary match the policy.
• VM-driver-specific init/process ancestry such as /init.krun should not cause an otherwise valid network request to be denied, or the required policy shape should be documented.

Actual:

• The VM-driver sandbox denies the allowed request.
• Logs show:

DENIED /usr/local/bin/node -> search.internal.example:80
reason: ancestor integrity check failed for /init.krun: Failed to stat /init.krun

Impact:

• VM-backed OpenShell sandboxes cannot use an explicitly policy-authorized self-hosted search service.
• This blocks a PoC where VM-backed isolation is required, while Docker-driver isolation works with the same service and policy.

Reproduction Steps

Environment setup:

1. Run OpenShell 0.0.44 on Ubuntu on a KVM/nested-virtualization-capable x86_64 cloud VM. (EC2 instance type: c7i-flex.xlarge)
2. Configure one gateway with the VM driver.
3. Create a VM-backed sandbox.
4. Run a SearXNG service reachable from the host and from Docker sandboxes at an internal/private endpoint, for example:

http://search.internal.example

5. Apply this policy shape:

version: 1
filesystem_policy:
  include_workdir: true
  read_only:
    - /usr
    - /lib
    - /proc
    - /dev/urandom
    - /etc
    - /var/log
  read_write:
    - /tmp
    - /dev/null
    - /sandbox
landlock:
  compatibility: best_effort
process:
  run_as_user: sandbox
  run_as_group: sandbox
network_policies:
  searxng_search:
    name: searxng_search
    endpoints:
      - host: search.internal.example
        port: 80
        enforcement: enforce
        allowed_ips:
          - 10.0.0.25/32
    binaries:
      - path: /usr/local/bin/node

6. Run:

openshell sandbox exec -n <vm-sandbox> --no-tty -- node -e \
'(async()=>{const r=await fetch("http://search.internal.example/search?q=openclaw&format=json"); console.log(r.status, await r.text());})()'

Observed:

TypeError: fetch failed
cause: Request was cancelled.

Relevant VM-driver sandbox log:

NET:OPEN DENIED /usr/local/bin/node -> search.internal.example:80
[policy:- engine:opa]
[reason:ancestor integrity check failed for /init.krun: Failed to stat /init.krun]

Docker A/B Result

On the same host, I started a separate Docker-driver gateway on another port/state directory and created a Docker-backed sandbox using ghcr.io/nvidia/nemoclaw/sandbox-base:v0.0.55.

Positive test:

openshell --gateway-endpoint http://127.0.0.1:18080 \
  sandbox exec -n docker-searxng-ab --no-tty -- \
  node -e '(async()=>{const url="http://search.internal.example/search?q=openclaw&format=json"; const r=await fetch(url); const j=await r.json(); console.log(JSON.stringify({ok:r.ok,status:r.status,resultCount:(j.results||[]).length,first:(j.results||[])[0]?.title},null,2));})()'

Result:

{
  "ok": true,
  "status": 200,
  "resultCount": 13,
  "first": "OpenClaw — Personal AI Assistant"
}

Docker-driver policy log:

NET:OPEN ALLOWED /usr/local/bin/node -> search.internal.example:80 [policy:searxng_search engine:opa]
HTTP:GET ALLOWED GET http://search.internal.example/search

Negative control:

node -e '(async()=>{try{await fetch("https://example.com/"); console.log("unexpected");}catch(e){console.log("blocked", e.message)}})()'

Result:

{"blocked":true,"error":"Request was cancelled."}

Docker-driver policy log:

NET:OPEN DENIED /usr/local/bin/node -> example.com:443 [policy:- engine:opa] [reason:endpoint example.com:443 not in policy 'searxng_search']

Environment

• OS: Ubuntu 24.04 on a KVM/nested-virtualization-capable x86_64 cloud VM (EC2 instance type: c7i-flex.xlarge)
• OpenShell: 0.0.44
• VM driver: enabled explicitly with OPENSHELL_DRIVERS=vm / compute_drivers = ["vm"]
• Docker A/B driver: separate gateway with OPENSHELL_DRIVERS=docker
• Search service: SearXNG on a separate internal/private host, reachable from the host and Docker sandbox
• Target endpoint:

search.internal.example:80
allowed_ips: 10.0.0.25/32

Logs

VM failure:

NET:OPEN DENIED /usr/local/bin/node -> search.internal.example:80
[policy:- engine:opa]
[reason:ancestor integrity check failed for /init.krun: Failed to stat /init.krun]

Docker success:

NET:OPEN ALLOWED /usr/local/bin/node -> search.internal.example:80 [policy:searxng_search engine:opa]
HTTP:GET ALLOWED GET http://search.internal.example/search

Docker negative control:

NET:OPEN DENIED /usr/local/bin/node -> example.com:443 [policy:- engine:opa] [reason:endpoint example.com:443 not in policy 'searxng_search']

Agent-First Checklist

• I pointed my agent at the repo/docs and had it investigate this issue
• I loaded/reviewed relevant OpenShell docs
• My agent could not resolve this locally; the diagnostic above explains why

[JonasCir]

I encountered the exact same issue when following the official "Write Your First Sandbox Network Policy" tutorial.

[EislM0203]

Encountering the same issue

[rhuss]

📋 triage-agent

Classification: bug-confirmed

Summary: The VM driver's ancestor integrity check in the CONNECT proxy fails because /init.krun (the libkrun init process) does not exist as a stattable file on the filesystem, causing all policy-authorized network egress to be denied.

Investigation:

• Found the exact failure path at crates/openshell-supervisor-network/src/proxy.rs:1445-1457: the code walks up the process tree via collect_ancestor_identities() and calls verify_or_cache_process_exe() on each ancestor, which tries to stat() and hash the binary. For VM-driver sandboxes, the process tree root is /init.krun, which is a libkrun-internal init that doesn't exist on the guest filesystem.
• PR fix(sandbox): strip " (deleted)" suffix from unlinked /proc/<pid>/exe paths #844 (merged as commit ae7e901) fixed a related but different issue: stripping the " (deleted)" suffix from unlinked /proc/<pid>/exe paths. The /init.krun problem is a distinct issue where the binary path itself never exists on the filesystem.
• The reporter's Docker vs. VM A/B test conclusively demonstrates the issue is VM-driver-specific.
• Reported on v0.0.44; the code path is unchanged on current main (v0.0.80). Two additional users confirm the same behavior.
• The ancestor walk needs a termination condition or allowlist for known VM-driver init processes that cannot be statted.

Recommendation: The fix should either (a) skip the integrity check for ancestor PIDs whose binary path is a known VM init (e.g., /init.krun), (b) terminate the ancestor walk at the entrypoint PID boundary rather than walking into VM-specific init processes, or (c) treat a stat failure on an ancestor above the entrypoint as non-fatal. Option (b) seems safest since it avoids hardcoding VM-specific paths. This affects all VM-driver users with network policies.

[JonasCir]

This also blocks the resolution of #2149, see #2150 (comment)

[TaylorMutch]

📋 triage-agent

Triage Update

Classification: bug-confirmed

Summary

Re-evaluation against current main confirms the defect remains present. The newer #2150 reference adds impact—#1998 blocks validation of that VM work—but #2150 addresses a separate ext4 image-cache problem and is not a fix for this issue.

Investigation

The supervisor records the spawned workload PID as the entrypoint boundary. In openshell-supervisor-network, the ancestor walker reads the current process's parent before checking whether the current PID already equals that boundary. When node owns the connection and is itself the entrypoint, the walk therefore continues above the workload into PID 1 and tries to integrity-check libkrun's synthetic /init.krun. That virtual init has a procfs identity but no stattable guest file, so the fail-closed integrity check returns ENOENT and denies an otherwise-authorized connection. Related ancestor/cmdline helpers share the same stop-boundary semantics.

Recommendation

Correct the traversal boundary before reading the parent: if the current PID is the entrypoint, stop. Descendant processes should still include and verify the entrypoint exactly once. This preserves fail-closed integrity checks within the workload boundary; an /init.krun allowlist or globally ignoring ancestor stat failures would be less safe. Add unit coverage for owner-equals-entrypoint and descendant-owner cases, plus a VM E2E test with an explicit binary network rule.