Skip to content

feat(rest): integrate box network tunnel backend#992

Open
G4614 wants to merge 7 commits into
boxlite-ai:mainfrom
G4614:codex/rust-network-rest-standalone
Open

feat(rest): integrate box network tunnel backend#992
G4614 wants to merge 7 commits into
boxlite-ai:mainfrom
G4614:codex/rust-network-rest-standalone

Conversation

@G4614

@G4614 G4614 commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Integrate the REST network tunnel backend on top of the Rust core tunnel handle from PR991.

This PR also consolidates the tunnel endpoint work from PR993:

  • tunnel() eagerly establishes both local and REST tunnel transports.
  • endpoint() returns BoxEndpoint::Fd for local boxes and BoxEndpoint::Url for REST boxes.
  • open_stream() consumes the already-prepared connection; connect() remains a deprecated compatibility alias.

PR993 remains draft because its local endpoint work is now included here.

Test plan:

  • cargo fmt --all -- --check
  • git diff --check
  • Focused Rust tunnel tests are blocked locally because the dependency cache lacks aws-lc-sys; run in CI after PR991 lands.
  • Run Linux/debug-machine REST e2e after PR991 lands.

Dependency: this draft is intentionally stacked on PR991 and is not expected to compile independently before PR991 is merged.

Summary by CodeRabbit

  • New Features

    • Added network tunneling support for boxes.
    • Added APIs to retrieve a public service endpoint and establish authenticated HTTP tunnels to box ports.
    • Added support for both URL- and local stream-based tunnel endpoints.
    • Added upstream-unavailable error reporting for failed tunnel connections.
  • Improvements

    • REST-backed boxes and cloned boxes now support network tunneling.
    • Tunnel streams are prepared immediately for more reliable access.
    • Added TLS support for secure tunnel connections.

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

Failed to post review comments.

GitHub was unavailable or timed out while CodeRabbit was posting the review. Please request a new review later if the pull request still needs one. This happened while posting 2 inline comments. Use @coderabbitai full review to retry the review.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9adf0df0-ee79-491e-b6f8-5c80f563255e

📥 Commits

Reviewing files that changed from the base of the PR and between 2411669 and 771cb67.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (9)
  • openapi/box.openapi.yaml
  • src/boxlite/Cargo.toml
  • src/boxlite/src/litebox/box_impl.rs
  • src/boxlite/src/litebox/mod.rs
  • src/boxlite/src/litebox/network.rs
  • src/boxlite/src/rest/client.rs
  • src/boxlite/src/rest/litebox.rs
  • src/boxlite/src/rest/runtime.rs
  • src/boxlite/src/runtime/backend.rs
⏰ Context from checks skipped due to timeout. (1)
  • GitHub Check: BoxLite review
📝 Walkthrough

Walkthrough

Changes

The PR adds box-scoped network tunnel API contracts, introduces prepared URL and file-descriptor tunnel endpoints, and enables REST-backed authenticated HTTP CONNECT streams for REST-created boxes.

Network tunnel support

Layer / File(s) Summary
Network tunnel API contracts
openapi/box.openapi.yaml
Defines the Network tag, tunnel description and CONNECT operations, BoxServiceEndpoint, and UpstreamUnavailableError.
Prepared tunnel model
src/boxlite/src/litebox/network.rs, src/boxlite/src/litebox/box_impl.rs, src/boxlite/src/litebox/mod.rs, src/boxlite/src/runtime/backend.rs
Adds URL and file-descriptor endpoints, changes BoxTunnel to consume prepared streams, updates local tunnel creation, exports BoxEndpoint, and revises backend documentation and tests.
REST HTTP CONNECT integration
src/boxlite/src/rest/client.rs, src/boxlite/src/rest/litebox.rs, src/boxlite/src/rest/runtime.rs, src/boxlite/Cargo.toml
Adds authenticated HTTP CONNECT setup and tunnel description calls, wires REST network backends for regular and cloned boxes, and adds transport dependencies and coverage.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ApiClient
  participant RestBox
  participant BoxTunnel
  participant UnixStream
  RestBox->>ApiClient: Describe box tunnel
  ApiClient-->>RestBox: Return public URL
  RestBox->>ApiClient: Open HTTP CONNECT tunnel
  ApiClient->>UnixStream: Upgrade connection
  UnixStream-->>RestBox: Return prepared stream
  RestBox->>BoxTunnel: Combine URL and stream
Loading

Possibly related PRs

Suggested reviewers: dorianzheng

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description covers the main points, but it does not follow the required template headings and section structure. Add the required ## Summary, ## Changes, ## How to verify, and optional ## Risks / rollout headings, with brief content under each.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the main change: adding the REST box network tunnel backend.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 oasdiff (1.23.0)
openapi/box.openapi.yaml

Error: failed to load base spec from "/tmp/coderabbit-oasdiff-base.cYqtcY": map key "ValidationError" not found


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@G4614 G4614 force-pushed the codex/rust-network-rest-standalone branch from 5cecb35 to 6b39f16 Compare July 15, 2026 13:17
@cla-assistant

cla-assistant Bot commented Jul 16, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ G4614
❌ Gamnaam Song


Gamnaam Song seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@G4614 G4614 force-pushed the codex/rust-network-rest-standalone branch from c9ad161 to bf8f74c Compare July 16, 2026 11:31
@G4614 G4614 marked this pull request as ready for review July 16, 2026 11:47
@G4614 G4614 requested a review from a team as a code owner July 16, 2026 11:47
@boxlite-agent

boxlite-agent Bot commented Jul 16, 2026

Copy link
Copy Markdown

📦 BoxLite review — 2 issues · 771cb67

Review evidence

  • git diff --numstat origin/main...HEAD && git diff origin/main...HEAD — inspected full diff, 10 files, +541/-121
  • cargo test -p boxlite (litebox::network, rest::client) — cargo not installed in this microVM, no network toolchain
  • grep for other in-repo consumers of BoxTunnel::endpoint()/open_stream()/tunnel() — only used internally + tests; real consumers are downstream SDK users

Risk notes

  • eager tunnel establishment — NetworkHandle::tunnel() now always performs the full connect (2 REST round-trips for remote, 1 gvproxy connect for local) instead of lazily deferring; intentional per PR title/openapi contract, but every tunnel() call now holds a live upstream connection + spawned copy_bidirectional task until consumed or dropped — checked Drop path closes the paired UnixStream and unwinds the pump task, so no permanent leak.
  • BoxEndpoint::Fd aliasing — endpoint() dup(2)s the fd for local boxes instead of returning inert data like the Url case; verified via code reading that open_stream() consumes the original fd separately, so calling both yields two live handles on one connected socket — see finding.
  • REST CONNECT plumbing (client.rs) — reviewed hyper1 handshake/upgrade + timeout(30s) + copy_bidirectional bridge; logic looks correct (io::copy_bidirectional cleans up on either side closing), covered by a new integration-style test using a real TcpListener.
  • Cargo/build footprint — hyper-rustls added without default-features=false, pulling aws-lc-rs/aws-lc-sys/cmake into the prod build while dev-deps separately pin the ring backend for tests — extra native toolchain (cmake/cc) requirement, not verified buildable here since cargo is unavailable in this sandbox.
  • sampled/skipped — Cargo.lock (101 lines) and openapi/box.openapi.yaml (75 lines) reviewed only for contract consistency, not line-by-line; not a hand-authored risk surface.
src/boxlite/src/litebox/network.rs
  BoxTunnel::endpoint/open_stream, BoxEndpoint  +115/-88  lazy connector replaced by eager Fd/Url enum
src/boxlite/src/rest/client.rs
  ApiClient::connect_box_network_tunnel/describe_box_tunnel  +154/-0  new HTTP CONNECT tunnel + descriptor fetch
src/boxlite/src/rest/litebox.rs
  RestBox as BoxNetworkBackend::tunnel  +28/-3  wires REST box into real network backend
src/boxlite/src/litebox/box_impl.rs
  BoxImpl as BoxNetworkBackend::tunnel  +7/-10  eagerly converts local tunnel to fd
src/boxlite/src/runtime/backend.rs
  UnsupportedNetworkBackend removed  +3/-17  REST boxes no longer stubbed unsupported
openapi/box.openapi.yaml
  POST/CONNECT /boxes/{id}/network/tunnel  +75/-0  documents split describe vs CONNECT contract
src/boxlite/Cargo.toml, Cargo.lock
  hyper-rustls, rustls deps  +3/-0 / +101/-0  new TLS client stack for CONNECT
2 findings summary
  • ⚠️ src/boxlite/src/litebox/network.rs:21-29 endpoint() duplicates a live fd, not inert data — For a local box, BoxEndpoint::Fd wraps the actual connected tunnel socket; endpoint() dup(2)s it via try_clone while leaving the original fd in self.endpoint. A caller that calls endpoint() (e.g. to hand the fd to another language's socket layer, the documented into_fd FFI use case) and later also calls open_stream() gets two independent fds sharing one open file description — reads/writes on either steal bytes meant for the other, silently corrupting the tunnel. The Url variant has no such hazard since a String clone is inert; only Fd aliases a stateful, single-consumer resource. Prior code avoided this because endpoint() and connect() drained the same shared stream slot for local boxes rather than duplicating a live socket.
  • ⚠️ src/boxlite/src/rest/litebox.rs:387-405 tunnel() URL now depends on a successful live CONNECT — RestBox::tunnel unconditionally chains describe_box_tunnel (POST) then connect_box_network_tunnel (CONNECT). If the CONNECT leg fails (upstream box unreachable, 502 UpstreamUnavailableError) after the POST succeeded, tunnel() returns Err and the caller can no longer obtain just the public URL — a regression from the previous lazy design where fetching the endpoint and opening the stream were independent operations that could fail separately.

reviewed 771cb67 in a BoxLite microVM · @boxlite-agent review to re-run · powered by BoxLite

@G4614 G4614 enabled auto-merge July 16, 2026 11:47

@boxlite-agent boxlite-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📦 BoxLite review — 2 issues

Comment on lines +21 to +29
fn try_clone(&self) -> BoxliteResult<Self> {
match self {
Self::Url(url) => Ok(Self::Url(url.clone())),
Self::Fd(fd) => fd
.try_clone()
.map(Self::Fd)
.map_err(|error| BoxliteError::Network(format!("clone local tunnel fd: {error}"))),
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ endpoint() duplicates a live fd, not inert data
For a local box, BoxEndpoint::Fd wraps the actual connected tunnel socket; endpoint() dup(2)s it via try_clone while leaving the original fd in self.endpoint. A caller that calls endpoint() (e.g. to hand the fd to another language's socket layer, the documented into_fd FFI use case) and later also calls open_stream() gets two independent fds sharing one open file description — reads/writes on either steal bytes meant for the other, silently corrupting the tunnel. The Url variant has no such hazard since a String clone is inert; only Fd aliases a stateful, single-consumer resource. Prior code avoided this because endpoint() and connect() drained the same shared stream slot for local boxes rather than duplicating a live socket.

Comment on lines +387 to +405
#[async_trait]
impl BoxNetworkBackend for RestBox {
async fn tunnel(&self, target: SocketAddr) -> BoxliteResult<BoxTunnel> {
if target.ip().to_string() != crate::net::constants::GUEST_IP {
return Err(BoxliteError::Unsupported(
"REST box tunnels only support service ports on the guest IP".into(),
));
}

let port = target.port();
let box_id = self.box_id_str();
let endpoint = self.client.describe_box_tunnel(&box_id, port).await?;
let stream = self
.client
.connect_box_network_tunnel(&box_id, port)
.await?;
Ok(BoxTunnel::new(
Some(BoxEndpoint::Url(endpoint)),
Some(crate::net::BoxInternalTunnel::from_local(stream, target)),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ tunnel() URL now depends on a successful live CONNECT
RestBox::tunnel unconditionally chains describe_box_tunnel (POST) then connect_box_network_tunnel (CONNECT). If the CONNECT leg fails (upstream box unreachable, 502 UpstreamUnavailableError) after the POST succeeded, tunnel() returns Err and the caller can no longer obtain just the public URL — a regression from the previous lazy design where fetching the endpoint and opening the stream were independent operations that could fail separately.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@openapi/box.openapi.yaml`:
- Around line 873-874: Align the OpenAPI upstream failure responses, including
the additional occurrence, with the status and error type handled by the Rust
mapping in error.rs. Update the UpstreamUnavailableError contract and its
example so they use the same HTTP status and typed error classification
recognized by the Rust error mapping, preserving consistency in both layers.

In `@src/boxlite/src/rest/client.rs`:
- Around line 391-395: Update the rejected CONNECT branch in the REST client to
read the response body with the existing bounded-body mechanism and pass the
status and body through the shared error mapper instead of always constructing
BoxliteError::Network. Preserve the mapper’s established handling for HTTP
statuses and structured OpenAPI error responses, while retaining the current
success path for StatusCode::OK.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9adf0df0-ee79-491e-b6f8-5c80f563255e

📥 Commits

Reviewing files that changed from the base of the PR and between 2411669 and 771cb67.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (9)
  • openapi/box.openapi.yaml
  • src/boxlite/Cargo.toml
  • src/boxlite/src/litebox/box_impl.rs
  • src/boxlite/src/litebox/mod.rs
  • src/boxlite/src/litebox/network.rs
  • src/boxlite/src/rest/client.rs
  • src/boxlite/src/rest/litebox.rs
  • src/boxlite/src/rest/runtime.rs
  • src/boxlite/src/runtime/backend.rs

Comment thread openapi/box.openapi.yaml
Comment on lines +873 to +874
"502":
$ref: "#/components/responses/UpstreamUnavailableError"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Align the upstream failure contract with the Rust error mapping.

The OpenAPI contract returns HTTP 502 with example type NetworkError, while src/boxlite/src/rest/error.rs Lines 148-153 recognizes this code only as HTTP 503/UpstreamUnavailableError. A contract-compliant 502 response will bypass the typed mapping. Make the status and type consistent across both layers.

Also applies to: 1211-1222

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openapi/box.openapi.yaml` around lines 873 - 874, Align the OpenAPI upstream
failure responses, including the additional occurrence, with the status and
error type handled by the Rust mapping in error.rs. Update the
UpstreamUnavailableError contract and its example so they use the same HTTP
status and typed error classification recognized by the Rust error mapping,
preserving consistency in both layers.

Comment on lines +391 to +395
if response.status() != hyper::StatusCode::OK {
return Err(BoxliteError::Network(format!(
"CONNECT request rejected with status {}",
response.status()
)));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Preserve REST error semantics for rejected CONNECT requests.

Every rejection becomes BoxliteError::Network, so responses such as 401/403/404 and structured OpenAPI errors lose their established mappings. Read the bounded response body and route the status/body through the shared error mapper before returning.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/boxlite/src/rest/client.rs` around lines 391 - 395, Update the rejected
CONNECT branch in the REST client to read the response body with the existing
bounded-body mechanism and pass the status and body through the shared error
mapper instead of always constructing BoxliteError::Network. Preserve the
mapper’s established handling for HTTP statuses and structured OpenAPI error
responses, while retaining the current success path for StatusCode::OK.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant