feat(proxy): add namespace- or cluster-shared proxy placement (draft RFC)

[dhirajsb]

Problem Statement

The current sandbox-local proxy model gives each sandbox a nearby policy and credential enforcement point. That is a useful isolation property, but deploying the same proxy runtime alongside every sandbox also repeats its baseline CPU, memory, connection pools, caches, and lifecycle work. At hundreds or thousands of concurrent sandboxes, the repeated fixed cost can become material even when individual sandboxes generate little traffic.

This draft proposes an optional multi-tenant shared proxy deployment. An operator could run a proxy pool per namespace or per cluster instead of one proxy process per sandbox. The design must retain the important security property of the current model: a request from sandbox A must never select sandbox B's policy, provider state, credential material, cache entries, or audit identity.

The goal is an additional deployment option, not a replacement for the sandbox-local proxy. Local proxies remain appropriate when operators prioritize the smallest failure domain, offline operation, or strict process-level separation.

Proposed Design

Deployment modes

Add an operator-controlled proxy placement setting with three values:

[openshell.drivers.kubernetes.proxy]
placement = "sandbox" # sandbox | namespace | cluster
replicas = 3

• sandbox preserves the current placement and remains the default.
• namespace deploys an independently scalable proxy pool for sandboxes in one namespace or tenant boundary.
• cluster deploys a shared proxy pool for multiple namespaces, with stronger admission, identity, and policy-partitioning requirements.

Workloads cannot choose their own placement. The gateway and Kubernetes driver publish the selected proxy endpoint and bootstrap identity when creating a sandbox.

flowchart LR
    subgraph T1[Namespace or tenant A]
        A1[Sandbox A1]
        A2[Sandbox A2]
    end
    subgraph T2[Namespace or tenant B]
        B1[Sandbox B1]
    end
    A1 -->|Authenticated tunnel| P[Shared OpenShell proxy pool]
    A2 -->|Authenticated tunnel| P
    B1 -->|Authenticated tunnel| P
    P --> G[Gateway policy and credential authority]
    P --> U1[Backend service 1]
    P --> U2[Backend service 2]

Loading

Sandbox identity and tunnel establishment

Each sandbox receives a short-lived, non-exportable or tightly scoped client identity during admission. One possible implementation is a client certificate whose SAN identifies the gateway, tenant, namespace, and immutable sandbox ID. An equivalent workload-identity token is acceptable if it provides audience restriction, replay resistance, expiry, and proof of possession.

The sandbox opens an authenticated, encrypted tunnel to the shared proxy. HTTP CONNECT, HTTP/2, or a small framed transport can carry original destination metadata and proxied bytes inside that tunnel. The proxy derives tenant and sandbox context only from the authenticated connection. It ignores or rejects caller-supplied identity headers.

The gateway maintains the authoritative binding:

(gateway_id, tenant_id, sandbox_id, identity_key)
    -> policy_revision
    -> allowed provider/profile references
    -> credential-resolution capability

Connections are rejected when the identity is expired, revoked, belongs to another gateway, or cannot be bound to a current sandbox record.

Per-request isolation

The shared process may serve many sandboxes, but every request executes within an immutable SandboxRequestContext created from the authenticated tunnel:

SandboxRequestContext {
  gateway_id,
  tenant_id,
  namespace,
  sandbox_id,
  policy_revision,
  connection_id,
}

Policy lookup, DNS resolution, provider selection, credential resolution, middleware, rate limits, caches, metrics, and audit events must require this context. APIs that accept only a user-provided sandbox ID are not sufficient.

sequenceDiagram
    participant S as Sandbox A
    participant P as Shared proxy
    participant G as Gateway
    participant C as Credential provider
    participant U as Backend service

    S->>P: Establish tunnel with sandbox-bound identity
    P->>G: Validate identity and fetch policy revision
    G-->>P: Signed or authenticated sandbox context
    S->>P: Request backend route
    P->>P: Evaluate policy using authenticated context
    alt route allowed
        P->>G: Resolve credential using sandbox-scoped capability
        G->>C: Mint or retrieve credential for Sandbox A
        C-->>G: Short-lived credential
        G-->>P: Credential for this route and context
        P->>U: Inject credential and call backend
        U-->>P: Response
        P-->>S: Response without credential material
    else route denied
        P-->>S: Deny and audit as Sandbox A
    end

Loading

The following are required isolation invariants:

• Authentication precedes policy lookup and credential resolution.
• A credential-resolution capability is bound to sandbox identity, provider/profile, destination or audience, and expiry. The proxy cannot change the sandbox ID to retrieve another sandbox's material.
• Raw credentials are never sent to the sandbox. They are injected only on the authorized upstream request.
• Credential and policy caches use a compound tenant and sandbox key. Shared or global cache keys are prohibited for sandbox-specific data.
• Cache entries have bounded lifetimes and are invalidated on sandbox deletion, identity revocation, provider rotation, and policy revision changes.
• Per-request state is cleared before a worker handles another request. Connection pooling to upstreams may be shared only when the authentication protocol and destination permit it; authorization headers and request metadata are reconstructed for every call.
• Logs and metrics carry tenant and sandbox identity while excluding credential values. Administrative diagnostics require explicit authorization and preserve the same partitioning.
• NetworkPolicy limits sandbox workloads to the proxy endpoint and required control-plane services. Proxy workloads run under distinct service accounts and network policies.

Policy distribution and failure behavior

The gateway remains the policy and credential authority. Proxy replicas receive either a signed policy snapshot or an authenticated response containing the sandbox ID, revision, and expiry. A replica may cache that snapshot for availability, but it fails closed when the snapshot expires or its revision is revoked.

Proxy replicas should be stateless apart from bounded caches. A service distributes tunnels across replicas; long-lived connections remain pinned to one replica until drained. If a replica fails, the sandbox reconnects and re-authenticates to another replica. Horizontal scaling therefore does not require moving credential state between proxies.

Namespace placement limits the operational and security blast radius and is a reasonable first implementation. Cluster placement can reuse the same protocol after cross-namespace identity, authorization, quota, and audit isolation pass conformance tests.

Resource model

The shared design changes fixed cost from approximately one runtime baseline per sandbox to one baseline per proxy replica, plus per-active-connection state:

sandbox-local:  N * proxy_baseline + active_request_cost
shared pool:    R * proxy_baseline + connection_context_cost + active_request_cost

where N is the number of sandboxes and R is the number of proxy replicas. Normally R is sized for traffic, availability, and failure-domain requirements rather than sandbox count. The proposal should be evaluated with measured idle memory, CPU, connection, tail-latency, and credential-refresh costs; it does not assume that shared placement is more efficient for every workload pattern.

Rollout and validation

1. Define the authenticated tunnel and SandboxRequestContext contract without changing the default placement.
2. Implement namespace-scoped proxy pools with per-sandbox policy and credential capabilities.
3. Add adversarial conformance tests: cross-sandbox ID substitution, stale identity replay, cache-key collision, connection reuse, policy-revision rollback, provider rotation, replica failover, and direct proxy access without a tunnel.
4. Measure resource use and latency against sandbox-local placement at representative idle and active sandbox counts.
5. Consider cluster placement only after namespace placement meets isolation, availability, quota, and observability requirements.

Risks and tradeoffs

• A shared proxy increases blast radius. A defect, overload event, or compromise can affect multiple sandboxes. Namespace pools, least-privilege service accounts, replica isolation, quotas, and independent failure domains reduce but do not remove that risk.
• The authenticated tunnel and context plumbing add complexity to every proxy code path. Compile-time context requirements and negative isolation tests are important safeguards.
• An extra network hop may increase latency relative to an in-pod proxy. Pooling and independent scaling may offset some connection setup cost, but this should be measured rather than assumed.
• Noisy-neighbor traffic can consume shared capacity. Per-tenant and per-sandbox concurrency, bandwidth, queue, and credential-mint quotas are required.
• A central pool needs high availability and upgrade draining. Sandbox-local placement has more processes but smaller correlated failure domains.
• Some compliance environments may require process or node separation per tenant. The local or namespace placement remains available for those deployments.

Open questions

• Should namespace placement be the only initial scope, with cluster placement treated as a later extension?
• Which identity mechanism best fits the existing supervisor-to-gateway trust model while providing proof of possession at the proxy?
• Should policy snapshots be pushed to proxies, fetched on tunnel establishment, or evaluated remotely for every request?
• Which upstream connection pools can be shared safely, and which must be partitioned by sandbox or credential identity?
• What quantitative resource and latency thresholds should justify enabling shared placement?
• How should operators express tenant boundaries when they do not align one-to-one with Kubernetes namespaces?

Feedback is welcome on the isolation invariants, the smallest useful first phase, and cases where sandbox-local placement should remain mandatory. The proposal is intended to leave room for a different tunnel or identity mechanism if it provides the same security properties with less complexity.

Alternatives Considered

Keep one proxy per sandbox

This provides the simplest identity binding and smallest failure domain. It remains the default and may be preferable for low sandbox counts, strict separation requirements, or highly variable proxy versions. Its main cost is repeated runtime and operational overhead as sandbox count grows.

Namespace-level proxy only

This is simpler than supporting both namespace and cluster pools and aligns naturally with common tenant boundaries. It is a strong candidate for the first implementation. The broader interface should avoid preventing a later cluster pool if operators need it.

Node-local shared proxy

A daemon on each node reduces network distance and limits failures to a node. It couples proxy availability and upgrades to node scheduling, and secure sandbox identity still cannot be inferred from source IP alone. It could be another placement behind the same authenticated-tunnel contract.

Shared proxy with identity headers

Passing a sandbox ID in an HTTP header is operationally simple but forgeable by the workload unless another trusted component overwrites and authenticates it. This does not meet the isolation requirement and is not proposed.

Share only credential refresh workers

Centralizing refresh work while retaining local proxies could reduce some repeated provider activity without changing the request path. It would not remove each proxy runtime's fixed memory, listeners, and caches, but it may be a useful smaller optimization.

Do nothing

The current placement remains secure and understandable. Operators would continue sizing capacity for a proxy runtime per sandbox and could pursue implementation-level footprint reductions instead of a new topology.

Agent Investigation

• Reviewed the current RFC process, feature-request template, gateway architecture, proxy-related code search results, and the supervisor middleware deployment-options appendix.
• Reviewed related topology and isolation issues feat: Support restricted SecurityContextConstraints for managed Kubernetes platforms #899, Proposal: Split Supervisor and Agent into Separate Pods with gVisor Isolation #981, rfc-0005: propose sandbox proxy egress adapter model #1511, feat: establish the Isolation Backend interface #1737, and cleanup(supervisor-*): Cleanup supervisor crates & prepare for split-pod topology #1810.
• Searched existing issues for namespace-level, cluster-level, multi-tenant, and shared-proxy proposals. No issue found described the combination of authenticated sandbox tunnels, sandbox-scoped credential capabilities, and namespace/cluster proxy placement.
• The design intentionally treats source IP, namespace, DNS, and caller-provided headers as insufficient identity. Credential isolation is anchored in a cryptographic sandbox identity and gateway-authorized context.
• The proposal is compatible in direction with the isolation-backend discussion in feat: establish the Isolation Backend interface #1737; maintainers may prefer to define the shared proxy as one backend or deployment of that interface.

Checklist

• I've reviewed existing issues and the architecture docs
• This is a design proposal, not a "please build this" request