Problem Statement
Sandbox pods/containers currently request CAP_SYS_ADMIN, CAP_NET_ADMIN, CAP_SYS_PTRACE, and CAP_SYSLOG. With Kubernetes user namespaces (GA in 1.36, already supported via enable_user_namespaces) the first three become namespaced and lose their host-level power — but CAP_SYSLOG does not. The kernel checks syslog(2)//dev/kmsg access against the initial user namespace (capable(), not ns_capable()), so in a hostUsers: false pod the granted CAP_SYSLOG is a no-op and the bypass detection monitor silently degrades.
The only consumer of CAP_SYSLOG is the bypass detection monitor in openshell-supervisor-process, which tails dmesg --follow for nftables log entries (prefix openshell:bypass:<ns>:) emitted by the per-workload netns REJECT rules. Reading the host-global kernel ring buffer from inside a sandbox is also a mild information leak, and requires the nf_log_all_netns sysctl on the node.
Proposed Design
Switch bypass log delivery from kernel-printk to nfnetlink_log (NFLOG):
- During netns setup (privileged startup, before supervisor seccomp hardening), bind an
AF_NETLINK/NETLINK_NETFILTER socket to an NFLOG group inside the workload netns. Netns affinity is fixed at socket() time; binding needs only CAP_NET_ADMIN over the netns owner, which works under user namespaces.
- Generate the nftables log rules as
log prefix <p> group <N> when the socket bind succeeds; keep the existing log ... flags skuid printk rules as an automatic fallback when it fails (e.g. kernel without nfnetlink_log).
- The bypass monitor consumes NFLOG packet messages (prefix attribute for filtering,
NFULA_UID for workload UID, raw IP payload for dst/ports/proto) and reports through the existing OCSF dual-emit path. The dmesg path remains as fallback (still useful in the VM driver, which runs with full capabilities).
- Drop
CAP_SYSLOG from the Kubernetes, Docker, and Podman driver capability sets.
Enforcement is unaffected: the nftables REJECT rules do not depend on the log delivery mechanism. NFLOG delivery is netns-scoped, so the monitor no longer reads host-global kernel logs, and nf_log_all_netns is no longer needed on the NFLOG path.
Alternatives Considered
- Keep dmesg, relax
kernel.dmesg_restrict on nodes — exposes host kernel logs to every pod on the node; strictly worse.
- eBPF-based detection —
bpf() is also initial-userns-gated (CAP_BPF/CAP_SYS_ADMIN in init ns), so it has the same problem as CAP_SYSLOG, plus a new dependency.
- Netlink crate dependency (
netlink-packet-netfilter / libnetfilter_log bindings) — the required protocol subset (group bind, copy mode, packet attr parsing) is ~stable kernel ABI and small; a hand-rolled libc implementation avoids new dependencies in a security-sensitive crate and matches its existing raw-FFI style.
Agent Investigation
CAP_SYSLOG's only consumer is bypass_monitor::spawn (crates/openshell-supervisor-process/src/bypass_monitor/mod.rs); drivers request it in openshell-driver-kubernetes/src/driver.rs, openshell-driver-docker/src/lib.rs, openshell-driver-podman/src/container.rs.
- No eBPF usage anywhere in the workspace; netns/nftables management shells out to
iproute2/nft, so NFLOG covers the whole detection path.
- The supervisor seccomp prelude does not restrict
socket(); the workload runtime filter blocks non-NETLINK_ROUTE netlink sockets, so the sandboxed workload cannot bind the NFLOG group itself.
- The sidecar topology has no bypass log listener today (no
CAP_SYSLOG there either); it keeps printk rules unchanged.
- Implementation is complete on a branch with unit tests (netlink message layout, datagram parsing, IPv4/IPv6 packet parsing, ruleset generation); Linux
cargo check/clippy (pedantic+nursery)/tests pass. PR to follow.
Problem Statement
Sandbox pods/containers currently request
CAP_SYS_ADMIN,CAP_NET_ADMIN,CAP_SYS_PTRACE, andCAP_SYSLOG. With Kubernetes user namespaces (GA in 1.36, already supported viaenable_user_namespaces) the first three become namespaced and lose their host-level power — butCAP_SYSLOGdoes not. The kernel checkssyslog(2)//dev/kmsgaccess against the initial user namespace (capable(), notns_capable()), so in ahostUsers: falsepod the grantedCAP_SYSLOGis a no-op and the bypass detection monitor silently degrades.The only consumer of
CAP_SYSLOGis the bypass detection monitor inopenshell-supervisor-process, which tailsdmesg --followfor nftableslogentries (prefixopenshell:bypass:<ns>:) emitted by the per-workload netns REJECT rules. Reading the host-global kernel ring buffer from inside a sandbox is also a mild information leak, and requires thenf_log_all_netnssysctl on the node.Proposed Design
Switch bypass log delivery from kernel-printk to nfnetlink_log (NFLOG):
AF_NETLINK/NETLINK_NETFILTERsocket to an NFLOG group inside the workload netns. Netns affinity is fixed atsocket()time; binding needs onlyCAP_NET_ADMINover the netns owner, which works under user namespaces.log prefix <p> group <N>when the socket bind succeeds; keep the existinglog ... flags skuidprintk rules as an automatic fallback when it fails (e.g. kernel withoutnfnetlink_log).NFULA_UIDfor workload UID, raw IP payload for dst/ports/proto) and reports through the existing OCSF dual-emit path. Thedmesgpath remains as fallback (still useful in the VM driver, which runs with full capabilities).CAP_SYSLOGfrom the Kubernetes, Docker, and Podman driver capability sets.Enforcement is unaffected: the nftables REJECT rules do not depend on the log delivery mechanism. NFLOG delivery is netns-scoped, so the monitor no longer reads host-global kernel logs, and
nf_log_all_netnsis no longer needed on the NFLOG path.Alternatives Considered
kernel.dmesg_restricton nodes — exposes host kernel logs to every pod on the node; strictly worse.bpf()is also initial-userns-gated (CAP_BPF/CAP_SYS_ADMINin init ns), so it has the same problem asCAP_SYSLOG, plus a new dependency.netlink-packet-netfilter/libnetfilter_logbindings) — the required protocol subset (group bind, copy mode, packet attr parsing) is ~stable kernel ABI and small; a hand-rolledlibcimplementation avoids new dependencies in a security-sensitive crate and matches its existing raw-FFI style.Agent Investigation
CAP_SYSLOG's only consumer isbypass_monitor::spawn(crates/openshell-supervisor-process/src/bypass_monitor/mod.rs); drivers request it inopenshell-driver-kubernetes/src/driver.rs,openshell-driver-docker/src/lib.rs,openshell-driver-podman/src/container.rs.iproute2/nft, so NFLOG covers the whole detection path.socket(); the workload runtime filter blocks non-NETLINK_ROUTEnetlink sockets, so the sandboxed workload cannot bind the NFLOG group itself.CAP_SYSLOGthere either); it keeps printk rules unchanged.cargo check/clippy(pedantic+nursery)/tests pass. PR to follow.