Skip to content

feat(api): capture client IP via proxy headers & SPIFFE (#358 follow-up)#908

Open
ymh1874 wants to merge 2 commits into
openstack-experimental:mainfrom
ymh1874:feature/358-proxy-headers-spiffe
Open

feat(api): capture client IP via proxy headers & SPIFFE (#358 follow-up)#908
ymh1874 wants to merge 2 commits into
openstack-experimental:mainfrom
ymh1874:feature/358-proxy-headers-spiffe

Conversation

@ymh1874

@ymh1874 ymh1874 commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to #842 for the reopened #358. #842 captured the raw TCP peer on the public HTTP listener; per the maintainer comment two gaps remained (proxy header parsing, SPIFFE), while UDS is acceptable to skip.

Proxy header parsing (config-gated, off by default)

New [oslo_middleware] enable_proxy_headers_parsing flag mirroring upstream Python Keystone's oslo.middleware (HTTPProxyToWSGI), off by default. When enabled, a middleware on the public interface parses RFC 7239 Forwarded (preferred) then X-Forwarded-For, takes the leftmost originating client, and overwrites ConnectInfo<SocketAddr>. Every downstream consumer (the request tracing span's client.addr, the API-Key IP allowlist, any future IP-based login control) transparently sees the real client.

Enabling it asserts the immediate peer is a trusted proxy; left off by default, a deployment not behind such a proxy cannot be tricked into trusting a spoofed header. The layer is wired only on the public interface — never on the internal SPIFFE / admin listeners.

SPIFFE internal interface (client_info was not covered)

The internal mTLS listener uses hyper::service::service_fn, which bypasses axum's make-service, so it never populated ConnectInfo. The accepted TCP peer address is now injected by hand alongside the existing SVID / Interface extensions, so client.addr is captured on the internal interface too. Extracted into a small attach_request_context helper for testability.

The admin UDS interface is intentionally left uncovered — a Unix socket has no meaningful SocketAddr (acceptable per the issue).

Tests

  • Proxy-header parsing units (IPv4, ip:port, bracketed/bare IPv6, RFC 7239 params, obfuscated identifiers, precedence) + in-process middleware tests driving the real into_make_service_with_connect_info path.
  • A composed public-listener test proving proxy rewrite + trailing-slash normalization (Implement default redirects for trailing slash in the url #734) + connect-info all compose, and that the layer runs before routing/normalization.
  • A [oslo_middleware] config deserialization test.
  • A SPIFFE attach_request_context unit test asserting ConnectInfo/Interface/SVID are attached.

cargo fmt --check and cargo clippy --lib --tests clean; full keystone lib + config suites pass.

🤖 Generated with Claude Code

@ymh1874 ymh1874 force-pushed the feature/358-proxy-headers-spiffe branch from 22bd084 to 2a4b898 Compare July 3, 2026 11:50
@ymh1874 ymh1874 requested a review from gtema July 3, 2026 11:54

@gtema gtema left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While it is good that you add certain compliance to oslo_middleware and rely on it's configuration, it does not by default implement sufficient security:

Proper model: don't blindly trust leftmost entry from any peer just cuz flag on. Standard practice (nginx, oslo.middleware, Go's httputil, etc):

  1. Trusted-proxy allowlist, not just boolean flag.
    Config should hold list of trusted proxy CIDRs (trusted_proxies = [...]). Middleware only rewrites ConnectInfo if immediate TCP peer is in that list. Else ignore header entirely — untrusted client hitting listener directly could forge X-Forwarded-For and set whatever IP it wants (bypass IP allowlist, spoof audit logs).

  2. Walk chain right-to-left, strip trusted hops.
    X-Forwarded-For: client, proxy1, proxy2 — proxy2 (closest, direct peer) appended last. Correct algo:

  • start from rightmost entry
  • if it's a trusted proxy, pop it, check next-left
  • first entry NOT in trusted set = real client
  • leftmost entry alone (current impl) is trivially spoofable — attacker just prepends fake IP as first hop, no proxy needed to strip it since nothing validates chain length/authenticity of entries left of the real edge proxy.
  1. Cap hop count / header length. Guard against unbounded comma list (DoS on parsing) — set max hops parsed (e.g. 5-10).

  2. Same treatment for Forwarded header — parse for= chain right-to-left same way, by= optionally cross-checked against known proxy identity.


I would at least extend the oslo_middleware config with the 'trusted_proxies' to do more secure extraction

Comment thread crates/config/src/lib.rs Outdated
#[serde(default)]
pub mapping: MappingProvider,

/// `[oslo_middleware]` configuration (proxy header parsing, issue #358).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you do not need to refer to the issue here. It make sense only for some critical behavior (vulnerabilities, bugs, etc), but not a regular feature trackers

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — removed the issue reference from the doc comment (and the other gratuitous #358 references in the touched comments).

@ymh1874 ymh1874 force-pushed the feature/358-proxy-headers-spiffe branch from 2a4b898 to 45731b5 Compare July 3, 2026 22:12
@ymh1874

ymh1874 commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks — you were right that the boolean flag plus leftmost-X-Forwarded-For was spoofable. Reworked the extraction to the trusted-proxy model:

  1. Trusted-proxy allowlist — added [oslo_middleware] trusted_proxies (CIDR list). The forwarding header is honoured only when the immediate TCP peer is within it; an empty list (default) trusts no one, so the raw peer is always kept.
  2. Right-to-left walk — the effective client is now the rightmost address in the chain that is not itself a trusted proxy, so a prepended spoofed leftmost entry is never taken.
  3. Hop cap — parsing is bounded to the rightmost 10 hops to guard against an unbounded comma list.
  4. Forwarded too — the same right-to-left for= walk is applied to the RFC 7239 header.

As you noted this correlates with the API-key ingress, so rather than a second near-identical resolver I extracted the logic into a shared core::api::forwarded::resolve_client_ip that both the SCIM ingress and this middleware call; the shared version also picked up Forwarded support and the hop cap. ConnectInfo is only overwritten when a different upstream client is recovered, so a direct client keeps its real source port. Tests cover the peer-trust gate, the right-to-left walk, spoofed-leftmost, Forwarded, IPv6/port, and the hop cap.

@ymh1874 ymh1874 requested a review from gtema July 3, 2026 22:13
@ymh1874 ymh1874 force-pushed the feature/358-proxy-headers-spiffe branch from 45731b5 to 93d3e46 Compare July 4, 2026 10:29
ymh1874 and others added 2 commits July 4, 2026 19:49
Follow-up to openstack-experimental#842 for the reopened issue openstack-experimental#358. That PR captured the raw
TCP peer on the public HTTP listener; two gaps remained.

Proxy header parsing (config-gated, off by default): add an
`[oslo_middleware] enable_proxy_headers_parsing` flag mirroring upstream
Python Keystone's oslo.middleware. When enabled, a middleware on the
public interface parses RFC 7239 `Forwarded` (preferred) and
`X-Forwarded-For`, takes the leftmost originating client, and overwrites
`ConnectInfo<SocketAddr>` so every downstream consumer (the request span,
the API-Key IP allowlist, future IP-based login control) transparently
sees the real client. Left off by default so a deployment not behind a
trusted proxy cannot be tricked into trusting a spoofed header.

SPIFFE internal interface: the mTLS listener bypasses axum's make-service,
so it never populated `ConnectInfo`. Inject the accepted TCP peer address
by hand alongside the existing SVID/interface extensions, so `client.addr`
is captured on the internal interface too. The admin UDS interface is
intentionally left uncovered (no meaningful `SocketAddr`).

Tests: proxy-header parsing and middleware units; a composed
public-listener test proving proxy rewrite + trailing-slash normalization
(openstack-experimental#734) + connect-info compose; a `[oslo_middleware]` config test; and a
SPIFFE `attach_request_context` unit test.

Note: This commit was done with the help of AI.
Signed-off-by: Yousef Hussein <ymh1874@gmail.com>
Address review feedback: the previous middleware trusted any peer once
the flag was on and took the leftmost `X-Forwarded-For` entry, which a
direct client can trivially spoof.

Extraction now follows the trusted-proxy model already used by the
API-key ingress:

- Add `[oslo_middleware] trusted_proxies`, a CIDR allowlist. The header
  is honoured only when the immediate TCP peer falls within it; an empty
  list (the default) trusts no one, so the raw peer is always kept.
- Resolve the client by walking the chain right-to-left and returning
  the first address that is not itself a trusted proxy, instead of
  blindly taking the leftmost entry.
- Cap the number of parsed hops to guard against an unbounded
  comma-list (parsing DoS).
- Apply the same right-to-left walk to the RFC 7239 `Forwarded` header.

The resolution logic is extracted into a shared `core::api::forwarded`
module so the SCIM API-key ingress and this middleware use one
implementation rather than two near-identical copies; the shared version
also gains `Forwarded` support and the hop cap. `ConnectInfo` is only
overwritten when a different upstream client is recovered, so a direct
client's source port is preserved.

Also drop the issue-tracker reference from the `[oslo_middleware]`
config doc comment.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Yousef Hussein <ymh1874@gmail.com>
@ymh1874 ymh1874 force-pushed the feature/358-proxy-headers-spiffe branch from 93d3e46 to e64d3a0 Compare July 4, 2026 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants