Skip to content

feat(helm): scrape Envoy Gateway proxy metrics via PodMonitor#48

Open
dmathur0 wants to merge 4 commits into
mainfrom
davesh/envoy-proxy-metrics
Open

feat(helm): scrape Envoy Gateway proxy metrics via PodMonitor#48
dmathur0 wants to merge 4 commits into
mainfrom
davesh/envoy-proxy-metrics

Conversation

@dmathur0

@dmathur0 dmathur0 commented Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Add a PodMonitor that selects the Envoy Gateway data-plane proxy pods (app.kubernetes.io/name=envoy, component=proxy) and scrapes their Prometheus metrics ("metrics" port at /stats/prometheus). Gated behind ingress.type=envoyGateway plus monitoring.podMonitors.envoyProxy.enabled (off by default), with the gateway namespace defaulting to networkPolicy.gatewayNamespace.

Description

Validation

  • Standard CI passes.
  • Kind integration passes, or this PR explains why it was not run.

The kind integration test is manual due to taking ~30 min to complete. When the PR is ready for review,
run Actions -> Kind Integration -> Run workflow against the copy-pr-bot generated
pull-request/<PR_NUMBER> branch. Use the default test_path for the full suite,
or narrow it only while debugging.

Passing Kind Integration run:

Checklist

  • I am familiar with the contributing guidelines in CONTRIBUTING.md.
  • Commits are signed off for DCO compliance.
  • New or existing tests cover these changes, or the PR explains why tests are not needed.
  • Documentation is updated for user-facing behavior changes.
  • Generated artifacts are updated when applicable, such as OpenAPI specs,
    docs screenshots, or Helm/rendered outputs.

Summary by CodeRabbit

  • New Features

    • Added support for monitoring Envoy Gateway proxy pods with Prometheus when observability is enabled.
    • The installer now creates the required monitoring resource automatically during setup.
  • Documentation

    • Clarified that proxy pod metrics are intentionally scraped as a single cluster-scoped resource to avoid duplicate collection.
  • Bug Fixes

    • Improved monitoring setup consistency so proxy metrics are discovered in the expected namespace and configuration.

Add a PodMonitor that selects the Envoy Gateway data-plane proxy pods
(app.kubernetes.io/name=envoy, component=proxy) and scrapes their
Prometheus metrics ("metrics" port at /stats/prometheus). Gated behind
ingress.type=envoyGateway plus monitoring.podMonitors.envoyProxy.enabled
(off by default), with the gateway namespace defaulting to
networkPolicy.gatewayNamespace.
@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds installer-managed Envoy Gateway proxy PodMonitor generation and conditional installation when Envoy Gateway and observability are enabled, documents that the chart does not scrape those pods, and adds unit plus live-cluster validation for the PodMonitor manifest and deployed resource.

Changes

Envoy Gateway Proxy PodMonitor

Layer / File(s) Summary
Manifest builder and shape test
installer/src/nv_config_manager_installer/deployer.py, installer/tests/test_deployer.py
Defines Envoy Gateway namespace and PodMonitor name constants, builds the PodMonitor manifest, and checks the manifest fields.
Conditional installer wiring
installer/src/nv_config_manager_installer/deployer.py, installer/tests/test_deployer.py
Updates _install_crds to apply the proxy PodMonitor only when Envoy Gateway and observability are enabled, with tests for apply and skip command paths.
Chart note and live-cluster test
deploy/helm/values.yaml, src/tests/integration/test_envoy_proxy_podmonitor.py
Adds a chart note about proxy metrics scraping and a live-cluster test that fetches and inspects the installed PodMonitor.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

  • ryanheffernan
  • polarweasel
  • jojung1
  • zsblevins
  • susanhooks

Poem

🐰 I hopped through manifests, soft and bright,
With proxy metrics in cluster sight.
The chart said “not me,” the installer said “go,”
And the PodMonitor sniffed /stats/prometheus below.
Thump, thump—green hops tonight!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 42.11% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding Envoy Gateway proxy metric scraping via a PodMonitor.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch davesh/envoy-proxy-metrics

Comment @coderabbitai help to get the list of available commands.

@dmathur0

Copy link
Copy Markdown
Collaborator Author

our clusters already came with this podmonitor from the envoy gateway install looks like

@github-actions

Copy link
Copy Markdown

dmathur0 added 2 commits June 26, 2026 17:49
Envoy Gateway proxies are a shared, cluster-scoped resource and only emit
prometheus.io/scrape annotations (no PodMonitor), which operator/CR-based
scrapers like the bundled Alloy stack ignore. A per-release chart PodMonitor
also duplicate-scraped every gateway proxy and collided with the broad
PodMonitor that gateway-owning deployments (e.g. prod ArgoCD) already ship.

Move scraping to whoever installs Envoy Gateway: drop the chart-level
monitoring.podMonitors.envoyProxy PodMonitor and instead have the TUI
installer apply one cluster-scoped PodMonitor in envoy-gateway-system,
gated on install_envoy_gateway AND observability_enabled (so the monitoring
CRDs and the Alloy scraper both exist), applied after the CRDs register.
Add a live-cluster integration test that verifies the installer-managed
cluster-scoped envoy-proxy PodMonitor exists in envoy-gateway-system with the
expected proxy selector and metrics/stats endpoint. Gated behind a skipif on
OBSERVABILITY (mirroring test_probe_labels.py): skipped at collection unless the
kind job deploys the observability stack, where a missing PodMonitor is then a
real failure of the envoy-gateway + observability install path.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@installer/src/nv_config_manager_installer/deployer.py`:
- Around line 702-706: The selector in the Envoy scrape manifest is too loose
because it only matches component=proxy and managed-by=envoy-gateway, missing
the stable app.kubernetes.io/name=envoy label. Update the selector in the
deployer logic that builds this manifest to include the Envoy pod label set,
then adjust the related unit test and live-cluster assertion so they expect the
tightened selector from the same manifest-building path.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b6356ea3-68ca-4390-8f8e-8a3b74fe17ea

📥 Commits

Reviewing files that changed from the base of the PR and between 543d037 and ec3a320.

📒 Files selected for processing (4)
  • deploy/helm/values.yaml
  • installer/src/nv_config_manager_installer/deployer.py
  • installer/tests/test_deployer.py
  • src/tests/integration/test_envoy_proxy_podmonitor.py
✅ Files skipped from review due to trivial changes (1)
  • deploy/helm/values.yaml

Comment on lines +702 to +706
"selector": {
"matchLabels": {
"app.kubernetes.io/component": "proxy",
"app.kubernetes.io/managed-by": "envoy-gateway",
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Add the stable Envoy pod label to the selector.

This manifest does not implement the PR’s advertised target contract: it matches component=proxy plus managed-by=envoy-gateway, but omits app.kubernetes.io/name=envoy. With namespaceSelector.any: true, that makes the scrape target depend on a looser implementation detail instead of the stated Envoy pod label set. Please tighten the selector here, then update the unit and live-cluster assertions to match.

Suggested change
             "selector": {
                 "matchLabels": {
+                    "app.kubernetes.io/name": "envoy",
                     "app.kubernetes.io/component": "proxy",
                     "app.kubernetes.io/managed-by": "envoy-gateway",
                 },
             },
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"selector": {
"matchLabels": {
"app.kubernetes.io/component": "proxy",
"app.kubernetes.io/managed-by": "envoy-gateway",
},
"selector": {
"matchLabels": {
"app.kubernetes.io/name": "envoy",
"app.kubernetes.io/component": "proxy",
"app.kubernetes.io/managed-by": "envoy-gateway",
},
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@installer/src/nv_config_manager_installer/deployer.py` around lines 702 -
706, The selector in the Envoy scrape manifest is too loose because it only
matches component=proxy and managed-by=envoy-gateway, missing the stable
app.kubernetes.io/name=envoy label. Update the selector in the deployer logic
that builds this manifest to include the Envoy pod label set, then adjust the
related unit test and live-cluster assertion so they expect the tightened
selector from the same manifest-building path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant