Skip to content

[HDX-4029] Support all OTel Collector Contrib components in builder-config.yaml#2120

Closed
wrn14897 wants to merge 2 commits intomainfrom
warren/HDX-4029-otel-contrib-all-components
Closed

[HDX-4029] Support all OTel Collector Contrib components in builder-config.yaml#2120
wrn14897 wants to merge 2 commits intomainfrom
warren/HDX-4029-otel-contrib-all-components

Conversation

@wrn14897
Copy link
Copy Markdown
Member

Summary

Update packages/otel-collector/builder-config.yaml to include all components from the upstream opentelemetry-collector-contrib distribution.

Previously, only the HyperDX-specific subset of components was included in the OCB builder manifest. If a user provided a custom OTel config referencing a component not in our list (e.g. kafkareceiver, elasticsearchexporter, k8sattributesprocessor), the collector binary would fail to load.

This change adds the full set of:

  • Receivers (~100 contrib components)
  • Processors (~28 contrib components)
  • Exporters (~48 contrib components)
  • Extensions (~43 contrib components, including encoding and observer sub-packages)
  • Connectors (~12 contrib components)
  • Configuration Providers (~4 contrib providers)

All existing HyperDX-required components remain included. The version placeholder pattern (__OTEL_COLLECTOR_VERSION__ / __OTEL_COLLECTOR_CORE_VERSION__) is preserved for Docker build-time substitution. Components are sorted alphabetically within each section for maintainability.

How to test locally or on Vercel

  1. Build the OTel Collector Docker image — the builder-config.yaml should be valid and OCB should resolve all listed modules
  2. Provide a custom OTel config that references a previously-unsupported component (e.g. kafkareceiver) and verify it loads successfully
  3. Verify existing HyperDX OTel pipeline still functions (otlp receiver, batch processor, clickhouse exporter, etc.)

References

….yaml

Expand the OCB builder manifest to include all receivers, processors,
exporters, extensions, connectors, and configuration providers from the
upstream opentelemetry-collector-contrib distribution. This allows users
to reference any supported component in their custom OTel config files
without the collector binary failing to load.

Previously only the HyperDX-specific subset was included, which meant
any user-referenced component not in our list would cause a config load
failure.

Refs: HDX-4029
@wrn14897 wrn14897 added the ai-generated AI-generated content; review carefully before merging. label Apr 15, 2026
@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Apr 15, 2026

🦋 Changeset detected

Latest commit: b104ba3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@hyperdx/otel-collector Minor
@hyperdx/api Minor
@hyperdx/app Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Copy Markdown

vercel bot commented Apr 15, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hyperdx-oss Ready Ready Preview, Comment Apr 15, 2026 3:51am

Request Review

@wrn14897 wrn14897 marked this pull request as ready for review April 15, 2026 03:48
@github-actions github-actions bot added the review/tier-4 Critical — deep review + domain expert sign-off label Apr 15, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 15, 2026

🔴 Tier 4 — Critical

Touches auth, data models, config, tasks, OTel pipeline, ClickHouse, or CI/CD.

Why this tier:

  • Critical-path files (1):
    • packages/otel-collector/builder-config.yaml

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats
  • Files changed: 2
  • Lines changed: 355
  • Branch: warren/HDX-4029-otel-contrib-all-components
  • Author: wrn14897

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 15, 2026

E2E Test Results

All tests passed • 133 passed • 3 skipped • 970s

Status Count
✅ Passed 133
❌ Failed 0
⚠️ Flaky 2
⏭️ Skipped 3

Tests ran across 4 shards in parallel.

View full report →

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 15, 2026

PR Review

  • ⚠️ Binary size and CI build time will increase significantly (from ~15 to 200+ components) → Acceptable tradeoff per the PR intent, but worth verifying the Docker image size regression doesn't exceed deployment limits and CI timeout budgets before merging
  • ⚠️ No auto-sync mechanism — this manifest was hand-copied from upstream and will drift as contrib adds/removes components → Consider adding a comment or script reference noting how to regenerate (e.g., a link to the upstream manifest + a note to run a diff periodically)
  • ✅ Version placeholder usage is consistent: __OTEL_COLLECTOR_VERSION__ for all core+contrib components, __OTEL_COLLECTOR_CORE_VERSION__ for confmap providers only — matches the sed substitution in the Dockerfile
  • ✅ Changeset correctly marks this as a minor bump for @hyperdx/otel-collector
  • ✅ All previously included HyperDX-required components (clickhouseexporter, otlpreceiver, fluentforwardreceiver, resourcedetectionprocessor, etc.) are still present

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-generated AI-generated content; review carefully before merging. review/tier-4 Critical — deep review + domain expert sign-off

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant