[HDX-3994] Deprecate clickhouse.json feature gate in favor of per-exporter json config

[wrn14897]

Summary

Deprecate the upstream-deprecated --feature-gates=clickhouse.json CLI flag in favor of the per-exporter json: true config option, as recommended by the OpenTelemetry ClickHouse exporter v0.149.0.

This introduces a new env var HYPERDX_OTEL_EXPORTER_CLICKHOUSE_JSON_ENABLE that controls JSON mode at the exporter config level. The old OTEL_AGENT_FEATURE_GATE_ARG env var remains backward-compatible — when it contains clickhouse.json, the entrypoint strips that gate, maps it to the new env var, and prints a deprecation warning. Other feature gates are preserved and passed through to the collector.

Key changes:

• docker/otel-collector/entrypoint.sh — Detects clickhouse.json in OTEL_AGENT_FEATURE_GATE_ARG, strips it, sets HYPERDX_OTEL_EXPORTER_CLICKHOUSE_JSON_ENABLE=true, and prints a deprecation warning. Remaining feature gates are still passed through to the collector in both standalone and supervisor modes.
• docker/otel-collector/config.standalone.yaml — Added json: ${env:HYPERDX_OTEL_EXPORTER_CLICKHOUSE_JSON_ENABLE:-false} to both ClickHouse exporter configs
• packages/api/src/opamp/controllers/opampController.ts — Added json field to the CollectorConfig type and both ClickHouse exporter configs for OpAMP-managed collectors
• docker/otel-collector/supervisor_docker.yaml.tmpl — Feature gate pass-through preserved for non-clickhouse.json gates (entrypoint strips the deprecated gate before supervisor template renders)
• smoke-tests/otel-collector/ — Added a JSON-enabled otel-collector service and smoke tests verifying:
  • ResourceAttributes and LogAttributes columns in otel_logs are JSON type (not Map)
  • Log data with various attribute types (string, int, boolean) is inserted and queryable via JSON path access

How to test locally or on Vercel

1. Run yarn dev to start the dev stack
2. Verify the otel-collector-json container starts without errors (the clickhouse.json feature gate is stripped, not passed to the collector)
3. Check container logs for the deprecation warning when OTEL_AGENT_FEATURE_GATE_ARG contains clickhouse.json
4. Verify the non-JSON otel-collector service continues to work normally (json defaults to false)
5. Run smoke tests: cd smoke-tests/otel-collector && bats json-exporter.bats

References

• Linear Issue: https://linear.app/hyperdx/issue/HDX-3994
• Upstream deprecation: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/clickhouseexporter#experimental-json-support

[changeset-bot]

🦋 Changeset detected

Latest commit: 414acd2

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@hyperdx/otel-collector	Patch
@hyperdx/api	Patch
@hyperdx/app	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ready	Preview, Comment	Apr 15, 2026 3:43pm

[github-actions]

E2E Test Results

✅ All tests passed • 132 passed • 3 skipped • 1057s

Status	Count
✅ Passed	132
❌ Failed	0
⚠️ Flaky	4
⏭️ Skipped	3

Tests ran across 4 shards in parallel.

View full report →

[github-actions]

🔴 Tier 4 — Critical

Touches auth, data models, config, tasks, OTel pipeline, ClickHouse, or CI/CD.

Why this tier:

• Critical-path files (3):
  • docker/otel-collector/config.standalone.yaml
  • docker/otel-collector/entrypoint.sh
  • docker/otel-collector/supervisor_docker.yaml.tmpl

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats

• Files changed: 14
• Lines changed: 193
• Branch: warren/HDX-3994-deprecate-clickhouse-json-feature-gate
• Author: wrn14897

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

[github-actions]

PR Review

• ⚠️ grep -q "clickhouse.json" at entrypoint.sh:8 — . is a regex metachar, could match clickhouseXjson. Use grep -qF "clickhouse.json" for literal string match.

• ⚠️ sed 's/--feature-gates=//' at entrypoint.sh:14 — only strips the first occurrence. If OTEL_AGENT_FEATURE_GATE_ARG contains gates with +/- prefixes (e.g. +clickhouse.json), the grep -v 'clickhouse.json' filter won't strip them since the pattern includes the prefix. For the documented use case (--feature-gates=clickhouse.json) this is fine, but consider grep -v 'clickhouse\.json' (escaped dot) for correctness.

• ⚠️ Silent override of explicit HYPERDX_OTEL_EXPORTER_CLICKHOUSE_JSON_ENABLE=false — if both env vars are set with conflicting intent, the deprecation migration unconditionally sets it to true with no warning about the override. At minimum, check if it's already set before overriding.

• ✅ supervisor_docker.yaml.tmpl fix (moving args: inside the {{- if }} block) is correct — previously args: was always emitted even with no list items.

• ✅ Backward compatibility, smoke tests, and changeset all look good.