fix(otelite): honor traces endpoint env semantics#778
Draft
schickling-assistant wants to merge 1 commit into
Draft
fix(otelite): honor traces endpoint env semantics#778schickling-assistant wants to merge 1 commit into
schickling-assistant wants to merge 1 commit into
Conversation
ae77b02 to
7da2aa1
Compare
Storybook PreviewsNo storybooks were deployed. |
CI Measurementspartial - advisory gate - readiness
devenv / quality gates
nix / closures / packages
Unchanged / 0-impact measurements (16)These rows had compatible baseline data, but their semantic impact rounded to 0.00x because the movement was below the configured budget, below the noise floor, or inside the robust noise band. devenv / devenv cli
devenv / devenv shell
devenv / genie
devenv / quality gates
devenv / workspace setup
nix / closures / packages
Diagnostic / ungated measurements (7)source / effect-utils
devenv / devenv shell
All measurements
Source-of-truth JSON{
"schemaVersion": 1,
"title": "CI Measurements",
"status": "partial",
"gate": "advisory",
"readiness": "partial (17/26 enabled observations gateable)",
"commit": {
"shortSha": "7da2aa1",
"sha": "7da2aa145347942ffb710b80f83eb2cc42708a2c"
},
"run": {
"id": "27332476724",
"attempt": "1",
"url": "https://github.com/overengineeringstudio/effect-utils/actions/runs/27332476724"
},
"baseline": null,
"protocol": "devenv-perf-warm-median-v2",
"chart": {
"meaning": "semantic-impact",
"zeroImpactMeaning": "no actionable PR impact after budgets, noise floor, and robust evidence checks",
"svg": "https://raw.githubusercontent.com/overengineeringstudio/effect-utils/ci-measurement-assets/ci-measurements/pr-778/7da2aa145347942ffb710b80f83eb2cc42708a2c/run-27332476724-attempt-1/ci-measurements.svg",
"lightPng": "https://raw.githubusercontent.com/overengineeringstudio/effect-utils/ci-measurement-assets/ci-measurements/pr-778/7da2aa145347942ffb710b80f83eb2cc42708a2c/run-27332476724-attempt-1/ci-measurements.png",
"darkPng": "https://raw.githubusercontent.com/overengineeringstudio/effect-utils/ci-measurement-assets/ci-measurements/pr-778/7da2aa145347942ffb710b80f83eb2cc42708a2c/run-27332476724-attempt-1/ci-measurements-dark.png"
},
"measurements": [
{
"id": "task_check_quick_warm",
"label": "Warm cached check:quick",
"group": "devenv / quality gates",
"path": [
"devenv",
"quality gates",
"check:quick"
],
"groupPath": [
"devenv",
"quality gates"
],
"status": "pass",
"direction": "improved",
"gateable": true,
"gateReason": "eligible",
"confidence": "within_budget",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 3.611,
"current": 3.319,
"delta": -0.29200000000000026,
"ratio": 0.9191359734145665,
"semanticImpactScore": -0.24369980614788123,
"semanticImpactKind": "improvement",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -4.147,
"evidenceDeltaUpper": -0.088,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"workload": "cached-no-op",
"taskCacheMode": "warm",
"probe": "task_check_quick_warm",
"probeLabel": "Warm cached check:quick",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "27332476724-1-4c6d5438a367ebdc6a7b2fe89a4f4297be599a67",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "nix.closure.nar_size",
"label": "Total closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"total",
"closure-size",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 561373688,
"current": 561873128,
"delta": 499440,
"ratio": 1.0008896747579663,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10728033.76,
"evidenceDeltaUpper": 11726913.76,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.serialized_nar_size",
"label": "Total serialized NAR size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"total",
"serialized-nar-size",
"nix closure diagnostics"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "regressed",
"gateable": true,
"gateReason": "eligible",
"confidence": "within_budget",
"comparisonMode": "diagnostic",
"unit": "bytes",
"baseline": 561373688,
"current": 561873128,
"delta": 499440,
"ratio": 1.0008896747579663,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 21,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -55637928.800000004,
"evidenceDeltaUpper": 56636808.800000004,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total",
"sizeKind": "nar"
}
},
{
"id": "nix.closure.nar_size",
"label": "Total closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"total",
"closure-size",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 163874192,
"current": 163879208,
"delta": 5016,
"ratio": 1.0000306088465718,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10480744,
"evidenceDeltaUpper": 10490776,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.serialized_nar_size",
"label": "Total serialized NAR size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"total",
"serialized-nar-size",
"nix closure diagnostics"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "regressed",
"gateable": true,
"gateReason": "eligible",
"confidence": "within_budget",
"comparisonMode": "diagnostic",
"unit": "bytes",
"baseline": 163874192,
"current": 163879208,
"delta": 5016,
"ratio": 1.0000306088465718,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 21,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -16382403.200000001,
"evidenceDeltaUpper": 16392435.200000001,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total",
"sizeKind": "nar"
}
},
{
"id": "source.lines",
"label": "Genie runtime lines",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"packages",
"genie",
"source / genie"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "regressed",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "lines",
"baseline": 19423,
"current": 19809,
"delta": 386,
"ratio": 1.0198733460330536,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 23,
"currentSamples": 66,
"pairedSamples": 0,
"evidenceDeltaLower": -1556.3000000000002,
"evidenceDeltaUpper": 2328.3,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "genie_runtime"
}
},
{
"id": "source.lines",
"label": "Genie CI workflow helpers lines",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"genie",
"ci-workflow",
"source / ci"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "regressed",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "lines",
"baseline": 7318,
"current": 7418,
"delta": 100,
"ratio": 1.0136649357748018,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 23,
"currentSamples": 9,
"pairedSamples": 0,
"evidenceDeltaLower": -631.8000000000001,
"evidenceDeltaUpper": 831.8000000000001,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "genie_ci_workflow"
}
},
{
"id": "source.files",
"label": "Genie runtime files",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"packages",
"genie",
"source / genie"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "regressed",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "count",
"baseline": 64,
"current": 66,
"delta": 2,
"ratio": 1.03125,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 23,
"currentSamples": 66,
"pairedSamples": 0,
"evidenceDeltaLower": -4.4,
"evidenceDeltaUpper": 8.4,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "genie_runtime"
}
},
{
"id": "source.lines",
"label": "Nix workspace tools lines",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"nix",
"workspace-tools",
"source / nix"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "regressed",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "lines",
"baseline": 3631,
"current": 3632,
"delta": 1,
"ratio": 1.0002754062241808,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 23,
"currentSamples": 14,
"pairedSamples": 0,
"evidenceDeltaLower": -362.1,
"evidenceDeltaUpper": 364.1,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "nix_workspace_tools"
}
},
{
"id": "task_genie_run",
"label": "Genie run task",
"group": "devenv / genie",
"path": [
"devenv",
"genie"
],
"groupPath": [
"devenv",
"genie"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 1.231,
"current": 1.177,
"delta": -0.05400000000000005,
"ratio": 0.9561332250203086,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -0.054,
"evidenceDeltaUpper": -0.023,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "task_genie_run",
"probeLabel": "Genie run task",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "27332476724-1-4c6d5438a367ebdc6a7b2fe89a4f4297be599a67",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "task_check_quick_forced",
"label": "Forced check:quick",
"group": "devenv / quality gates",
"path": [
"devenv",
"quality gates",
"check:quick"
],
"groupPath": [
"devenv",
"quality gates"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 11.805,
"current": 11.763,
"delta": -0.041999999999999815,
"ratio": 0.9964421855146125,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 3,
"currentSamples": 3,
"pairedSamples": 3,
"evidenceDeltaLower": -0.394,
"evidenceDeltaUpper": -0.021,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"workload": "forced-task-cache",
"taskCacheMode": "refresh",
"probe": "task_check_quick_forced",
"probeLabel": "Forced check:quick",
"status": 0,
"sampleCount": 6,
"warmupCount": 0,
"measuredSampleCount": 3,
"pairedSampleCount": 3,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "27332476724-1-4c6d5438a367ebdc6a7b2fe89a4f4297be599a67",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "shell_eval_warm",
"label": "Warm shell eval",
"group": "devenv / devenv shell",
"path": [
"devenv",
"devenv shell"
],
"groupPath": [
"devenv",
"devenv shell"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 1.903,
"current": 1.88,
"delta": -0.02300000000000013,
"ratio": 0.9879138202837624,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -0.06,
"evidenceDeltaUpper": -0.02,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "shell_eval_warm",
"probeLabel": "Warm shell eval",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "27332476724-1-4c6d5438a367ebdc6a7b2fe89a4f4297be599a67",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "genie_check_direct",
"label": "Genie check direct",
"group": "devenv / genie",
"path": [
"devenv",
"genie"
],
"groupPath": [
"devenv",
"genie"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 5.281,
"current": 5.276,
"delta": -0.004999999999999893,
"ratio": 0.9990532096193903,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -0.005,
"evidenceDeltaUpper": 0.025,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "genie_check_direct",
"probeLabel": "Genie check direct",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "27332476724-1-4c6d5438a367ebdc6a7b2fe89a4f4297be599a67",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "task_pnpm_install",
"label": "pnpm install task",
"group": "devenv / workspace setup",
"path": [
"devenv",
"workspace setup"
],
"groupPath": [
"devenv",
"workspace setup"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 0.741,
"current": 0.744,
"delta": 0.0030000000000000027,
"ratio": 1.0040485829959513,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 5,
"currentSamples": 5,
"pairedSamples": 5,
"evidenceDeltaLower": -0.052,
"evidenceDeltaUpper": 0.005,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "task_pnpm_install",
"probeLabel": "pnpm install task",
"status": 0,
"sampleCount": 11,
"warmupCount": 1,
"measuredSampleCount": 5,
"pairedSampleCount": 5,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "27332476724-1-4c6d5438a367ebdc6a7b2fe89a4f4297be599a67",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "processes_help",
"label": "devenv processes --help",
"group": "devenv / devenv cli",
"path": [
"devenv",
"devenv cli"
],
"groupPath": [
"devenv",
"devenv cli"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 0.022,
"current": 0.021,
"delta": -0.0009999999999999974,
"ratio": 0.9545454545454547,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 9,
"currentSamples": 9,
"pairedSamples": 9,
"evidenceDeltaLower": -0.002,
"evidenceDeltaUpper": 0,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "processes_help",
"probeLabel": "devenv processes --help",
"status": 0,
"sampleCount": 19,
"warmupCount": 1,
"measuredSampleCount": 9,
"pairedSampleCount": 9,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "27332476724-1-4c6d5438a367ebdc6a7b2fe89a4f4297be599a67",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "tasks_list",
"label": "devenv tasks list",
"group": "devenv / devenv cli",
"path": [
"devenv",
"devenv cli"
],
"groupPath": [
"devenv",
"devenv cli"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "paired",
"unit": "seconds",
"baseline": 0.043,
"current": 0.043,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 9,
"currentSamples": 9,
"pairedSamples": 9,
"evidenceDeltaLower": -0.003,
"evidenceDeltaUpper": 0.002,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"probe": "tasks_list",
"probeLabel": "devenv tasks list",
"status": 0,
"sampleCount": 19,
"warmupCount": 1,
"measuredSampleCount": 9,
"pairedSampleCount": 9,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "27332476724-1-4c6d5438a367ebdc6a7b2fe89a4f4297be599a67",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
},
{
"id": "source.files",
"label": "Genie CI workflow helpers files",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"genie",
"ci-workflow",
"source / ci"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "unchanged",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "count",
"baseline": 9,
"current": 9,
"delta": 0,
"ratio": 1,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 23,
"currentSamples": 9,
"pairedSamples": 0,
"evidenceDeltaLower": -1,
"evidenceDeltaUpper": 1,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "genie_ci_workflow"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Nix sources closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"buckets",
"nix-sources",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "nix-sources"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Nix sources closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"buckets",
"nix-sources",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "nix-sources"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Nix sources closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"buckets",
"nix-sources",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "nix-sources"
}
},
{
"id": "source.files",
"label": "Nix workspace tools files",
"group": "source / effect-utils",
"path": [
"source",
"effect-utils",
"nix",
"workspace-tools",
"source / nix"
],
"groupPath": [
"source",
"effect-utils"
],
"status": "pass",
"direction": "unchanged",
"gateable": false,
"gateReason": "disabled",
"confidence": "diagnostic",
"comparisonMode": "budget",
"unit": "count",
"baseline": 14,
"current": 14,
"delta": 0,
"ratio": 1,
"semanticImpactScore": null,
"semanticImpactKind": "diagnostic",
"baselineSources": 23,
"currentSamples": 14,
"pairedSamples": 0,
"evidenceDeltaLower": -1.4000000000000001,
"evidenceDeltaUpper": 1.4000000000000001,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"scope": "nix_workspace_tools"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Node / pnpm closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"buckets",
"node",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "node"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Node / pnpm closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"buckets",
"node",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "node"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Node / pnpm closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"buckets",
"node",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "node"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Rust closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"buckets",
"rust",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "rust"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Rust closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"buckets",
"rust",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "rust"
}
},
{
"id": "nix.closure.bucket.nar_size",
"label": "Rust closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"buckets",
"rust",
"nix closure buckets"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "unknown",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "unknown",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 0,
"current": 0,
"delta": 0,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": "unknown",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "rust"
}
},
{
"id": "nix.closure.path_count",
"label": "Total closure path count",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"genie",
"total",
"path-count",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "count",
"baseline": 79,
"current": 79,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10,
"evidenceDeltaUpper": 10,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.path_count",
"label": "Total closure path count",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"total",
"path-count",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "count",
"baseline": 5,
"current": 5,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10,
"evidenceDeltaUpper": 10,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.path_count",
"label": "Total closure path count",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"oxlint-npm",
"total",
"path-count",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "count",
"baseline": 8,
"current": 8,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10,
"evidenceDeltaUpper": 10,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.nar_size",
"label": "Total closure size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"total",
"closure-size",
"nix closure"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "budget",
"unit": "bytes",
"baseline": 151364760,
"current": 151364760,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 22,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -10485760,
"evidenceDeltaUpper": 10485760,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total"
}
},
{
"id": "nix.closure.serialized_nar_size",
"label": "Total serialized NAR size",
"group": "nix / closures / packages",
"path": [
"nix",
"closures",
"packages",
"megarepo",
"total",
"serialized-nar-size",
"nix closure diagnostics"
],
"groupPath": [
"nix",
"closures",
"packages"
],
"status": "pass",
"direction": "unchanged",
"gateable": true,
"gateReason": "eligible",
"confidence": "noise_floor",
"comparisonMode": "diagnostic",
"unit": "bytes",
"baseline": 151364760,
"current": 151364760,
"delta": 0,
"ratio": 1,
"semanticImpactScore": 0,
"semanticImpactKind": "neutral",
"baselineSources": 21,
"currentSamples": 1,
"pairedSamples": 0,
"evidenceDeltaLower": -15136476,
"evidenceDeltaUpper": 15136476,
"pairedEvidenceQuantile": 0.25,
"dimensions": {
"bucket": "total",
"sizeKind": "nar"
}
},
{
"id": "shell_eval_traced",
"label": "Shell eval with OTEL trace",
"group": "devenv / devenv shell",
"path": [
"devenv",
"devenv shell"
],
"groupPath": [
"devenv",
"devenv shell"
],
"status": "missing_baseline",
"direction": "unknown",
"gateable": false,
"gateReason": "missing_baseline",
"confidence": "missing_baseline",
"comparisonMode": "historical",
"unit": "seconds",
"baseline": null,
"current": 24.583,
"delta": null,
"ratio": null,
"semanticImpactScore": null,
"semanticImpactKind": null,
"baselineSources": 0,
"currentSamples": 1,
"pairedSamples": null,
"evidenceDeltaLower": null,
"evidenceDeltaUpper": null,
"pairedEvidenceQuantile": null,
"dimensions": {
"probe": "shell_eval_traced",
"probeLabel": "Shell eval with OTEL trace",
"status": 0,
"sampleCount": 2,
"warmupCount": 0,
"measuredSampleCount": 1,
"pairedSampleCount": 1,
"pairedOrderProtocol": "balanced-seeded-alternating-v1",
"pairedOrderSeed": "27332476724-1-4c6d5438a367ebdc6a7b2fe89a4f4297be599a67",
"measurementProtocol": "devenv-perf-warm-median-v2",
"aggregation": "median",
"phase": "warm",
"devenvRev": "2cf62a010000b70f15c78a72761fad7c9e6fb47a",
"otelServiceName": "devenv-perf-ci"
}
}
]
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PR #777 added env-backed otelite test harness support, but review caught one OTLP/HTTP endpoint semantics bug after the PR had already merged:
withEnv({ endpointVar: "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT" })wrote the otelite base endpoint. Per-signal OTLP endpoint variables are consumed as full URLs, so a subprocess using that variable would POST to the receiver root instead of/v1/traces.Goal
Make env-backed harness tests work with the standard per-signal traces endpoint variable while preserving the existing base-endpoint behavior for
OTEL_EXPORTER_OTLP_ENDPOINTand custom base endpoint variables.Decisions
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT, because that is the standard traces per-signal variable with full-URL semantics.otlpTracesUrlhelper so the/v1/tracessuffix stays one source of truth.Verification
Locally passed:
The
OTEL_MODE=offoverride is local-only: this shell hasOTEL_MODE=systembut nootelCLI, which otherwise blocks devenv shell entry before repo tests run.Complexity
No new abstraction beyond a tiny endpoint-value helper in the harness.
Concerns
Custom non-standard endpoint variable names still receive the base endpoint. That matches the previous custom-env behavior and avoids guessing semantics from arbitrary names.
Friction & bottlenecks
PR #777 merged while the review fix was being prepared, so this is split into a follow-up PR instead of landing on the original branch.
Follow-ups
None.
References
Follow-up to #777. Addresses the unresolved Codex review thread on #777 about
OTEL_EXPORTER_OTLP_TRACES_ENDPOINTsemantics.Posted on behalf of @schickling
agent_nameagent_session_idagent_toolagent_tool_versionagent_runtimeagent_modelruntime_profileskills_manifestworktreemachinetooling_profile