[otel-advisor] OTel improvement: normalize INPUT_JOB_NAME in action_conclusion_otlp.cjs to restore missing gh-aw.agent.agent span and GenAI att [Content truncated due to length]

[github-actions]

📡 OTel Instrumentation Improvement: restore the dedicated agent span and GenAI semantic attributes

Analysis Date: 2026-05-21
Priority: High
Effort: Small (< 2h)

Problem

In actions/setup/js/send_otlp_span.cjs:1666 the conclusion-span builder reads the job name directly from the environment:

const jobName = process.env.INPUT_JOB_NAME || "";

but actions/setup/js/action_conclusion_otlp.cjs does not normalize the GitHub Actions hyphen form (INPUT_JOB-NAME) into the canonical underscore form before invoking sendJobConclusionSpan — in contrast to action_setup_otlp.cjs:80-84, which explicitly does that normalization:

// action_setup_otlp.cjs (has the fix)
const inputJobName = getActionInput("JOB_NAME");
if (inputJobName) {
  process.env.INPUT_JOB_NAME = inputJobName;
}

When the runner exposes the input only under the hyphen form during the action's post phase, process.env.INPUT_JOB_NAME is empty inside sendJobConclusionSpan. As a result, the jobName === "agent" branch is never taken on conclusion in production, so:

1. The dedicated gh-aw.agent.agent span (the one that wraps pure LLM execution time) is never emitted.
2. The OTel GenAI semantic-convention attributes gated by if (jobName === "agent") at send_otlp_span.cjs:1787-1800 are silently dropped from gh-aw.agent.conclusion: gen_ai.operation.name, gen_ai.workflow.name, gen_ai.response.model, gen_ai.response.finish_reasons.
3. The gh-aw.job.name span attribute is missing on every conclusion span (activation, agent, detection, safe_outputs, conclusion).

Why This Matters (DevOps Perspective)

The gh-aw.agent.agent span is the only span that isolates LLM execution time from the broader agent job (AWF container boot, MCP proxy startup, workspace audit, post-processing). Without it, a DevOps engineer cannot answer:

• "How long did the LLM call actually take this run?" — gh-aw.agent.conclusion includes ~30s of pre/post-agent overhead, so any latency SLO built on it is noisy.
• "Is Anthropic/Copilot API latency degrading?" — today the agent.conclusion span mixes provider latency with workflow framework time, so a 20 % LLM slowdown is invisible until it dominates the total.
• "Why did this finish_reason occur?" — gen_ai.response.finish_reasons is meant to flag length-truncated runs in Grafana/Sentry dashboards, but it's currently missing on all spans.
• "Filter spans by job" — gh-aw.job.name is missing on every conclusion span, so dashboards must fall back to parsing span names with regex.

This also breaks out-of-the-box LLM dashboards (Grafana GenAI, Datadog LLM observability) that key on the OTel GenAI semantic-convention attributes — they show empty for gh-aw despite the code intending to emit them.

Current Behavior

actions/setup/js/action_conclusion_otlp.cjs:72-89:

async function run() {
  const endpoints = process.env.GH_AW_OTLP_ENDPOINTS;
  const spanName = buildSpanName(getActionInput("JOB_NAME"));
  // Use the job-start timestamp set by action_setup_otlp so the conclusion span
  // duration covers the actual job execution window, not just this step's overhead.
  const startMs = parseJobStartMs(process.env.GITHUB_AW_OTEL_JOB_START_MS);
  // ...
  await sendOtlpSpan.sendJobConclusionSpan(spanName, { startMs });
  // ...
}

getActionInput("JOB_NAME") correctly resolves the value from either form (so the span name is right: gh-aw.agent.conclusion), but the env var is never written back to the canonical INPUT_JOB_NAME. Then in actions/setup/js/send_otlp_span.cjs:1666:

const jobName = process.env.INPUT_JOB_NAME || "";
// ...
if (jobName) attributes.push(buildAttr("gh-aw.job.name", jobName));
// ...
if (jobName === "agent") {
  attributes.push(buildAttr("gen_ai.operation.name", "chat"));
  // ... (other GenAI semantic attributes)
}
// ...
const hasDedicatedAgentSpan = jobName === "agent" && /* ... */;

jobName ends up empty, so the gated attributes and the dedicated agent span are skipped.

Proposed Change

Mirror the normalization that already exists in action_setup_otlp.cjs so process.env.INPUT_JOB_NAME is canonical before sendJobConclusionSpan reads it:

// actions/setup/js/action_conclusion_otlp.cjs
async function run() {
  const endpoints = process.env.GH_AW_OTLP_ENDPOINTS;
  const inputJobName = getActionInput("JOB_NAME");
  // Normalize to the canonical underscore form so sendJobConclusionSpan
  // (which reads process.env.INPUT_JOB_NAME) always finds the value, even
  // when the runner only exposes the INPUT_JOB-NAME hyphen form in the post step.
  if (inputJobName) {
    process.env.INPUT_JOB_NAME = inputJobName;
  }
  const spanName = buildSpanName(inputJobName);
  const startMs = parseJobStartMs(process.env.GITHUB_AW_OTEL_JOB_START_MS);
  // ... rest unchanged
  await sendOtlpSpan.sendJobConclusionSpan(spanName, { startMs });
}

No changes required in send_otlp_span.cjs itself.

Expected Outcome

After this change:

• In Grafana Tempo / Honeycomb / Datadog: { name = "gh-aw.agent.agent" } returns real spans whose duration excludes setup/cleanup overhead, enabling LLM-only latency SLOs.
• gen_ai.operation.name, gen_ai.workflow.name, gen_ai.response.model, and gen_ai.response.finish_reasons populate on agent spans → out-of-the-box GenAI dashboards in Grafana/Datadog/Honeycomb start working without custom mappings.
• gh-aw.job.name appears on every conclusion span, so dashboards can group by gh-aw.job.name instead of regex-parsing span names.
• In the JSONL mirror: the same enrichments appear, so artifact-based debugging matches what's in the collector.
• For on-call engineers: a length-truncated agent run (e.g. gen_ai.response.finish_reasons = ["length"]) becomes discoverable in dashboards; today it's silently lost.

Implementation Steps

• Edit actions/setup/js/action_conclusion_otlp.cjs: add the inputJobName normalization shown above before computing spanName.
• Add a test case in actions/setup/js/action_otlp.test.cjs mirroring the existing setup test at line 185 ("uses job name from INPUT_JOB-NAME (hyphen form) in setup span"): assert that sendJobConclusionSpan sees jobName === "agent" when only INPUT_JOB-NAME is set, so gh-aw.agent.agent is emitted and the GenAI attributes are present.
• Run cd actions/setup/js && npx vitest run to confirm tests pass (including the existing tests at lines 273–290 for the underscore-form happy path).
• Run make fmt.
• Open a PR referencing this issue.

Evidence from Live OTel Data (Sentry/Grafana)

Grafana Tempo — span name enumeration (last 24 h, datasource grafanacloud-traces)

tempo_get-attribute-values name="name" → [
  "gh-aw.activation.conclusion", "gh-aw.activation.setup",
  "gh-aw.agent.conclusion", "gh-aw.agent.setup",
  "gh-aw.conclusion.conclusion", "gh-aw.conclusion.setup",
  "gh-aw.detection.conclusion", "gh-aw.detection.setup",
  "gh-aw.pre_activation.conclusion", "gh-aw.pre_activation.setup",
  "gh-aw.safe_outputs.conclusion", "gh-aw.safe_outputs.setup",
  "gh-aw.unlock.conclusion", "gh-aw.unlock.setup"
]

gh-aw.agent.agent is absent — zero occurrences across all gh-aw workflows in 24 h, despite jobName === "agent" and the agent span emission logic existing in code. A focused {name="gh-aw.agent.agent"} TraceQL query also returned 0 traces (inspectedBytes: 1.96 MB, 95/95 jobs completed).

Trace 8f9e94eb07269ea215125b1868b62d37 (workflow aw-portfolio-yield, run 26219695393, failed agent run):

• gh-aw.agent.setup attributes include gh-aw.job.name: "agent" — setup span correctly carries the attribute because action_setup_otlp.cjs already normalizes.
• gh-aw.agent.conclusion attributes do not include gh-aw.job.name, nor any of gen_ai.operation.name, gen_ai.workflow.name, gen_ai.response.model, gen_ai.response.finish_reasons — confirming jobName === "" at conclusion time.
• The span name gh-aw.agent.conclusion is correct, which proves getActionInput("JOB_NAME") resolves the value via the hyphen-form fallback — narrowing the bug to the missing process.env writeback in action_conclusion_otlp.cjs.

Trace a4879c57add4415850fc4a7ad5de5c71 (current advisor run) shows the same pattern: setup spans carry gh-aw.job.name, conclusion spans do not.

Existing setup-side regression test at actions/setup/js/action_otlp.test.cjs:185 covers the hyphen-form case for setup spans but has no equivalent for conclusion — which is why the bug was not caught.

Related Files

• actions/setup/js/action_conclusion_otlp.cjs — the file to edit
• actions/setup/js/action_setup_otlp.cjs — already contains the analogous fix to mirror
• actions/setup/js/send_otlp_span.cjs — reads process.env.INPUT_JOB_NAME at line 1666 and gates the dedicated agent span / GenAI attributes on it
• actions/setup/js/action_input_utils.cjs — defines getActionInput, which handles both env-var forms
• actions/setup/js/action_otlp.test.cjs — add the hyphen-form regression test next to line 185

---

Generated by the Daily OTel Instrumentation Advisor workflow

Generated by 📊 Daily OTel Instrumentation Advisor · ● 20.7M · ◷

• expires on May 28, 2026, 10:34 AM UTC