Skip to content

(WIP) feat: subagent & workflow observability — Agents panel#4220

Open
t3dotgg wants to merge 17 commits into
mainfrom
t3code/subagent-workflow-sidebar
Open

(WIP) feat: subagent & workflow observability — Agents panel#4220
t3dotgg wants to merge 17 commits into
mainfrom
t3code/subagent-workflow-sidebar

Conversation

@t3dotgg

@t3dotgg t3dotgg commented Jul 21, 2026

Copy link
Copy Markdown
Member

Subagent & workflow observability: Agents panel

Live visibility into delegated work — Claude Code subagents and workflows, and Codex v2 collab agents — directly in the thread UI: who is running, what they're doing right now, token burn, elapsed time, and result summaries.

Plans (reviewed by Fable + gpt-5.6-sol, rev 3): docs/project/plans/subagent-observability-spec.html · docs/project/plans/subagent-observability-ui.html

Design in one paragraph

No new pipeline. Adapters normalize provider signals into the existing task.* runtime-event family (extended with typed usage and agent-linkage fields); an in-memory reducer in ProviderRuntimeIngestion folds them into one ThreadAgentSnapshot per agent and, on material transitions only, appends the full roster as a single agent.snapshot activity through the existing thread.activity.append command. The client derives panel state latest-wins from the newest such activity — the same pattern the context-window meter already uses. No new WS event types, subscriptions, projection tables, or capability flags; old clients decode the new activity kind as a benign info row.

What's included

  • contracts: ThreadAgentSnapshot (identity + activations model: Codex agents are resumable, so idlecompleted; activationCount, firstStartedAt/lastActivityAt), agent.snapshot payload schema, task.updated runtime event, typed RuntimeTaskUsage, agent-linkage fields (incl. workflow phases, scriptPath, runId, outputFile) with a timelineBypass marker.
  • ClaudeAdapter: carries previously-dropped fields (subagent_type, tool_use_id, workflow_name, prompt, output_file); handles task_updated (was: unknown-subtype warning); schema-guarded parse of the undeclared workflow_progress into per-workflow-agent child task events (label, phase, model, state, tokens, attempt); captures WorkflowOutput.scriptPath/runId from tool results.
  • CodexSessionRuntime/CodexAdapter (probe-gated slice): learns v2 collab children from subAgentActivity items (a live wire probe on codex-cli 0.144.1 confirmed children emit no thread/started; nicknames come from agentPath), diverts their auto-subscribed notifications into synthetic collab/agentActivity events, and maps them to timeline-bypassing task.* events. turn/completedidle (resumable), waitingOnApprovalwaiting, per-child token breakdowns from thread/tokenUsage/updated.
  • Ingestion reducer: material-transition filter (status/name/model/phase/activity changes, 25k-token usage steps — 1s ticks stay in memory), hydration from the latest persisted snapshot on restart, orphan sweep to stopped on session exit, 50-cap settled retention, dispatch counter.
  • Web: new agents right-panel surface (tab, "+" menu, empty-state card) with grouped cards — workflow phase headers (✓ done / running counts / pending), status dots in the existing pill color language, self-ticking elapsed timers (no React commits), token counters (deliberately not bars — burn has no denominator), expandable recent-activity feed, "{} script" affordance; plus a collapsed live strip above the composer that opens the panel. Shared derivation helpers live in client-runtime for mobile reuse.

Verification

  • Focused vitest: 152 tests across ClaudeAdapter, ProviderRuntimeIngestion (reducer: material filtering, latest-wins roster, orphan sweep, activation counts, retention), CodexAdapter (collab mapping), CodexSessionRuntime — all passing.
  • tsgo --noEmit clean for contracts, client-runtime, server, web; vp lint clean on changed files.
  • Integrated web verification: isolated vp run dev environment, paired browser, real Claude subagent spawn — panel, live strip, settled card, and empty state confirmed (screenshots in PR comments if useful).
  • Codex v2 protocol behavior established by a live app-server wire probe (/tmp/codex-v2-probe/report.md) rather than source-reading alone.

Deliberately out of scope (per plan)

Per-agent stop/steer (needs provider command plumbing), full transcript drill-in RPC (phase 2; outputFile is already captured), ACP providers (no wire signal), cost USD attribution, mobile surfaces (derivation helpers are shared and ready).

🤖 Generated with Claude Code


Note

Medium Risk
Touches core provider runtime ingestion and transcript projection (timeline bypass, revert cache invalidation); behavior is heavily tested but wrong roster or hydration logic could misrepresent agent state across restarts and session boundaries.

Overview
Adds live subagent and workflow observability by folding provider task.* signals into per-thread agent rosters and persisting them as agent.snapshot thread activities (latest-wins by revision), without new wire types or projections.

Server: New contracts (ThreadAgentSnapshot, task.updated, typed RuntimeTaskUsage, timelineBypass). ClaudeAdapter maps task_updated, workflow progress, and normalized usage; Codex diverts v2 collab child notifications to collab/agentActivity → bypassing task.* events. ProviderRuntimeIngestion runs foldTaskAgentEvent / pruneSettledAgents, appends snapshots on material changes, hydrates from persisted activities on restart, orphans agents on session exit, and clears roster cache on checkpoint revert. task.updated and bypass events no longer create timeline rows; work logs skip agent.snapshot.

Client: Shared deriveLatestAgentSnapshot / panel grouping in client-runtime. Web adds an Agents right-panel tab, AgentsPanel (phases, timers, tokens, activity feed), and AgentsLiveStrip above the composer when agents are live.

Reviewed by Cursor Bugbot for commit a40376b. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Add Agents panel with live subagent and workflow observability to the chat right panel

  • Introduces an agent.snapshot activity kind that carries a roster of ThreadAgentSnapshot records, persisted by the ingestion layer and decoded client-side via deriveLatestAgentSnapshot in threadAgents.ts.
  • Adds AgentsPanel.tsx with grouped workflow/phase/agent cards, live elapsed timers, usage counters, expandable activity feeds, and a footer summary; accessible as a new agents right panel surface.
  • Adds AgentsLiveStrip.tsx, a one-line strip shown above the composer when agents are pending/running/waiting, with a button to open the Agents panel.
  • The Claude adapter (ClaudeAdapter.ts) and Codex adapter (CodexAdapter.ts) now emit structured task.* events with agent linkage, normalized usage, and workflow phase data; child collab threads are synthesized as collab/agentActivity notifications and kept off the parent timeline.
  • ProviderRuntimeIngestion.ts folds task events into per-thread agent rosters via foldTaskAgentEvent, appends agent.snapshot activities on material changes or activity pressure, marks non-terminal agents stopped on session exit, and rehydrates from the latest persisted snapshot across restarts.
  • Behavioral Change: agent.snapshot activities and task.updated events are suppressed from the normal thread timeline; RIGHT_PANEL_STORAGE_VERSION is bumped from 7 to 8, clearing any persisted panel layout.

Macroscope summarized a40376b.

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: cfd75ce4-617a-4fba-8d04-2b30ff0fb5e5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch t3code/subagent-workflow-sidebar

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. label Jul 21, 2026
@t3dotgg

t3dotgg commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

Integrated verification note: in the browser run (isolated dev env, real Claude subagent), the settled card briefly showed Σ 0 tok — usage for very short agents arrives only on the terminal task_notification, and the final roster snapshot (verified in SQLite: totalTokens 22798) updates the panel when it lands. Screenshots live at /tmp/codex-computer-use.lvQ4hP/ locally; FILE_HOST_TOKEN was unset in this session so they aren't embedded here.

@github-actions github-actions Bot added the size:XXL 1,000+ changed lines (additions + deletions). label Jul 21, 2026
Comment thread packages/client-runtime/src/state/threadAgents.ts
Comment thread apps/server/src/provider/Layers/CodexSessionRuntime.ts
Comment thread apps/web/src/components/AgentsPanel.tsx Outdated
Comment thread apps/server/src/provider/Layers/CodexSessionRuntime.ts
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
Comment thread apps/web/src/components/AgentsPanel.tsx
Comment thread apps/server/src/provider/Layers/CodexAdapter.ts Outdated
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts Outdated
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
Comment thread packages/contracts/src/threadAgents.ts
Comment thread apps/server/src/provider/Layers/CodexAdapter.ts
@macroscopeapp

macroscopeapp Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Needs human review

This WIP PR introduces a substantial new feature (Agents panel for subagent/workflow observability) with significant new server-side logic, client state management, and UI components. Additionally, there's an unresolved high-severity bug identified in the activity change detection logic.

You can customize Macroscope's approvability policy. Learn more.

Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts Outdated
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
@t3dotgg t3dotgg changed the title feat: subagent & workflow observability — Agents panel (WIP) feat: subagent & workflow observability — Agents panel Jul 21, 2026
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts Outdated
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts Outdated
@t3dotgg

t3dotgg commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

Addressed the review handoff (all four P1s + six P2s) in 0dd586c:

P1

  • Workflow agent identity: keyed on stable row index (<workflowTaskId>:wf:<index>) instead of per-attempt agentId — queued/blocked/pre-spawn-error rows now surface, retries update one card.
  • Snapshot ordering: agent.snapshot payloads carry a monotonic per-thread revision (seeded from the hydrated snapshot); the client selects the highest revision instead of trusting list position.
  • Codex turn status: turn/completed now decodes turn.status — failed→failed (with error message), interrupted→stopped, completed→idle.
  • Hydration cost: roster hydration is lazy — only agent-touching events (task.* / session.exited) trigger it, so message deltas never cause a full thread-detail load.

P2: reducer re-derives kind from event data (order-independent), reactivation counts any settled→active transition, workflow containers excluded from worker counts / usage not double-counted, settled cards lead with error/result over stale activity, newest agents array is authoritative on decode failure (no stale-roster resurrection), idle agents exempt from the terminal retention pool, Codex usage consistently uses the cumulative total breakdown.

160 focused tests passing (new regression coverage for the identity, revision, decode-authority, and retention behaviors); typecheck clean across contracts/client-runtime/server/web.

Comment thread apps/server/src/provider/Layers/ClaudeAdapter.ts
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts Outdated
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts Outdated
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts Outdated
Comment thread apps/server/src/provider/Layers/ClaudeAdapter.test.ts Outdated
t3dotgg and others added 12 commits July 21, 2026 19:59
…lity

ThreadAgentSnapshot + agent.snapshot activity payload, typed task usage,
task.updated runtime event, and agent-linkage fields on task payloads.
Includes the reviewed HTML plans.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New right-panel 'agents' surface (Option A) + collapsed live strip near the
composer (Option B). Roster derived latest-wins from agent.snapshot
activities via shared client-runtime helpers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CodexSessionRuntime learns children from subAgentActivity items (probe:
no child thread/started arrives) and diverts their notifications into
synthetic collab/agentActivity events; CodexAdapter maps those to
timeline-bypassing task.* events (turn lifecycle, waiting flags, token
usage, item summaries). turn/completed maps to idle, not terminal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…reducer

ClaudeAdapter carries subagent linkage (agentType/toolUseId/workflowName/
prompt/outputFile), handles task_updated, parses workflow_progress into
child workflow_agent task events, normalizes task usage, and captures
Workflow tool results (scriptPath/runId). Ingestion folds task.* events
into per-thread ThreadAgentSnapshot rosters with material-transition
filtering, projection hydration, orphan sweep, and 50-cap retention,
appended as agent.snapshot activities.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… payload

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…bile work log

Codex review P2: active-with-no-flags thread status now maps back to
running so agents don't stay 'waiting' after an approval resolves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- CI: client-runtime test imports @effect/vitest (vitest types unresolved
  in package context)
- CodexSessionRuntime: divert child-thread events that arrive before
  subAgentActivity registers them (probe: child status/changed lands
  first); emit a terminal event on subAgent interruption
- CodexAdapter: systemError -> failed; cumulative totalProcessedTokens
  for child usage (last-turn count can shrink on follow-ups)
- Ingestion: waiting agents counted as active in roster summaries;
  reactivation clears stale resultSummary/errorMessage; usage fields
  validated as non-negative integers; failed snapshot dispatch drops the
  hydration marker so the roster re-syncs from persisted state
- Panel/strip: idle counts as settled everywhere; phase labels show
  active counts (not '0 running'); parseTimestampDate for ISO parsing
- Drop docs/project/plans HTML files

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- session.exited now releases the per-thread reducer state after the
  final snapshot persists (unbounded retention leak)
- failed snapshot dispatch keeps the newer in-memory roster and flags
  the thread for re-dispatch on the next event instead of discarding
  memory (previous fix could lose transitions)
- document the 500-cap hydration boundary: past the cap both server and
  clients have lost the snapshot, so empty hydration is consistent

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… cap eviction

- A failed agent.snapshot append no longer aborts the rest of the
  event's ingestion (Bugbot: repeated failures were dropping message
  deltas/session status). Failures are logged, interrupts propagate, and
  the redispatch flag retries on the next event.
- While agents exist, the roster is re-published every ~400 timeline
  activities so the latest snapshot cannot age past the projector's
  500-row cap (Macroscope: hydration and client scans always find one).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pend success only on success

Bugbot round 4: after restart the latest agent.snapshot can already sit
deep in the capped projection, so the refresh counter now seeds from its
actual index distance instead of zero; the 'appended' debug log moved
into the success branch so a swallowed failure no longer logs success.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
P1:
- Claude workflow agents keyed by stable row index (parent:wf:N), not
  per-attempt agentId — queued/blocked rows now appear, retries update
  one card instead of duplicating
- agent.snapshot payloads carry a monotonic per-thread revision; client
  latest-wins selection uses it (deterministic under same-ms appends)
- Codex child turn/completed decodes turn.status: failed→failed with
  error summary, interrupted→stopped, completed→idle
- roster hydration is lazy (only on agent-touching events), no longer a
  full thread-detail load on every first event per thread

P2:
- reducer re-derives kind from event taskType/parent (no first-guess
  pinning); reactivation counts any settled→active transition
- workflow container rows excluded from worker counts; container usage
  only counted when it has no member rows
- settled cards lead with error/result, never stale activity text
- newest agents array is authoritative on the client — undecodable
  rosters no longer resurrect older running snapshots
- idle (resumable) agents no longer share the terminal retention pool
- Codex usage uses the cumulative total breakdown consistently

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
t3dotgg and others added 4 commits July 21, 2026 20:00
…ortized refresh, queued workflow agents, kind materiality

- Hydration selects the persisted snapshot by highest revision (same rule
  as the client), so the reducer can't resume from a stale roster that
  landed later in the list
- Ordinary traffic now counts toward unhydrated threads and triggers
  amortized hydration+refresh, so a settled snapshot can't age out of
  the 500-row cap after a restart with no agent events
- Claude workflow_agent state:'start' without startedAt (queued) maps to
  pending instead of running
- kind/parentAgentId upgrades count as material transitions

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A payload carrying only totalTokens (e.g. terminal task_notification)
no longer wipes previously known input/output/toolUses breakdowns.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The SDK stream emits this roster snapshot but its subtype is absent from
the typed union, so it fell through to the unknown-subtype warning path
and rendered as a spurious error row ('no displayable text content') in
client work logs. The task_* lifecycle events carry the authoritative
per-agent data; the typed background_tasks control request remains the
reconciliation source.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…4244

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@t3dotgg
t3dotgg force-pushed the t3code/subagent-workflow-sidebar branch from 37b0d73 to e9a498e Compare July 22, 2026 03:01
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
Comment thread apps/web/src/components/AgentsPanel.tsx Outdated
Comment thread packages/client-runtime/src/state/threadAgents.ts
Comment thread apps/web/src/components/AgentsPanel.tsx
Comment thread apps/server/src/orchestration/Layers/ProviderRuntimeIngestion.ts
Comment thread apps/web/src/components/ChatView.tsx
Bot rounds (Macroscope/Bugbot):
- taskKind: explicit shell/monitor types win over parentTaskId fallback
- lastStartedAt per activation; elapsed timers exclude idle gaps
- client revision selection uses (revision ?? -1) matching server hydration
- orphan-sweep append retries once on session.exited (often the last event)
- live-strip wrapper keyed off hasLiveAgents (no phantom spacer)

gpt-5.6-sol (high):
- thread.reverted now invalidates the cached roster (no resurrecting
  reverted agents); new domain-event subscription
- endedAt: explicit > existing > first-terminal ingestion time
- roster strings bounded at 180 chars (recentActivity + currentActivity)
- task_updated.patch.description maps to name
- empty failed workflow renders its container card

Fable:
- deriveLatestAgentSnapshot picks the winner by peeking revision, decodes
  only that roster (was: full schema-decode of every roster per recompute)
- idle Codex children get their own retention cap (25) — they never reach
  terminal status mid-session, so they were unbounded
- v1 collab receivers re-checked inside the v2 diversion branch (arrival-
  order race); task usage schema committed to typed RuntimeTaskUsage;
  Claude usage carries input/output/cache breakdowns; assorted nits

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a40376b. Configure here.

// Bounded: Codex child item summaries can carry full command/item
// text, and each entry is duplicated into every roster snapshot.
summary: boundAgentText(summary ?? lastToolName ?? "Activity updated"),
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Truncation breaks activity change detection

High Severity

The activityChanged check incorrectly compares the raw summary with the previously truncated currentActivity. This leads to false positives for activity changes when summaries are long, resulting in duplicate recentActivity entries and excessive agent.snapshot appends.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit a40376b. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🚧 In Progress size:XXL 1,000+ changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant