Skip to content

telemetry [13]: introduce command error reporting helpers - #778

Open
bfjelds (bfjelds) wants to merge 1 commit into
user/bfjelds/mjolnir/telemetry-tracestream-hardeningfrom
user/bfjelds/mjolnir/command-error-metric
Open

bfjelds (bfjelds) wants to merge 1 commit into
user/bfjelds/mjolnir/telemetry-tracestream-hardeningfrom
user/bfjelds/mjolnir/command-error-metric

Conversation

@bfjelds

@bfjelds bfjelds (bfjelds) commented Sep 4, 2026 •

Copy link
Copy Markdown
Member

Description

Add correlated command_error helpers with structured errors, panic handling, deduplication, and captured reboot context.

Rationale

Extracted from #778 to make the telemetry stack easier to review. Combined changes are unchanged.

Validation

Source Command or check Result Notes
Agent cargo test --locked --manifest-path /home/bfjelds/mjolnir-sandbox/trident-telemetry-split778-20261025-1743/Cargo.toml -p trident -p osutils -p trident_api --features trident/grpc-preview,trident/dangerous-options,trident_api/documentation -- command_kind:: logging:: cli:: engine::context:: config::host::tests virt:: Passed 115 targeted unit tests; run from /home/bfjelds/mjolnir-sandbox.
Agent Git tree comparison Passed Final split tip exactly matches original #778; restacked #779/#781 trees unchanged.
Agent VM functional tests Not run Require a VM; only host-safe unit tests were run.
CI Pipeline checks Not run Not evaluated as part of this split; results above are local.

Related PRs in stack:

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
1 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@bfjelds bfjelds (bfjelds) changed the title telemetry: add command_error metric (kind/subkind/location); move manual_rollback/runtime_update metrics here telemetry: additional metrics (error, consistent start/success) Sep 4, 2026
@bfjelds
bfjelds (bfjelds) requested a balanced review from Copilot September 5, 2026 01:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

CLI failure telemetry remains incomplete, and ACL detection failures are incorrectly reported as negative results.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds expanded telemetry for command failures, servicing lifecycle events, and startup context.

Changes:

  • Adds correlated command_error metrics for CLI and gRPC operations.
  • Adds manual rollback and runtime-update lifecycle metrics.
  • Enriches startup telemetry with ACL detection and system uptime.
File summaries
File Description
Cargo.toml Enables Nix time APIs.
docs/Reference/Agent-Configuration.md Documents command-error telemetry.
crates/trident/src/main.rs Instruments CLI servicing commands.
crates/trident/src/lib.rs Re-exports instrumentation and enriches startup metrics.
crates/trident/src/logging/operation_context.rs Implements and tests command-error reporting.
crates/trident/src/server/tridentserver/mod.rs Instruments gRPC servicing commands.
crates/trident/src/engine/runtime_update.rs Emits runtime-update success.
crates/trident/src/engine/manual_rollback/mod.rs Emits rollback start and runtime success.
crates/trident/src/engine/manual_rollback/utils.rs Makes rollback request kinds loggable.
crates/trident_api/src/error.rs Exposes error-origin locations.
Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread crates/trident/src/lib.rs Outdated
Comment thread crates/trident/src/main.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

CLI startup events lose correlation metadata, and runtime-update success is omitted from persisted metrics archives.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread crates/trident/src/engine/runtime_update.rs Outdated
Comment thread crates/trident/src/main.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Correlation-ID pre-warming creates missing datastores and bypasses the host-provisioning guard for several CLI commands.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread crates/trident/src/main.rs Outdated
@bfjelds
bfjelds (bfjelds) force-pushed the user/bfjelds/mjolnir/command-error-metric branch 2 times, most recently from f6f0da6 to 5e7b215 Compare September 5, 2026 19:51
@bfjelds
bfjelds (bfjelds) requested a balanced review from Copilot September 5, 2026 20:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Initial install/update telemetry can remain uncorrelated, and runtime rollback persistence is duplicated.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

crates/trident/src/main.rs:202

  • Issue: The missing-datastore check also skips pre-warming for install and update, so the first such CLI invocation still emits command_start without a correlation_id. Evidence: Those commands are intentionally exempt from the provisioning guard below, and Trident::new creates the ID only after run_command has already emitted command_start. Suggestion: Skip creation only for commands that require an existing datastore; allow install/update to create and attach the ID before entering run_command.
                    if !agent_config.datastore_path().exists() {
                        return Ok(None);
  • Files reviewed: 10/10 changed files
  • Comments generated: 2
  • Review effort level: Balanced

Comment thread crates/trident/src/engine/manual_rollback/mod.rs Outdated
Comment thread crates/trident/src/server/tridentserver/mod.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Several non-servicing CLI failures still bypass the promised command_error telemetry.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread crates/trident/src/main.rs Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Read-only and response-encoded gRPC failures remain outside the new command-error instrumentation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread crates/trident/src/server/tridentserver/mod.rs Outdated
@bfjelds
bfjelds (bfjelds) force-pushed the user/bfjelds/mjolnir/command-error-metric branch from 79b5d01 to ee0bf7c Compare September 5, 2026 21:29
@bfjelds
bfjelds (bfjelds) requested a balanced review from Copilot September 5, 2026 21:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The scoped helper additions have no identified blocking issues in error propagation, panic reporting, context cleanup, or public exports.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The scoped helper changes have no identified blocking correctness or compatibility issues; reported tests were not independently rerun.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused helper changes preserve return and unwind behavior, with no unresolved correctness or compatibility findings.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No blocking issues remain in the scoped reporting helpers, context handling, or error-location accessor.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The scoped helper changes preserve error propagation and context cleanup, with no unresolved blocking findings.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved issues were identified in the scoped helper changes, public accessor, or accompanying tests.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Panic propagation and thread-local deduplication need human validation because independent runtime verification remains incomplete.

1 open issue

Review effort: Balanced

Comment thread crates/trident/src/logging/operation_context.rs

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The additive helper changes preserve existing result and panic behavior, with no unresolved blocking issues identified.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are blocking the command-error reporting changes.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the reusable command-error reporting helpers.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the command-error reporting helpers.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the shared command-error reporting changes.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the shared command-error reporting changes.

No open issues 🎉

Review effort: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: d0e7cadc-8f44-443f-affe-6ce8e856a218

Copilot-Session: 43cb811b-f191-4d90-852f-e3373041ca11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The shared command-error reporting changes have no active review issues.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are identified for the shared command-error reporting changes.

No open issues 🎉

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants