Add monitor ingestion upload and validate tools by msalaman · Pull Request #3954 · microsoft/mcp

msalaman · 2026-10-14T20:44:55Z

What does this PR do?

Adds two tools to the Monitor toolset (combined in one PR at the requester's request):

monitor ingestion upload (azmcp-monitor-ingestion-upload): uploads a JSON array of log records via the Logs Ingestion API (--ingestion-endpoint, --data-collection-rule immutable ID, --stream, --log-data). Returns status and record count.
monitor ingestion validate (azmcp-monitor-ingestion-data-validate): checks payload shape (JSON array of objects, ≤1 MB) and, using the stream declared in the DCR (looked up via ARM by name), column types. Returns errors/warnings as JSON.

Notes / design decisions for review:

Added a --stream option (required by the Logs Ingestion API; optional on validate when the DCR declares exactly one stream).
--data-collection-rule is the immutable ID for upload but the resource name (with --resource-group/--subscription) for validate; documented in option descriptions.
User-supplied ingestion endpoint is validated through IAzureService.ValidateAzureServiceEndpoint with a new monitor-ingestion allow-list (*.ingest.monitor.azure.{com,cn,us}), with tests.
Single request upload, no batching/gzip.

GitHub issue number?

Resolves #288
Resolves #295

Validation

dotnet test in tools/Azure.Mcp.Tools.Monitor/tests/Azure.Mcp.Tools.Monitor.Tests: passed (496 succeeded, 0 failed)
dotnet test in core/Microsoft.Mcp.Core/tests/Microsoft.Mcp.Core.Tests: passed (833 succeeded, 0 failed)

Pre-merge Checklist

Read contribution guidelines
Added unit tests
Changelog entry created
One tool per PR (this PR intentionally adds two tools)
Updated consolidated-tools.json (validate added; upload is not read-only)
Updated azmcp-commands.md and e2eTestPrompts.md
ToolDescriptionEvaluator, Update-AzCommandsMetadata.ps1, README

Invoking Livetests

Copilot submitted PRs are not trustworthy by default. Users with write access to the repo need to validate the contents of this PR before leaving a comment with the text /azp run mcp - pullrequest - live. This will trigger the necessary livetest workflows to complete required validation.

azure-pipelines · 2026-10-14T20:45:19Z

Azure Pipelines: Successfully started running 1 pipeline(s). There may be pipelines that require an authorized user to comment /azp run to run.

Copilot

🟡 Changes recommended

Missing upload routing and false-positive stream validation remain unresolved, alongside required test coverage and evaluation evidence.

4 open findings

What changed in this PR

Adds log upload and pre-ingestion validation to the Azure Monitor toolset, addressing #288 and #295.

Changes:

Adds ingestion commands, service integration, and structured results.
Adds payload/schema validation and cloud-specific endpoint allow-lists.
Adds unit tests, command documentation, prompts, and a changelog entry.

File	Description
tools/Azure.Mcp.Tools.Monitor/tests/Azure.Mcp.Tools.Monitor.Tests/Ingestion/IngestionValidateCommandTests.cs	Tests validation command behavior.
tools/Azure.Mcp.Tools.Monitor/tests/Azure.Mcp.Tools.Monitor.Tests/Ingestion/IngestionUploadCommandTests.cs	Tests upload command behavior.
tools/Azure.Mcp.Tools.Monitor/tests/Azure.Mcp.Tools.Monitor.Tests/Ingestion/IngestionLogDataValidatorTests.cs	Tests payload and schema checks.
tools/Azure.Mcp.Tools.Monitor/src/Validation/IngestionLogDataValidator.cs	Implements payload and column validation.
tools/Azure.Mcp.Tools.Monitor/src/Services/MonitorIngestionService.cs	Implements upload and DCR lookup.
tools/Azure.Mcp.Tools.Monitor/src/Services/IMonitorIngestionService.cs	Defines ingestion service contracts.
tools/Azure.Mcp.Tools.Monitor/src/Options/Ingestion/IngestionValidateOptions.cs	Defines validation inputs.
tools/Azure.Mcp.Tools.Monitor/src/Options/Ingestion/IngestionUploadOptions.cs	Defines upload inputs.
tools/Azure.Mcp.Tools.Monitor/src/MonitorSetup.cs	Registers both commands and service.
tools/Azure.Mcp.Tools.Monitor/src/Models/Ingestion/IngestionValidationResult.cs	Models validation results.
tools/Azure.Mcp.Tools.Monitor/src/Models/Ingestion/IngestionValidationIssue.cs	Models individual validation issues.
tools/Azure.Mcp.Tools.Monitor/src/Models/Ingestion/IngestionUploadResult.cs	Models upload status and count.
tools/Azure.Mcp.Tools.Monitor/src/Commands/MonitorJsonContext.cs	Registers result serialization.
tools/Azure.Mcp.Tools.Monitor/src/Commands/Ingestion/IngestionValidateCommand.cs	Exposes schema validation.
tools/Azure.Mcp.Tools.Monitor/src/Commands/Ingestion/IngestionUploadCommand.cs	Exposes log upload.
tools/Azure.Mcp.Tools.Monitor/src/Azure.Mcp.Tools.Monitor.csproj	Adds ingestion SDK dependency.
servers/Azure.Mcp.Server/src/Resources/consolidated-tools.json	Maps validation into consolidated discovery.
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md	Adds prompts for both tools.
servers/Azure.Mcp.Server/docs/azmcp-commands.md	Documents commands and parameters.
servers/Azure.Mcp.Server/changelog-entries/monitor-ingestion-upload-validate.yaml	Announces ingestion functionality.
core/Microsoft.Mcp.Core/tests/Microsoft.Mcp.Core.Tests/Helpers/EndpointValidatorTests.cs	Tests ingestion endpoint allow-lists.
core/Microsoft.Mcp.Core/src/Helpers/EndpointValidator.AllowLists.cs	Adds cloud-specific ingestion suffixes.

🧠 Review effort: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Alan Zimmer (alzimmermsft) · 2026-10-14T21:24:16Z

+        catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
+        {
+            throw;
+        }


Hmm, this is different than our normal design, could you add why. Is this something we'd want to do in other locations / generally?

Good point, looking over this special catch is necessary I don't think this is necessary, removed

Alan Zimmer (alzimmermsft) · 2026-10-14T21:25:25Z

+    ILogger<IngestionValidateCommand> logger,
+    IMonitorIngestionService ingestionService,
+    ISubscriptionResolver subscriptionResolver)
+    : SubscriptionCommand<IngestionValidateOptions, IngestionValidationResult>(subscriptionResolver)


Why does validate need a subscription when upload doesn't? Is it possible to validate against the specific ingestion endpoint?

Validate reads the stream schema from the data collection rule via ARM, which needs the subscription; a data collection endpoint doesn't expose schemas, so it can't validate against the endpoint alone. Upload only needs the endpoint and rule ID.

Alan Zimmer (alzimmermsft) · 2026-10-14T21:31:16Z

+        var shape = IngestionLogDataValidator.Validate(options.LogData, schema: null);
+        foreach (var error in shape.Errors)
+        {
+            validationResult.AddError(error.Message, "Invalid ingestion log data.");


Well, you've found an edge case I didn't think of when I added telemetry safe error information, if there are many errors here you'll have a lot of Invalid ingestion log data. in the telemetry. Maybe copy the design in the service implementation where we space concatenate the error messages and have a single call to add error.

Done: errors are space-concatenated into a single AddError call.

Alan Zimmer (alzimmermsft) · 2026-10-14T21:33:32Z

+            response.IsError ? "Failed" : "Succeeded",
+            dataCollectionRule,
+            stream,
+            shape.RecordCount);


If the response is an error, are any records uploaded? If the upload is an all or nothing batch this should be 0 on error, if partial batches are possible this number will also be wrong.

The upload is a single all-or-nothing request, so RecordCount is now 0 on error.

Alan Zimmer (alzimmermsft) · 2026-10-14T21:35:29Z

+        }
+        else
+        {
+            warnings.Add(new(null, null, $"Stream '{selectedStream}' has no declared schema in the data collection rule, so only payload shape was checked."));


This seems like it should be a thrown exception similar to when there are zero or many declarations when --stream isn't passed.

Done: an undeclared stream now throws CommandValidationException.

Alan Zimmer (alzimmermsft) · 2026-10-14T21:36:45Z

+            || !string.IsNullOrEmpty(endpoint.Query)
+            || !string.IsNullOrEmpty(endpoint.Fragment))
+        {
+            throw new CommandValidationException("--ingestion-endpoint must be an HTTPS data collection endpoint root URL without user information, path, query, or fragment.")


This message is telemetry safe already.

Removed the redundant TelemetrySafeMessage.

Alan Zimmer (alzimmermsft) · 2026-10-14T21:37:44Z

+        {
+            AzureCloudConfiguration.AzureCloud.AzureChinaCloud => LogsIngestionAudience.AzureChina,
+            AzureCloudConfiguration.AzureCloud.AzureUSGovernmentCloud => LogsIngestionAudience.AzureGovernment,
+            _ => LogsIngestionAudience.AzurePublicCloud


I think we should always throw on an unhandled cloud type rather than falling back to the public cloud. This could cause hard to debug issues in the future if we add support for more clouds.

Done: unsupported cloud types now throw NotSupportedException instead of falling back to public.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83d3236e-c51b-4809-861f-a45a9a63ae7c

msalaman requested review from a team, Jackson Weber (JacksonWeber), Leighton Chen (lzchen), Radhika Gupta (rads-1996), Rajkumar Rangaraj (rajkumar-rangaraj) and xiang17 as code owners October 14, 2026 20:44

msalaman requested review from JasonYeMSFT (JasonYeMSFT), KarishmaGhiya, Rick Winter (RickWinter), Alan Zimmer (alzimmermsft), Connie Yau (conniey), Steven Vukelich (vukelich) and Jeff Fisher (xirzec) and a balanced review from Copilot October 14, 2026 20:44

github-actions Bot added server-Azure.Mcp Azure.Mcp.Server tools-Monitor labels Oct 14, 2026

Copilot started reviewing on behalf of msalaman October 14, 2026 20:45

View session

Copilot AI reviewed Oct 14, 2026

View reviewed changes

Alan Zimmer (alzimmermsft) reviewed Oct 14, 2026

View reviewed changes

Add monitor ingestion upload and validate tools

f62bfa4

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83d3236e-c51b-4809-861f-a45a9a63ae7c

msalaman force-pushed the msalaman-monitor-ingestion-upload-validate branch from 6802587 to f62bfa4 Compare October 16, 2026 21:56

msalaman and others added 7 commits October 16, 2026 17:02

Address review feedback on monitor ingestion tools

72da196

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83d3236e-c51b-4809-861f-a45a9a63ae7c

Use standard exception handling in ingestion commands

0b3903b

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 83d3236e-c51b-4809-861f-a45a9a63ae7c

fixed merge conflicts

ca6cc99

added recorded tests

b276f09

added recorded tests

5a9448c

fix formatting

99f19fa

fixed recorded test assets

03fc766

Saswata Pandit (psaswata) mentioned this pull request Oct 21, 2026

Add Azure Resiliency Agent tools #3845

Open

14 tasks done

msalaman added 2 commits October 21, 2026 15:59

Merge branch 'main' into msalaman-monitor-ingestion-upload-validate

8e7c37d

update readme

236bc3b

Conversation

msalaman commented Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

GitHub issue number?

Validation

Pre-merge Checklist

Invoking Livetests

Uh oh!

azure-pipelines Bot commented Oct 14, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msalaman Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

msalaman commented Oct 14, 2026 •

edited

Loading

msalaman Oct 16, 2026 •

edited

Loading