feat: Add provisioning provider support to extension framework#7482
Open
wbreza wants to merge 3 commits intoAzure:mainfrom
Open
feat: Add provisioning provider support to extension framework#7482wbreza wants to merge 3 commits intoAzure:mainfrom
wbreza wants to merge 3 commits intoAzure:mainfrom
Conversation
Introduces extensibility for custom provisioning providers (e.g., Pulumi, CDK, scripts) via the existing gRPC extension framework, following the same MessageBroker-based patterns used by service targets and framework services. New components: - provisioning.proto: bidirectional streaming RPC with 9 request/response pairs for Initialize, State, Deploy, Preview, Destroy, EnsureEnv, and Parameters operations with progress streaming support - ProvisioningEnvelope: MessageEnvelope implementation for broker integration - ProvisioningManager: extension-side manager with factory-based provider creation and handler dispatch - ProvisioningService: server-side gRPC handler with capability verification and IoC container registration - ExternalProvisioningProvider: core-side adapter implementing provisioning.Provider that delegates to extensions via the broker - ExtensionHost.WithProvisioningProvider(): fluent API for extensions to register custom provisioning providers Wiring changes: - Added ProvisioningProviderCapability to extension registry - Added to listenCapabilities in extension middleware - Registered ProvisioningService in IoC container and gRPC server - Extended AzdClient with Provisioning() accessor - Relaxed ParseProvider() to accept any provider kind string - Added Options.Config field for extension-specific configuration Resolves Azure#7465 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Review fixes: - Add empty provider name validation in registration - Fix swallowed structpb.NewStruct error in config conversion - Extract getProvider() helper to eliminate DRY violation in handlers - Add error context wrapping to all ExternalProvisioningProvider methods - Return empty slices instead of nil from PlannedOutputs/Parameters - Use direct indexing in convertFromProtoParameters - Consolidate Register() lock acquisitions into single scope Demo extension: - Add DemoProvisioningProvider to microsoft.azd.demo extension - Register as 'demo' provider with WithProvisioningProvider() - Add provisioning-provider to extension capabilities Test fixes: - Add ProvisioningProviderCapability to ValidCapabilities list - Update TestListenCapabilities assertion count - Add capability to validation test coverage Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
wbreza
requested review from
JeffreyCA,
hemarina,
jongio,
rajeshkamal5050,
tg-msft,
vhvb1989 and
weikanglim
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends the existing azd gRPC extension framework to support extension-provided infrastructure provisioning providers, enabling azd core to resolve custom providers (by name) from IoC and invoke them over a bidirectional brokered stream.
Changes:
- Adds a new
ProvisioningServicegRPC stream + message envelope/manager to register and service provisioning providers from extensions. - Introduces a core-side adapter (
ExternalProvisioningProvider) to bridgeprovisioning.Providercalls to the extension over the broker (including progress streaming). - Relaxes
provisioning.ParseProvider()validation and addsinfra.configpass-through support viaprovisioning.Options.Config.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| cli/azd/grpc/proto/provisioning.proto | Defines the provisioning bidi-stream protocol and shared message types. |
| cli/azd/pkg/azdext/provisioning_envelope.go | Implements MessageEnvelope operations for provisioning messages (including progress). |
| cli/azd/pkg/azdext/provisioning_manager.go | Extension-side manager that registers a provider and dispatches incoming provisioning requests to it. |
| cli/azd/pkg/azdext/provisioning.pb.go | Generated protobuf stubs for provisioning messages. |
| cli/azd/pkg/azdext/provisioning_grpc.pb.go | Generated gRPC client/server stubs for provisioning stream. |
| cli/azd/pkg/azdext/extension_host.go | Adds WithProvisioningProvider() and wires provisioning manager into the host lifecycle. |
| cli/azd/pkg/azdext/azd_client.go | Adds AzdClient.Provisioning() service accessor. |
| cli/azd/internal/grpcserver/provisioning_service.go | Server-side stream handler with capability check + IoC registration of external providers. |
| cli/azd/internal/grpcserver/external_provisioning_provider.go | Core adapter implementing provisioning.Provider over the broker stream. |
| cli/azd/internal/grpcserver/server.go (+ tests) | Registers the new provisioning gRPC service on the server. |
| cli/azd/cmd/container.go | Registers ProvisioningService in IoC. |
| cli/azd/cmd/middleware/extensions.go (+ coverage test) | Adds provisioning-provider to listen capabilities. |
| cli/azd/pkg/extensions/registry.go + validate_registry* | Adds/validates the new ProvisioningProviderCapability. |
| cli/azd/pkg/infra/provisioning/provisioning.go (+ test) | Relaxes provider parsing to accept extension-defined kinds. |
| cli/azd/pkg/infra/provisioning/provider.go | Adds Options.Config map[string]any for extension-specific config. |
| cli/azd/extensions/microsoft.azd.demo/* | Demonstrates registering and implementing a provisioning provider in the demo extension. |
Comment on lines
53
to
54
| // Parses the specified IaC Provider to ensure whether it is valid or not | ||
| // Defaults to `Bicep` if no provider is specified |
There was a problem hiding this comment.
The doc comment for ParseProvider says it defaults to Bicep when no provider is specified, but ParseProvider now just returns the input unchanged. Defaulting to Bicep appears to happen later in Manager.newProvider when Provider==NotSpecified; consider updating this comment to reflect the current behavior (and/or rename the function if it's no longer validating).
Suggested change
| // Parses the specified IaC Provider to ensure whether it is valid or not | |
| // Defaults to `Bicep` if no provider is specified | |
| // ParseProvider returns the specified IaC provider unchanged. | |
| // Defaulting for NotSpecified is handled later during provider creation. |
Comment on lines
40
to
46
| Module string `yaml:"module,omitempty"` | ||
| Name string `yaml:"name,omitempty"` | ||
| DeploymentStacks map[string]any `yaml:"deploymentStacks,omitempty"` | ||
| // Config holds provider-specific configuration options | ||
| Config map[string]any `yaml:"config,omitempty"` | ||
| // Provisioning options for each individually defined layer. | ||
| Layers []Options `yaml:"layers,omitempty"` |
There was a problem hiding this comment.
Adding Options.Config introduces a new top-level infra field, but this struct is also used for infra.layers. Today Options.Validate rejects certain top-level fields when layers are set; consider clarifying whether config is allowed at the root when layers are present (and enforce it in Validate) to avoid silently ignoring root config when layers exist. Also ensure the azure.yaml JSON schemas are updated so editor/schema validation matches the new infra.config field and custom provider strings.
Comment on lines
+87
to
+99
| // State returns the current state of provisioned infrastructure. | ||
| func (p *ExternalProvisioningProvider) State( | ||
| ctx context.Context, | ||
| options *provisioning.StateOptions, | ||
| ) (*provisioning.StateResult, error) { | ||
| req := &azdext.ProvisioningMessage{ | ||
| RequestId: uuid.NewString(), | ||
| MessageType: &azdext.ProvisioningMessage_StateRequest{ | ||
| StateRequest: &azdext.ProvisioningStateRequest{ | ||
| Options: &azdext.ProvisioningStateOptions{}, | ||
| }, | ||
| }, | ||
| } |
There was a problem hiding this comment.
StateOptions contains a Hint() used by core flows (e.g., env refresh) to select a specific deployment, but the external provider adapter currently discards the options passed to State and the proto type ProvisioningStateOptions has no fields. This means extension providers cannot honor --hint and will behave differently from built-in providers. Consider adding the hint to the proto (and extension-side interface), populating it here, and wiring it through ProvisioningManager.onState.
Comment on lines
+262
to
+279
| func convertToProtoOptions( | ||
| options provisioning.Options, | ||
| ) (*azdext.ProvisioningOptions, error) { | ||
| deploymentStacks := make( | ||
| map[string]string, len(options.DeploymentStacks), | ||
| ) | ||
| for k, v := range options.DeploymentStacks { | ||
| deploymentStacks[k] = fmt.Sprintf("%v", v) | ||
| } | ||
|
|
||
| protoOptions := &azdext.ProvisioningOptions{ | ||
| Provider: string(options.Provider), | ||
| Path: options.Path, | ||
| Module: options.Module, | ||
| DeploymentStacks: deploymentStacks, | ||
| IgnoreDeploymentState: options.IgnoreDeploymentState, | ||
| } | ||
|
|
There was a problem hiding this comment.
convertToProtoOptions doesn’t include key fields from provisioning.Options that are required for feature parity, notably Options.Name (used for provisioning layers). Without passing the layer name, extension providers can’t scope deployments/state correctly when infra.layers is used. Consider extending ProvisioningOptions in the proto with name (and any other required layer-scoping fields) and mapping them here.
Comment on lines
+120
to
+124
| message ProvisioningStateOptions {} | ||
|
|
||
| message ProvisioningStateResult { | ||
| ProvisioningState state = 1; | ||
| } |
There was a problem hiding this comment.
ProvisioningStateOptions is currently an empty message, but core provisioning.StateOptions includes a deployment hint used by existing commands. This will prevent extension providers from receiving necessary context. Consider adding corresponding fields (e.g., hint) to this message and updating the adapter/manager to propagate them end-to-end.
| string module = 3; | ||
| map<string, string> deployment_stacks = 4; | ||
| bool ignore_deployment_state = 5; | ||
| google.protobuf.Struct config = 6; |
There was a problem hiding this comment.
ProvisioningOptions is missing fields present in core provisioning.Options that matter for correct behavior (e.g., the layer Name when infra.layers is used). Consider adding those fields (at least name) so extension providers receive the same initialization context as built-in providers, and update the Go conversions accordingly.
Suggested change
| google.protobuf.Struct config = 6; | |
| google.protobuf.Struct config = 6; | |
| string name = 7; |
Comment on lines
+48
to
+78
| // Stream handles the bi-directional streaming for provisioning operations. | ||
| func (s *ProvisioningService) Stream( | ||
| stream azdext.ProvisioningService_StreamServer, | ||
| ) error { | ||
| ctx := stream.Context() | ||
| extensionClaims, err := extensions.GetClaimsFromContext(ctx) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get extension claims: %w", err) | ||
| } | ||
|
|
||
| options := extensions.FilterOptions{ | ||
| Id: extensionClaims.Subject, | ||
| } | ||
|
|
||
| extension, err := s.extensionManager.GetInstalled(options) | ||
| if err != nil { | ||
| return status.Errorf( | ||
| codes.FailedPrecondition, "failed to get extension: %s", err.Error(), | ||
| ) | ||
| } | ||
|
|
||
| if !extension.HasCapability(extensions.ProvisioningProviderCapability) { | ||
| return status.Errorf( | ||
| codes.PermissionDenied, | ||
| "extension does not support provisioning-provider capability", | ||
| ) | ||
| } | ||
|
|
||
| // Create message broker for this stream | ||
| ops := azdext.NewProvisioningEnvelope() | ||
| broker := grpcbroker.NewMessageBroker(stream, ops, extension.Id, log.Default()) |
There was a problem hiding this comment.
There are no unit tests in this PR covering the new provisioning gRPC service registration flow (capability check, provider registration, broker cleanup) or the ExternalProvisioningProvider request/response mapping. Given this is new extension surface area, consider adding at least basic unit tests similar to other grpcserver service tests to lock in the expected behavior.
Collaborator
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
jongio
reviewed
Apr 3, 2026
Member
jongio
left a comment
jongio
left a comment
There was a problem hiding this comment.
Solid implementation - the MessageBroker + bidi stream pattern is clean and consistent with how service targets and framework services work. The plumbing (IoC, middleware, server registration, extension host API) is all well done.
My comments focus on forward compatibility (making it easier to extend this surface later) and a couple of UX gaps where extensions won't get the same treatment as built-in providers.
|
|
||
| message RegisterProvisioningProviderRequest { | ||
| string name = 1; | ||
| } |
Member
There was a problem hiding this comment.
RegisterProvisioningProviderRequest only has name. If you envision extensions that don't need the full lifecycle (e.g., a CDK extension that only generates IaC and delegates deploy to a built-in provider), it'd be hard to distinguish them from full-pipeline providers without a mode/type field.
Adding a field now is free. Adding it after extensions are published means treating its absence as a default.
Suggested change
| } | |
| message RegisterProvisioningProviderRequest { | |
| string name = 1; | |
| string mode = 2; // e.g., "generator" vs "provisioner" | |
| } |
| // Metadata capability enables extensions to provide comprehensive metadata about their commands and capabilities | ||
| MetadataCapability CapabilityType = "metadata" | ||
| // Provision provider enables extensions to provide a custom provisioning experience | ||
| ProvisioningProviderCapability CapabilityType = "provisioning-provider" |
Member
There was a problem hiding this comment.
provisioning-provider appears in extension.yaml manifests - once extensions publish with this, renaming it is a breaking change. If there's any chance the capability covers a broader scope in the future (e.g., CDK-style generators that only produce IaC without owning deploy), consider whether infra-provider or similar would be more future-proof. If provisioning-provider is the intended long-term name, no action needed.
| } | ||
|
|
||
| resp, err := p.broker.SendAndWaitWithProgress(ctx, req, func(msg string) { | ||
| log.Printf("provisioning progress: %s", msg) |
Member
There was a problem hiding this comment.
Deploy/Preview/Destroy all pass log.Printf as the progress callback. Extension progress messages won't appear in the CLI's interactive progress display - only in Go logs. Built-in providers surface progress through the console formatter.
Even a TODO here would help track the gap. A quick fix would be accepting an io.Writer or progress callback in the constructor so callers can wire up real console output.
| return deployResult | ||
| } | ||
|
|
||
| func convertFromProtoPreviewResult( |
Member
There was a problem hiding this comment.
convertFromProtoPreviewResult maps Summary to Status and returns empty Changes. The proto's ProvisioningDeploymentPreview only has a summary field - extensions can't report what resources will be created/modified/deleted like built-in providers do.
If resource-level preview is planned later, the proto message will need a repeated field for changes. If it's intentionally scoped out, a brief comment would prevent future confusion.
| } | ||
|
|
||
| // PlannedOutputs returns planned outputs. Not yet supported for external providers. | ||
| func (p *ExternalProvisioningProvider) PlannedOutputs( |
Member
There was a problem hiding this comment.
Returns empty with "Not yet supported" comment (good). Extensions that can declare their outputs ahead of time enable better azd orchestration - wiring outputs to downstream services before deploy. If this is on the roadmap, a tracking issue would help.
|
|
||
| // --- Conversion helpers --- | ||
|
|
||
| func convertToProtoOptions( |
Member
There was a problem hiding this comment.
The Copilot bot already flagged missing tests for provisioning_service.go. Adding to that: the convertFrom*/convertTo* helpers in this file are pure transforms - ideal for table-driven tests and likely to regress when proto messages evolve.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Introduces extensibility for custom provisioning providers (e.g., Pulumi, CDK, scripts) via the existing gRPC extension framework, following the same
MessageBroker-based patterns used by service targets and framework services.Epic: #7465
Architecture
New Components
grpc/proto/provisioning.protopkg/azdext/provisioning_envelope.goMessageEnvelopefor broker integrationpkg/azdext/provisioning_manager.goProvisioningProvider+ProvisioningManagerinternal/grpcserver/provisioning_service.gointernal/grpcserver/external_provisioning_provider.goExternalProvisioningProviderbridging core to extensionpkg/azdext/provisioning{.pb,.grpc.pb}.goModified Files
pkg/azdext/extension_host.goWithProvisioningProvider()fluent API +Run()lifecyclepkg/azdext/azd_client.goProvisioning()service client accessorpkg/extensions/registry.goProvisioningProviderCapabilityconstantcmd/middleware/extensions.golistenCapabilitiescmd/container.gointernal/grpcserver/server.gopkg/infra/provisioning/provisioning.goParseProvider()accepts any provider kindpkg/infra/provisioning/provider.goOptions.Configfield for extension configKey Design Decisions
MessageBroker,ExtensionHost,ExtensionError(not POC's bespoke approach from WIP: Adds support for provision providers viaazdextension framework #5381)ComponentManagerfor per-service instances), provisioning providers are project-scopedazure.yaml'sinfra.provider: "myprov"resolves viaResolveNamed("myprov")SendAndWaitWithProgress()Options.Config map[string]any→google.protobuf.Structfor extension-specific configExtension Author Usage
End User Usage
Testing
go build ./...passesgo vet ./...cleangofmt -s -l .cleango test ./internal/grpcserver/...passesgo test ./pkg/infra/provisioning/...passesgo test ./pkg/extensions/...passes