fix(patch): strip hasProviderDefault fields recursively through arrays#423
Merged
Merged
Conversation
Provider-default sub-fields inside nested list elements (e.g. ContainerDefinition.Cpu, ContainerDefinition.PortMappings.HostPort) caused spurious resource replacements on re-apply. The stored state, populated by cloud-provider Read, carried these fields; the evaluated desired state did not. jsonpatch's default set-based list comparison serializes each element to JSON bytes, so provider-populated elements never matched their desired counterparts, emitting add/remove ops on paths marked createOnly — and tripping needsReplacement for unchanged resources (most painfully, destroying running ECS services on reapply). Replace the prior document-only stripper with a two-sided walker that descends a dotted schema path through both document and patch. For pure object paths the conditional top-level behavior is preserved so user overrides keep diffing. For any path that traverses an array, the leaf key is stripped symmetrically from both sides on every reachable element — the only correct semantic under set-based list comparison. Applies recursively, so arbitrarily deep paths like ContainerDefinitions.PortMappings.HostPort work. Includes tests for the exact Grafana-style reproduction, mixed elements (one sets the field, one doesn't), two-level array nesting, a negative test where a real user change still forces a replace, and a guard against regressing the existing top-level provider-default override semantic.
c93b3db to
3e0aadd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Re-applying an unchanged forma could silently destroy working stateful resources. The most visible case: adding any unrelated resource to a forma that contains an
AWS::ECS::Servicetriggered a full replace of the Service (5–10 min outage), even though the Service's own inputs hadn't changed.Root cause was in the patch-generation path's handling of provider-defaulted sub-fields inside nested list elements:
AWS::ECS::ContainerDefinition.Cpuis annotatedhasProviderDefault = true. AWS Read returnsCpu: 0even when the user didn't set it. Desired state (from PKL eval) omits the field.jsonpatchcompares list-valued fields as sets by default — serializing each element to JSON bytes.{"Cpu":0,…}≠{…}as bytes, so container elements fail to match their counterparts.compareArraythen emitsadd/removepatch ops against/ContainerDefinitions/N. That path iscreateOnly→needsReplacement = true→ resource replace.TaskDefinitionresolvable diffs.The existing
hasProviderDefaultstripping worked only at the top level — not on sub-fields inside list elements.Change
Replace the prior document-only stripper with a two-sided walker (
stripProviderDefaultPath) that descends a dotted schema path through both the document and the patch simultaneously:BucketEncryption): preserve the original conditional behaviour — strip from the document only when the field is absent from the patch, so user overrides still diff.ContainerDefinitions.PortMappings.HostPortwork.A new helper
removeProviderDefaultFieldsBothreturns both stripped buffers;createPatchDocumentuses it in place of the prior document-only call. The pre-existing publicremoveProviderDefaultFieldsis kept for callers that only need the document side.Tests
New cases in
patch_document_test.go:TestGeneratePatch_ProviderDefaultInsideArray_SingleElement_NoReplace— Grafana-style repro, singleContainerDefinition,Cpu: 0vs absent.TestGeneratePatch_ProviderDefaultInsideArray_MixedElements_NoReplace— two containers, one setsCpu, the other doesn't.TestGeneratePatch_ProviderDefaultInsideNestedArray_PortMappingHostPortand..._Mixed— two levels of array nesting (ContainerDefinitions.PortMappings.HostPort).TestGeneratePatch_UserChangedFieldInsideArray_StillReplaces— negative test: a real user change to acreateOnlylist element still tripsneedsReplacement.TestGeneratePatch_TopLevelProviderDefaultOverride_StillDiffs— guard against regressing the existing top-level override semantic.Updated:
TestRemoveProviderDefaultFields_NestedFieldInsideArray— the old assertion required that an array-nested provider-default field present only in the patch be preserved in the document. That semantic was the bug under set-comparison. The test now asserts symmetric stripping, with a comment explaining why.Full
go test -tags=unit ./...suite green locally, includinginternal/workflow_tests.Related
formae-plugin-aws@main(commit073270f) annotatesPortMapping.hostPortwithhasProviderDefault = trueso the PKL emitter surfaces the new path.formae-plugin-aws/2026-04-20-ecs-spurious-replace-on-reapply.md.