fix(cdk-assets): re-tag docker image for multi-destination publishes#1591
Draft
calabrla wants to merge 3 commits into
Draft
fix(cdk-assets): re-tag docker image for multi-destination publishes#1591calabrla wants to merge 3 commits into
calabrla wants to merge 3 commits into
Conversation
auto-merge was automatically disabled
June 3, 2026 19:24
Head branch was pushed to by a user without write access
calabrla
force-pushed
the
fix/multi-region-docker-asset-tagging
branch
from
5a68328 to
e25181f
Compare
mrgrain
requested changes
Jun 4, 2026
Contributor
mrgrain
left a comment
mrgrain
left a comment
There was a problem hiding this comment.
Looks good, but we will need an integration test for this
When the same container image asset is published to multiple destinations (e.g. multi-region deployments), the WorkGraphBuilder deduplicates the asset-build node so that build() only runs for one destination. This means docker.tag() is only called for the first destination's ECR URI. When publish() subsequently runs for other destinations, docker push fails with "An image does not exist locally with the tag" because the local tag was never created for those repositories. This fix adds a check in publish() that verifies the destination imageUri exists locally before pushing. If it doesn't, it finds an existing local image with the same imageTag (content hash) and re-tags it for the current destination. This is a lightweight docker tag operation that avoids any rebuild. Fixes multi-region container image deployments failing on second+ region.
calabrla
force-pushed
the
fix/multi-region-docker-asset-tagging
branch
from
e25181f to
a9ff382
Compare
calabrla
force-pushed
the
fix/multi-region-docker-asset-tagging
branch
from
e3a681b to
a9ff382
Compare
…publishing Add two integration tests that validate the fix for publishing the same docker image asset to multiple destinations: 1. Same-region multi-dest: Deploys two stacks with the same docker image source to the same region. The work graph deduplicates the build node, exercising the re-tag path in publish(). 2. Cross-region: Deploys the same docker image to two different regions, requiring bootstrap of a secondary region. The secondary region is derived from the AWS_REGIONS pool to guarantee it differs from the primary. Gated by CDK_SECONDARY_REGION env var to avoid affecting other tests.
calabrla
force-pushed
the
fix/multi-region-docker-asset-tagging
branch
from
3ae1af2 to
7c9c43a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
WorkGraphBuilderdeduplicates theasset-buildnode for same-source assets, only one destination getsdocker tag'd duringbuild()— subsequentpublish()calls for other destinations fail because the local tag doesn't existdocker inspect+docker images --filter+docker tagfallback inpublish()to ensure the image is tagged locally before pushingRoot Cause
WorkGraphBuilder.addAsset()creates oneasset-buildnode per unique source (assetId+genericSourcehash). When the same image is deployed to multiple regions, only the first destination's handler getsbuild()called by the graph executor. The other destinations only receivepublishEntry(), which callsdocker pushon a URI that was never tagged locally.Test plan
publish re-tags image when local tag is missing for a destination— verifies full build+publish flow with multi-dest manifestpublish re-tags from existing image when build was not called for destination— simulates the graph executor scenario (buildAssets: false)ca-central-1andeu-west-2simultaneously — both regions succeededBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license