feat: Implement multi-arch build orchestration and image index assembly#2170
feat: Implement multi-arch build orchestration and image index assembly#2170anchi205 wants to merge 1 commit into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
d625a1c to
7ec0b3f
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds first-class multi-architecture build support to the BuildRun PipelineRun executor by fanning out per-platform build tasks (scheduled onto matching nodes), pushing/pulling source as an OCI artifact, and assembling a final OCI image index. It also extends status reporting to include per-platform results and updates docs/CRDs/tests accordingly.
Changes:
- Generate multi-arch PipelineRuns: per-platform
build-<os>-<arch>tasks +assemble-index, with per-task OS/arch nodeSelectors and EmptyDir workspaces. - Add image-processing CLI support for
--push-source-bundleand--assemble-index, plus anAssembleImageIndexhelper. - Report per-platform digests/sizes/vulnerabilities in
.status.platformResultsand set.status.output.digestto the index digest; add unit + e2e coverage and update docs/CRDs.
Reviewed changes
Copilot reviewed 19 out of 20 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| test/utils/v1beta1/taskruns.go | Add helper to list/sort TaskRuns for a PipelineRun by label for e2e assertions. |
| test/utils/v1beta1/image.go | Fix digest validation for multi-platform outputs by using remote.Get (index-safe). |
| test/e2e/v1beta1/e2e_pipelinerun_test.go | Add e2e coverage for multi-arch spec.output.platforms and index assembly behavior. |
| test/e2e/v1beta1/common_suite_test.go | Extend test build prototype to set spec.output.platforms. |
| pkg/reconciler/buildrun/resources/results.go | Populate .status.platformResults and set output digest from assemble-index results. |
| pkg/reconciler/buildrun/resources/results_test.go | Add unit tests for multi-arch result extraction/aggregation. |
| pkg/reconciler/buildrun/resources/resource_builders.go | Add EmptyDir workspace bindings for multi-arch PipelineRuns. |
| pkg/reconciler/buildrun/resources/pipelinerun_generator.go | Implement multi-arch PipelineRun generation + scheduling pinning via TaskRunSpecs. |
| pkg/reconciler/buildrun/resources/multiarch.go | New: multi-arch pipeline task generation (source bundle push/pull, per-platform tasks, assemble-index). |
| pkg/reconciler/buildrun/resources/multiarch_test.go | New: unit tests for multi-arch pipeline generation contracts. |
| pkg/reconciler/buildrun/buildrun.go | Reconcile: extract per-platform results from PipelineRun child TaskRuns into BuildRun status. |
| pkg/image/index.go | New: build an OCI image index from per-platform images with platform descriptors. |
| pkg/image/index_test.go | Unit tests for index assembly behavior and error cases. |
| pkg/apis/build/v1beta1/zz_generated.deepcopy.go | DeepCopy updates for new PlatformResults / PlatformBuildResult. |
| pkg/apis/build/v1beta1/buildrun_types.go | API: define PlatformBuildResult and clarify output digest semantics for multi-arch. |
| docs/buildrun.md | Document .status.platformResults and multi-arch digest semantics. |
| docs/build.md | Document spec.output.platforms behavior/requirements and examples. |
| deploy/crds/shipwright.io_buildruns.yaml | CRD schema updates for new status fields and digest description. |
| cmd/image-processing/main.go | Add CLI modes for pushing source bundles and assembling/pushing image indexes. |
| cmd/image-processing/main_test.go | Add tests for new CLI modes and improve short-mode behavior for network-dependent test. |
Files not reviewed (1)
- pkg/apis/build/v1beta1/zz_generated.deepcopy.go: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
7ec0b3f to
76a77cc
Compare
Generate per-platform PipelineTasks with architecture-specific nodeSelectors so each build runs on a matching node. Source code is bundled as an OCI artifact during source-acquisition and pulled on each platform node. Per-platform tasks suffix the output image tag with the platform to prevent registry GC from deleting intermediate images. After all platform builds complete, an assemble-index task creates an OCI image index referencing each platform image. Result aggregation populates BuildRunStatus.platformResults with per-platform digest, size, vulnerabilities, and status, and sets status.output.digest to the image index digest. Partial failure is tracked per platform so users can identify which architecture failed. CLI flags --push-source-bundle and --assemble-index are added to the image-processing binary, consumed by the source-acquisition and assemble-index pipeline tasks respectively. Closes shipwright-io#2143, closes shipwright-io#2076 Signed-off-by: Anchita Borah <anborah@redhat.com>
76a77cc to
80d29b6
Compare
Changes
Generate per-platform PipelineTasks with architecture-specific nodeSelectors so each build runs on a matching node. Source code is bundled as an OCI artifact during source-acquisition and pulled on each platform node. Per-platform tasks suffix the output image tag with the platform to prevent registry GC from deleting intermediate images.
After all platform builds complete, an assemble-index task creates an OCI image index referencing each platform image. Result aggregation populates BuildRunStatus.PlatformResults with per-platform digest, size, vulnerabilities, and status, and sets ManifestDigest and Output.Digest to the index digest. Partial failure is tracked per platform so users can identify which architecture failed.
CLI flags --push-source-bundle and --assemble-index are added to the image-processing binary, consumed by the source-acquisition and assemble-index pipeline tasks respectively.
Related Issues
Fixes #2143
Fixes #2076
Type of PR
/kind feature
Submitter Checklist
Release Notes