Skip to content

feat: Implement multi-arch build orchestration and image index assembly#2170

Open
anchi205 wants to merge 1 commit into
shipwright-io:mainfrom
anchi205:multiarch-orchestration
Open

feat: Implement multi-arch build orchestration and image index assembly#2170
anchi205 wants to merge 1 commit into
shipwright-io:mainfrom
anchi205:multiarch-orchestration

Conversation

@anchi205

Copy link
Copy Markdown
Member

Changes

Generate per-platform PipelineTasks with architecture-specific nodeSelectors so each build runs on a matching node. Source code is bundled as an OCI artifact during source-acquisition and pulled on each platform node. Per-platform tasks suffix the output image tag with the platform to prevent registry GC from deleting intermediate images.

After all platform builds complete, an assemble-index task creates an OCI image index referencing each platform image. Result aggregation populates BuildRunStatus.PlatformResults with per-platform digest, size, vulnerabilities, and status, and sets ManifestDigest and Output.Digest to the index digest. Partial failure is tracked per platform so users can identify which architecture failed.

CLI flags --push-source-bundle and --assemble-index are added to the image-processing binary, consumed by the source-acquisition and assemble-index pipeline tasks respectively.

Related Issues

Fixes #2143
Fixes #2076

Type of PR

/kind feature

Submitter Checklist

  • Includes tests if functionality changed/was added
  • Includes docs if changes are user-facing
  • Kind label has been set
  • Release notes block has been filled in, or marked NONE

Release Notes

Multi-architecture container image builds. When spec.output.platforms is set, the controller runs parallel per-platform builds on architecture-matched nodes and assembles an OCI image index. Per-platform status, digest, and vulnerabilities are reported in status.platformResults.

@openshift-ci openshift-ci Bot added kind/feature Categorizes issue or PR as related to a new feature. release-note Label for when a PR has specified a release note labels Apr 24, 2026
@openshift-ci openshift-ci Bot requested review from adambkaplan and qu1queee April 24, 2026 13:05
@openshift-ci

openshift-ci Bot commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign apoorvajagtap for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pull-request-size pull-request-size Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Apr 24, 2026
Comment thread pkg/apis/build/v1beta1/buildrun_types.go Outdated
Comment thread pkg/image/bundle.go Outdated
Comment thread pkg/image/bundle.go Outdated
@anchi205 anchi205 force-pushed the multiarch-orchestration branch from d625a1c to 7ec0b3f Compare May 26, 2026 05:06
Comment thread cmd/image-processing/main_test.go Outdated
Comment thread cmd/image-processing/main_test.go Outdated
Comment thread pkg/image/index_test.go
Comment thread pkg/reconciler/buildrun/resources/multiarch.go Outdated
Comment thread pkg/reconciler/buildrun/resources/multiarch.go
Comment thread pkg/reconciler/buildrun/resources/multiarch.go Outdated
Comment thread pkg/reconciler/buildrun/resources/resource_builders.go Outdated
Comment thread pkg/reconciler/buildrun/resources/results.go Outdated
Comment thread pkg/reconciler/buildrun/resources/results.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds first-class multi-architecture build support to the BuildRun PipelineRun executor by fanning out per-platform build tasks (scheduled onto matching nodes), pushing/pulling source as an OCI artifact, and assembling a final OCI image index. It also extends status reporting to include per-platform results and updates docs/CRDs/tests accordingly.

Changes:

  • Generate multi-arch PipelineRuns: per-platform build-<os>-<arch> tasks + assemble-index, with per-task OS/arch nodeSelectors and EmptyDir workspaces.
  • Add image-processing CLI support for --push-source-bundle and --assemble-index, plus an AssembleImageIndex helper.
  • Report per-platform digests/sizes/vulnerabilities in .status.platformResults and set .status.output.digest to the index digest; add unit + e2e coverage and update docs/CRDs.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
test/utils/v1beta1/taskruns.go Add helper to list/sort TaskRuns for a PipelineRun by label for e2e assertions.
test/utils/v1beta1/image.go Fix digest validation for multi-platform outputs by using remote.Get (index-safe).
test/e2e/v1beta1/e2e_pipelinerun_test.go Add e2e coverage for multi-arch spec.output.platforms and index assembly behavior.
test/e2e/v1beta1/common_suite_test.go Extend test build prototype to set spec.output.platforms.
pkg/reconciler/buildrun/resources/results.go Populate .status.platformResults and set output digest from assemble-index results.
pkg/reconciler/buildrun/resources/results_test.go Add unit tests for multi-arch result extraction/aggregation.
pkg/reconciler/buildrun/resources/resource_builders.go Add EmptyDir workspace bindings for multi-arch PipelineRuns.
pkg/reconciler/buildrun/resources/pipelinerun_generator.go Implement multi-arch PipelineRun generation + scheduling pinning via TaskRunSpecs.
pkg/reconciler/buildrun/resources/multiarch.go New: multi-arch pipeline task generation (source bundle push/pull, per-platform tasks, assemble-index).
pkg/reconciler/buildrun/resources/multiarch_test.go New: unit tests for multi-arch pipeline generation contracts.
pkg/reconciler/buildrun/buildrun.go Reconcile: extract per-platform results from PipelineRun child TaskRuns into BuildRun status.
pkg/image/index.go New: build an OCI image index from per-platform images with platform descriptors.
pkg/image/index_test.go Unit tests for index assembly behavior and error cases.
pkg/apis/build/v1beta1/zz_generated.deepcopy.go DeepCopy updates for new PlatformResults / PlatformBuildResult.
pkg/apis/build/v1beta1/buildrun_types.go API: define PlatformBuildResult and clarify output digest semantics for multi-arch.
docs/buildrun.md Document .status.platformResults and multi-arch digest semantics.
docs/build.md Document spec.output.platforms behavior/requirements and examples.
deploy/crds/shipwright.io_buildruns.yaml CRD schema updates for new status fields and digest description.
cmd/image-processing/main.go Add CLI modes for pushing source bundles and assembling/pushing image indexes.
cmd/image-processing/main_test.go Add tests for new CLI modes and improve short-mode behavior for network-dependent test.
Files not reviewed (1)
  • pkg/apis/build/v1beta1/zz_generated.deepcopy.go: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/reconciler/buildrun/resources/results.go Outdated
Comment thread pkg/reconciler/buildrun/resources/results.go Outdated
Comment thread pkg/reconciler/buildrun/resources/pipelinerun_generator.go
Comment thread pkg/reconciler/buildrun/resources/multiarch.go
Comment thread pkg/reconciler/buildrun/resources/multiarch.go
@anchi205 anchi205 force-pushed the multiarch-orchestration branch from 7ec0b3f to 76a77cc Compare June 15, 2026 13:27
Generate per-platform PipelineTasks with architecture-specific
nodeSelectors so each build runs on a matching node. Source code is
bundled as an OCI artifact during source-acquisition and pulled on
each platform node. Per-platform tasks suffix the output image tag with
the platform to prevent registry GC from deleting intermediate images.

After all platform builds complete, an assemble-index task creates an
OCI image index referencing each platform image. Result aggregation
populates BuildRunStatus.platformResults with per-platform digest,
size, vulnerabilities, and status, and sets status.output.digest to the
image index digest. Partial failure is tracked per platform so users can
identify which architecture failed.

CLI flags --push-source-bundle and --assemble-index are added to the
image-processing binary, consumed by the source-acquisition and
assemble-index pipeline tasks respectively.

Closes shipwright-io#2143, closes shipwright-io#2076

Signed-off-by: Anchita Borah <anborah@redhat.com>
@anchi205 anchi205 force-pushed the multiarch-orchestration branch from 76a77cc to 80d29b6 Compare June 15, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature Categorizes issue or PR as related to a new feature. release-note Label for when a PR has specified a release note size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[FEATURE] Implement Per-Arch Builds [FEATURE] Implement image index for multi-arch images

4 participants