ceph-dev-pipeline: Add a README - #2672
Open
djgalloway wants to merge 1 commit into
Open
Conversation
Document how the job works: the stage flow, the two distinct sources of scripts in a matrix cell's workspace (the ceph-build clone providing ./scripts/, and the ceph source tarball from the setup job providing ./dist/ceph/), and where the builder container's base OS image actually comes from -- build-with-container.py's DefaultImage map applied to Dockerfile.build, not anything pinned in ceph-build. Also clarifies the frequently-conflated builder image (ceph-build) vs. the runtime Ceph dev image built by scripts/build_container, and records why SETUP_JOB exists: reef predated the build-with-container.py backport and had to be built by ceph-dev-new-setup/ceph-dev-new instead. Signed-off-by: David Galloway <david.galloway@ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documents how
ceph-dev-pipelineactually works. No functional changes — this addsceph-dev-pipeline/README.mdonly.What it covers
./scripts/from theceph/ceph-buildclone at$CEPH_BUILD_BRANCH(upload/shaman/pulp/podman glue)./dist/ceph/extracted from the source tarball produced by the setup job (build-with-container.py,Dockerfile.build,install-deps.sh,debian/,container/)So everything running inside the build container comes from the branch being built, not from a checkout.
Dockerfile.buildisFROM $DISTRO, whereDISTROis a--build-argset frombuild-with-container.py'sDefaultImagemap (centos9 →quay.io/centos/centos:stream9, rocky10 →docker.io/rockylinux/rockylinux:10, noble →docker.io/ubuntu:24.04, etc.). Includes the tag scheme, the/cephbind mount, and whyREGISTRY_AUTH_FILEis set.scripts/build_container→container/build.shwith its own hardcodedFROM_IMAGEmap.SETUP_JOBexists —build-with-container.pylanded in ceph.git in Aug 2024 and wasn't backported to reef until Feb 2025 (ceph#61683), so reef branches had to go throughceph-dev-new-setup+ceph-dev-newinstead.ceph-trigger-buildroutes between the two on theCEPH-BUILD-JOBtrailer.ceph-source-distbecame the pipeline's default setup job in 6e1f737 withceph-dev-new-setupkept as an option; the two are interchangeable because both archive the samedist/**layout the copy-artifacts stage filters on.Notes for reviewers
Content was derived by reading the Jenkinsfile, the JJB definition,
scripts/, and ceph.git'sbuild-with-container.py/Dockerfile.build/container/Containerfile— worth a sanity check on the base-image table and theSETUP_JOBhistory in particular.