From 1a33f6fc85e015a95a08a3c8a57f4564dbe34f71 Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Mon, 22 Jun 2026 12:01:45 +0200 Subject: [PATCH 01/10] Document CI/CD pipeline --- docs/developer-guide/ci-cd-pipeline.md | 101 +++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 docs/developer-guide/ci-cd-pipeline.md diff --git a/docs/developer-guide/ci-cd-pipeline.md b/docs/developer-guide/ci-cd-pipeline.md new file mode 100644 index 00000000..311cedfb --- /dev/null +++ b/docs/developer-guide/ci-cd-pipeline.md @@ -0,0 +1,101 @@ +--- +id: ci-cd-pipeline +title: CI/CD Pipeline +sidebar_label: CI/CD Pipeline +--- + +# CI/CD Pipeline + +This document describes the continuous integration and continuous deployment (CI/CD) pipeline for OpenRemote, introducing the concepts and workflows added in our recent pipeline migration (introduced in PR [#2696](https://github.com/openremote/openremote/pull/2696)). + +## How to make changes to the pipeline and test them + +When modifying the pipeline, you should generally branch off `master` and test your changes by opening a Pull Request. Since the workflows run automatically for PRs, you can iterate on pipeline changes within the PR itself. + +To test changes without polluting the main repository, you can: +1. Push to your own fork and test against custom branches. +2. Test dispatchable workflows manually. Note that if a workflow file has not been merged to `master` yet, you cannot trigger it from the GitHub Actions UI. Instead, you must use the [GitHub CLI](https://cli.github.com/) to trigger it on your feature branch: + ```bash + gh workflow run --ref + ``` + * **Note on new dispatchable workflows:** If you are adding a completely new workflow file, it won't be recognized by GitHub at all until it is registered. A common workaround is to temporarily add an `on: [push]` event to the workflow, push your branch so it runs once, and then remove the `push` event. After this, it will be available to trigger via the CLI. + +## "Environment" and "Job" Workflows + +The pipeline is structured using a separation of concerns between environments and jobs: + +- **Environment Workflows**: These are entry points that handle specific contexts (e.g., `review.yml` for Pull Requests, `staging.yml` for push to `master` and releases, `custom-review.yml` and `custom-staging.yml` for custom projects). They act as the orchestrator for different triggers. +- **Job Workflows**: These are reusable workflows (`workflow_call`) that contain the actual steps for specific tasks (like running tests, building dists, or publishing artifacts). Environment workflows call these job workflows, which prevents duplication. + +### How Reusable Workflows Work +Job workflows are configured as reusable workflows. This allows them to be shared across multiple environment orchestrators. + +**Handling Skipped Jobs:** +Because GitHub Actions treats skipped jobs differently in the status check context (a skipped job in a required chain might cause downstream dependencies to hang or fail incorrectly), you often need to handle skipping *inside* the reusable workflow itself rather than skipping the entire `workflow_call`. This is achieved by checking conditionals within the steps or the jobs of the reusable workflow to exit cleanly. + +## Deployments + +Deployments are handled by the `deploy.yml` dispatch workflow. This workflow can be triggered manually or via specific staging outcomes. For deployments to test servers, the setup has been minimized to reduce the deployment time significantly. + +## Optimizing Jobs and Job Dependencies + +We utilize several strategies to ensure the pipeline is as fast and efficient as possible: + +### Detect Changes +We use a "detect changes" job early in the workflow to determine which files have been modified. Subsequent jobs use the outputs of this job to conditionally skip themselves. This is why UI tests don't run if only backend Java code was touched, significantly speeding up the pipeline. + +### Minimal Dependency Installation +Jobs should only install what they need. For Node/Yarn-based projects, instead of running a full `yarn install` everywhere, we use focused installs, e.g., `yarn focus @openremote/util @openremote/test`. This installs the bare minimum dependencies for the scoped workspace, drastically reducing setup time. + +### Checking out with Git LFS +Only use `lfs: true` in the `actions/checkout` step if the job actually requires large files (like images, binaries, or specific test assets). Checking out LFS files adds significant time and bandwidth overhead, so it is omitted where unnecessary. + +### Build Tooling and File Modified Metadata Tricks +To avoid rebuilding components when passing artifacts between jobs, we use tricks like updating file modified metadata (e.g., using `touch`) after downloading artifacts. This tricks build tools (like Gradle) into recognizing that the downloaded artifact is the newest version, preventing the tool from re-triggering unnecessary compilations. + +## Artifacts: When and What to Include + +Artifacts are used to share files between isolated jobs. + +**The most important rule for artifacts is to split them where possible and create artifacts containing only the bare necessities for each specific job.** + +- **What to include:** Compiled distributions, test reports, and coverage files. Do not upload `node_modules` or full `build` folders; compress and selectively upload only the required distributions. +- **When to use them:** Only pass artifacts when a downstream job strictly requires the output of an upstream job. Artifact upload/download takes time and consumes storage. Avoid sharing sensitive data through public artifacts. +- **Optimization through splitting:** Uploading separate, minimal artifacts (e.g., saving the manager dist and UI dists as separate artifacts rather than one massive archive) ensures downstream jobs only download exactly what they need, significantly speeding up the pipeline. + +## Configuration: Gradle vs. Workflows + +A good rule of thumb is: +- **Use Gradle** to manage the internal build lifecycle, task graphs, dependencies, and code compilation. +- **Use Workflows** for orchestration, conditional job execution (e.g., detect changes), environment provisioning, and deployment. + +If a task can be optimized or defined within Gradle (such as skipping a task with `-x`), prefer that over complex bash scripts in the workflow. However, if skipping a process depends on the GitHub PR context (like changes to specific directories), manage it at the workflow level. + +## Caching Strategies + +We implement several caching strategies to speed up builds: +- **Docker Images:** Docker layer caching is used to avoid rebuilding unmodified layers. We utilize the GitHub Actions cache backend for Docker buildx. + + +## GitHub Concepts + +### Composite Actions +Composite actions are defined under `.github/actions/` and act as reusable steps (unlike reusable workflows, which act as reusable jobs). We use them to keep things DRY. +To write or update a composite action: +1. Create an `action.yml` file in a dedicated folder. +2. Define `runs: using: "composite"`. +3. Reference the action in a workflow using `uses: ./.github/actions/your-action-name`. + +### Special Conditions +You will often see special conditionals used in the pipeline: +- `if: always()`: Ensures a step runs regardless of whether previous steps failed. This is typically used to ensure test results and coverage reports are uploaded even when tests fail. +- `if: failure()`: Used to clean up environments or post error notifications only when a job crashes. + +## Good and Bad Practices + +### Good Practices +- **Timeout restrictions:** Always add `timeout-minutes: 20` (or an appropriate limit) to any step or job that could potentially hang (such as tests or network-dependent operations). This prevents stalled runners from consuming pipeline minutes indefinitely. +- **Explicit Secret Passing:** When calling reusable workflows, explicitly pass required secrets using `secrets: { MY_SECRET: ${{ secrets.MY_SECRET }} }`. Alternatively, use `secrets: inherit` if you trust the downstream workflow. + +### Bad Practices +- **Avoid `toJSON(secrets)`:** Never pass secrets to another workflow by serializing the entire secrets object with `toJSON(secrets)`. This is a security risk as it exposes all environment secrets to potentially untrusted outputs or logs. From ad275ee075fa444ba461314ec642ea9f240e1937 Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Mon, 19 Oct 2026 14:13:31 +0200 Subject: [PATCH 02/10] Address review comments and outdated references --- docs/developer-guide/170-ci-cd-pipeline.md | 219 +++++++++++++++++++++ docs/developer-guide/ci-cd-pipeline.md | 101 ---------- 2 files changed, 219 insertions(+), 101 deletions(-) create mode 100644 docs/developer-guide/170-ci-cd-pipeline.md delete mode 100644 docs/developer-guide/ci-cd-pipeline.md diff --git a/docs/developer-guide/170-ci-cd-pipeline.md b/docs/developer-guide/170-ci-cd-pipeline.md new file mode 100644 index 00000000..5caa0281 --- /dev/null +++ b/docs/developer-guide/170-ci-cd-pipeline.md @@ -0,0 +1,219 @@ +# CI/CD Pipeline + +This document describes the continuous integration and continuous deployment (CI/CD) pipeline of the [core repository](https://github.com/openremote/core), introducing the concepts and workflows added in our pipeline migration (introduced in PR [#2696](https://github.com/openremote/core/pull/2696)). The pipeline runs on GitHub Actions and its workflows are in the [`.github/workflows`](https://github.com/openremote/core/tree/main/.github/workflows) directory. + +## How to make changes to the pipeline and test them + +When modifying the pipeline, you should generally branch off `main` and test your changes by opening a Pull Request. Since the workflows run automatically for PRs, you can iterate on pipeline changes within the PR itself. + +Changes to `.yml` files do not count as code changes when [detecting changes](#detect-changes), so a PR that only changes workflows skips the build and test jobs. To run the jobs you changed, temporarily include a change to a source file in your PR. + +To test changes without polluting the main repository, you can: +1. Push to your own fork and test against custom branches. +2. Test dispatchable workflows manually. Note that if a workflow file has not been merged to `main` yet, you cannot trigger it from the GitHub Actions UI. Instead, you must use the [GitHub CLI](https://cli.github.com/) to trigger it on your feature branch: + ```bash + gh workflow run --ref + ``` + * **Note on new dispatchable workflows:** If you are adding a completely new workflow file, it won't be recognized by GitHub at all until it is registered. A common workaround is to temporarily add an `on: [push]` event to the workflow, push your branch so it runs once, and then remove the `push` event. After this, it will be available to trigger via the CLI. + +## "Environment" and "Job" Workflows + +The pipeline is structured using a separation of concerns between environments and jobs: + +- **Environment Workflows**: These are entry points that handle specific contexts. They act as the orchestrator for different triggers: + - `review.yml` for Pull Requests and the merge queue. + - `staging.yml` for pushes to `main` and release tags. It builds, tests and publishes the Docker images, Maven artifacts and npm packages. + - `deploy.yml` for manual [deployments](#deployments). + - `custom-review.yml`, `custom-staging.yml` and `custom-deploy.yml`, which [custom projects](#custom-projects) call from their own workflows. +- **Job Workflows**: These are reusable workflows (`workflow_call`) that contain the actual steps for specific tasks (like running tests, building dists, or publishing artifacts). Environment workflows call these job workflows, which prevents duplication. + +The previous pipeline, `ci_cd.yml`, is kept for custom projects that have not migrated to the new workflows yet. + +### How Reusable Workflows Work + +Job workflows are configured as reusable workflows. This allows them to be shared across multiple environment orchestrators. + +A job that calls a reusable workflow reports a status check for each job in that workflow, named ` / `. For example, the `unit-tests` job in `review.yml` is named `Test` and calls `unit-tests.yml`, whose job is named `Unit Tests`, so the check is `Test / Unit Tests`. Branch protection on `main` requires these checks by name. + +### Handling Skipped Jobs + +If the calling job is skipped, GitHub only reports a check for the calling job (e.g. `Test`), so required checks like `Test / Unit Tests` are never reported and the PR waits for them indefinitely. A skipped check does satisfy branch protection, so you need to handle skipping *inside* the reusable workflow itself rather than skipping the entire `workflow_call`. + +Each job workflow therefore has a `skip` input that its job checks, and the environment workflow passes the skip condition: + +```yaml +# review.yml +unit-tests: + name: Test + needs: detect-changes + if: ${{ !failure() && !cancelled() }} + uses: "./.github/workflows/unit-tests.yml" + with: + skip: ${{ needs.detect-changes.outputs.backend_any_modified != 'true' }} +``` + +```yaml +# unit-tests.yml +on: + workflow_call: + inputs: + skip: + type: boolean + description: Skip condition to allow branch protection checks to pass + default: false + +jobs: + unit-tests: + name: Unit Tests + if: ${{ !inputs.skip }} +``` + +The `!failure() && !cancelled()` condition on the calling job is explained in [Special Conditions](#special-conditions). + +### Sharding UI App Tests + +To speed up the UI app tests, `ui-app-tests.yml` splits them into four shards using a matrix job. Each shard runs a part of the tests in parallel using the Playwright `--shard` option and uploads its own test report. `fail-fast: false` lets the other shards finish when one fails, so you see all failing tests at once. + +A matrix job reports a check for each shard, but when the job is skipped it reports a single check without the matrix values. The checks of the individual shards therefore can't be required by branch protection. Instead, the `UI App Tests Status` job runs after the shards with `if: always()` and combines their results: it fails when a shard failed or was cancelled, and succeeds when the shards succeeded or were skipped. Branch protection requires this check instead of the shard checks. + +## Deployments + +Deployments are done by the reusable `deployment.yml` workflow. It builds the deployment, determines which environments to deploy to and runs the `.ci_cd/deploy.sh` script for each of them. When the workflow is triggered manually, it deploys to the environment given in the `ENVIRONMENT` input. Otherwise, it uses the `deploy` configuration in `.ci_cd/ci_cd.json`. + +In the core repository, `staging.yml` does not deploy. Deployments, for example to test servers, are triggered manually with the `deploy.yml` dispatch workflow. To reduce the deployment time significantly, it only runs the jobs a deployment requires (`install-dist` and `deployment`) and skips the tests. + +Custom projects deploy with `custom-staging.yml` and `custom-deploy.yml`, see [Custom Projects](#custom-projects). + +## Custom Projects + +[Custom projects](./040-creating-a-custom-project.md) reuse the pipeline by calling the following workflows of the core repository from their own workflows: + +| Workflow | Trigger in the custom project | Jobs | +|----------------------|-------------------------------|---------------------------------------------------------------------------------------------------------------------------| +| `custom-review.yml` | Pull Requests | Lint, build, unit tests and integration tests. | +| `custom-staging.yml` | Pushes | The same jobs as `custom-review.yml`, followed by a deployment when `.ci_cd/ci_cd.json` configures one for the pushed branch. | +| `custom-deploy.yml` | Manual (`workflow_dispatch`) | A deployment using `deployment.yml`. | + +The custom project workflows differ from the core workflows in the following ways: + +- **Build:** They build with `custom-install-dist.yml`, which only runs `./gradlew installDist`. The archive steps in `install-dist.yml` assume the layout of the core repository (e.g. `ui/util`), and no custom project job downloads those artifacts. The build includes the deployment, so a broken deployment build already fails in review. +- **Code coverage:** Not run yet, because `code-coverage.yml` requires JaCoCo outputs and the `combinedCoverageReport` Gradle task. +- **Manager version:** Deployments use the manager Docker image matching the `openremote` version in `gradle/libs.versions.toml`. Snapshot versions are not supported. +- **Deploy script:** When a custom project has no `.ci_cd/deploy.sh`, `deployment.yml` uses the script of the core repository at the same version. Up to 1.32.0, this script reads the `IS_MAIN_REPO` environment variable and the `openremote/.ci_cd` directory, so `deployment.yml` provides both. +- **Manual deployments:** Custom projects call `custom-deploy.yml` instead of `deploy.yml` or `deployment.yml`, so they don't depend on the inputs of `deployment.yml`, which we plan to change. It must be called from a `workflow_dispatch` workflow: + +```yaml +name: Deploy + +on: + workflow_dispatch: + inputs: + ENVIRONMENT: + description: "Environment to use (if any)" + CLEAN_INSTALL: + description: "Delete data before starting" + type: boolean + COMMIT: + description: "Repo branch or commit SHA to checkout" + OR_HOSTNAME: + description: "Host to deploy to (e.g. demo.openremote.app)" + OR_ADMIN_PASSWORD: + description: "Admin password override" + +jobs: + deploy: + name: Deploy + uses: openremote/core/.github/workflows/custom-deploy.yml@ + with: + INPUTS: ${{ toJSON(inputs) }} + secrets: inherit +``` + +Reference the core workflows by commit SHA rather than a branch like `main`, so changes to the core pipeline don't unexpectedly break custom projects. + +## Optimizing Jobs and Job Dependencies + +We utilize several strategies to ensure the pipeline is as fast and efficient as possible: + +### Detect Changes + +We use a "detect changes" job early in the workflow to determine which files have been modified. Subsequent jobs use the outputs of this job as their `skip` condition: + +- `backend_any_modified`: Unit and integration tests only run when backend code changed. +- `ui_any_modified`: UI component tests only run when UI code changed. +- `any_modified`: The build, the manager image and the UI app tests run for any backend or UI change, because the UI app tests run against the whole stack. +- `backend_spotless_any_modified` and `ui_spotless_any_modified`: Linting only runs when files checked by Spotless changed. + +For example, a PR that only touches backend Java code skips the UI component tests but still runs the UI app tests, and a PR that only changes Markdown files skips the build and all tests. + +### Minimal Dependency Installation + +Jobs should only install what they need. For example, `install-browsers.yml` installs just the `chromium-headless-shell` Playwright browser, at the Playwright version from `ui/test/package.json`, and shares it with the UI test jobs as an artifact. Likewise, jobs download the build outputs of `install-dist` instead of building them again. + +### Checking out with Git LFS + +Only use `lfs: true` in the `actions/checkout` step if the job actually requires files stored in Git LFS (like the `*.mbtiles` map tiles). Checking out LFS files adds significant time and bandwidth overhead, so it is omitted where unnecessary. + +### File Modified Metadata Tricks + +To avoid rebuilding components when passing artifacts between jobs, we update file modified metadata after downloading artifacts. For example, jobs that use the downloaded UI component build run `find . -name "*.tsbuildinfo" -exec touch {} +`. This makes the TypeScript compiler recognize that the downloaded build output is up to date, preventing unnecessary compilations. Run `npx tsc -b --verbose` to see why TypeScript considers a build out of date. + +## Artifacts: When and What to Include + +Artifacts are used to share files between isolated jobs. + +**The most important rule for artifacts is to split them where possible and create artifacts containing only the bare necessities for each specific job.** + +- **What to include:** Only the files downstream jobs need, such as build outputs, test reports and coverage files. Select specific build outputs, like `install-dist.yml` does for the UI component build, rather than uploading full `build` folders. Only upload a full folder when downstream jobs need all of it: the `or-manager-build` artifact contains `manager/build`, because the manager image build, the UI app tests and deployments use the manager distribution in it. Never upload `node_modules`. +- **When to use them:** Only pass artifacts when a downstream job strictly requires the output of an upstream job. Artifact upload/download takes time and consumes storage. Anyone with read access to the repository can download its artifacts, so never include secrets or other sensitive data. +- **Optimization through splitting:** Uploading separate, minimal artifacts (e.g., `install-dist.yml` saves the manager build, UI component build and UI app build as separate artifacts rather than one massive archive) ensures downstream jobs only download exactly what they need, significantly speeding up the pipeline. + +## Configuration: Gradle vs. Workflows + +A good rule of thumb is: +- **Use Gradle** to manage the internal build lifecycle, task graphs, dependencies, and code compilation. +- **Use Workflows** for orchestration, conditional job execution (e.g., detect changes), environment provisioning, and deployment. + +If a task can be optimized or defined within Gradle (such as skipping a task with `-x`), prefer that over complex bash scripts in the workflow. For example, `install-dist.yml` runs `./gradlew installDist -x :deployment:installDist`, because the core repository builds the deployment in `deployment.yml`. However, if skipping a process depends on the GitHub PR context (like changes to specific directories), manage it at the workflow level. + +## Caching Strategies + +We implement several caching strategies to speed up builds: +- **Gradle:** `actions/setup-java` with `cache: "gradle"` restores the Gradle dependencies and wrapper at the beginning of jobs, to skip downloading them again. +- **Docker Images:** Docker layer caching is used to avoid rebuilding unmodified layers. We utilize the GitHub Actions cache backend for Docker buildx (`type=gha`). + +## GitHub Concepts + +### Composite Actions + +Composite actions are defined under `.github/actions/` and act as reusable steps (unlike reusable workflows, which act as reusable jobs). We use them to keep things DRY. +To write or update a composite action: +1. Create an `action.yml` file in a dedicated folder under `.github/actions/`. +2. Define `runs: using: "composite"`. +3. Reference the action in a workflow using `uses: openremote/core/.github/actions/your-action-name@`. + +Don't reference actions with a local path like `./.github/actions/your-action-name` in workflows that custom projects also use. A local path is resolved in the checked-out repository, which is the custom project in that case. Because the reference contains a commit SHA, update it in the workflows after changing an action. + +### Special Conditions + +You will often see special conditionals used in the pipeline: +- `if: ${{ !failure() && !cancelled() }}`: Used on most jobs in the environment workflows. By default, a job is skipped when a job it needs was skipped. This condition lets it run anyway, unless a job it needs failed or the workflow run was cancelled. +- `if: always()`: Ensures a step or job runs regardless of whether previous steps or jobs failed. This is typically used to ensure test results and coverage reports are uploaded even when tests fail, and for the `UI App Tests Status` job. +- `if: failure()`: Runs a step only when a previous step in the job failed, or a job only when a job it needs failed. Use it to clean up environments or post error notifications. + +## Good and Bad Practices + +### Good Practices + +- **Timeout restrictions:** Always add `timeout-minutes: 20` (or an appropriate limit) to any step or job that could potentially hang (such as tests or network-dependent operations). This prevents stalled runners from consuming pipeline minutes indefinitely. +- **Explicit Secret Passing:** When calling reusable workflows, explicitly pass the required secrets: + ```yaml + secrets: + DOCKERHUB_USER: ${{ secrets._TEMP_DOCKERHUB_USER }} + DOCKERHUB_PASSWORD: ${{ secrets._TEMP_DOCKERHUB_PASSWORD }} + ``` + Alternatively, use `secrets: inherit` if you trust the downstream workflow and it requires all secrets, like `deployment.yml`. + +### Bad Practices + +- **Avoid `toJSON(secrets)`:** Never pass secrets to another workflow by serializing the entire secrets object with `toJSON(secrets)`. This is a security risk as it exposes all environment secrets to potentially untrusted outputs or logs. diff --git a/docs/developer-guide/ci-cd-pipeline.md b/docs/developer-guide/ci-cd-pipeline.md deleted file mode 100644 index 311cedfb..00000000 --- a/docs/developer-guide/ci-cd-pipeline.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -id: ci-cd-pipeline -title: CI/CD Pipeline -sidebar_label: CI/CD Pipeline ---- - -# CI/CD Pipeline - -This document describes the continuous integration and continuous deployment (CI/CD) pipeline for OpenRemote, introducing the concepts and workflows added in our recent pipeline migration (introduced in PR [#2696](https://github.com/openremote/openremote/pull/2696)). - -## How to make changes to the pipeline and test them - -When modifying the pipeline, you should generally branch off `master` and test your changes by opening a Pull Request. Since the workflows run automatically for PRs, you can iterate on pipeline changes within the PR itself. - -To test changes without polluting the main repository, you can: -1. Push to your own fork and test against custom branches. -2. Test dispatchable workflows manually. Note that if a workflow file has not been merged to `master` yet, you cannot trigger it from the GitHub Actions UI. Instead, you must use the [GitHub CLI](https://cli.github.com/) to trigger it on your feature branch: - ```bash - gh workflow run --ref - ``` - * **Note on new dispatchable workflows:** If you are adding a completely new workflow file, it won't be recognized by GitHub at all until it is registered. A common workaround is to temporarily add an `on: [push]` event to the workflow, push your branch so it runs once, and then remove the `push` event. After this, it will be available to trigger via the CLI. - -## "Environment" and "Job" Workflows - -The pipeline is structured using a separation of concerns between environments and jobs: - -- **Environment Workflows**: These are entry points that handle specific contexts (e.g., `review.yml` for Pull Requests, `staging.yml` for push to `master` and releases, `custom-review.yml` and `custom-staging.yml` for custom projects). They act as the orchestrator for different triggers. -- **Job Workflows**: These are reusable workflows (`workflow_call`) that contain the actual steps for specific tasks (like running tests, building dists, or publishing artifacts). Environment workflows call these job workflows, which prevents duplication. - -### How Reusable Workflows Work -Job workflows are configured as reusable workflows. This allows them to be shared across multiple environment orchestrators. - -**Handling Skipped Jobs:** -Because GitHub Actions treats skipped jobs differently in the status check context (a skipped job in a required chain might cause downstream dependencies to hang or fail incorrectly), you often need to handle skipping *inside* the reusable workflow itself rather than skipping the entire `workflow_call`. This is achieved by checking conditionals within the steps or the jobs of the reusable workflow to exit cleanly. - -## Deployments - -Deployments are handled by the `deploy.yml` dispatch workflow. This workflow can be triggered manually or via specific staging outcomes. For deployments to test servers, the setup has been minimized to reduce the deployment time significantly. - -## Optimizing Jobs and Job Dependencies - -We utilize several strategies to ensure the pipeline is as fast and efficient as possible: - -### Detect Changes -We use a "detect changes" job early in the workflow to determine which files have been modified. Subsequent jobs use the outputs of this job to conditionally skip themselves. This is why UI tests don't run if only backend Java code was touched, significantly speeding up the pipeline. - -### Minimal Dependency Installation -Jobs should only install what they need. For Node/Yarn-based projects, instead of running a full `yarn install` everywhere, we use focused installs, e.g., `yarn focus @openremote/util @openremote/test`. This installs the bare minimum dependencies for the scoped workspace, drastically reducing setup time. - -### Checking out with Git LFS -Only use `lfs: true` in the `actions/checkout` step if the job actually requires large files (like images, binaries, or specific test assets). Checking out LFS files adds significant time and bandwidth overhead, so it is omitted where unnecessary. - -### Build Tooling and File Modified Metadata Tricks -To avoid rebuilding components when passing artifacts between jobs, we use tricks like updating file modified metadata (e.g., using `touch`) after downloading artifacts. This tricks build tools (like Gradle) into recognizing that the downloaded artifact is the newest version, preventing the tool from re-triggering unnecessary compilations. - -## Artifacts: When and What to Include - -Artifacts are used to share files between isolated jobs. - -**The most important rule for artifacts is to split them where possible and create artifacts containing only the bare necessities for each specific job.** - -- **What to include:** Compiled distributions, test reports, and coverage files. Do not upload `node_modules` or full `build` folders; compress and selectively upload only the required distributions. -- **When to use them:** Only pass artifacts when a downstream job strictly requires the output of an upstream job. Artifact upload/download takes time and consumes storage. Avoid sharing sensitive data through public artifacts. -- **Optimization through splitting:** Uploading separate, minimal artifacts (e.g., saving the manager dist and UI dists as separate artifacts rather than one massive archive) ensures downstream jobs only download exactly what they need, significantly speeding up the pipeline. - -## Configuration: Gradle vs. Workflows - -A good rule of thumb is: -- **Use Gradle** to manage the internal build lifecycle, task graphs, dependencies, and code compilation. -- **Use Workflows** for orchestration, conditional job execution (e.g., detect changes), environment provisioning, and deployment. - -If a task can be optimized or defined within Gradle (such as skipping a task with `-x`), prefer that over complex bash scripts in the workflow. However, if skipping a process depends on the GitHub PR context (like changes to specific directories), manage it at the workflow level. - -## Caching Strategies - -We implement several caching strategies to speed up builds: -- **Docker Images:** Docker layer caching is used to avoid rebuilding unmodified layers. We utilize the GitHub Actions cache backend for Docker buildx. - - -## GitHub Concepts - -### Composite Actions -Composite actions are defined under `.github/actions/` and act as reusable steps (unlike reusable workflows, which act as reusable jobs). We use them to keep things DRY. -To write or update a composite action: -1. Create an `action.yml` file in a dedicated folder. -2. Define `runs: using: "composite"`. -3. Reference the action in a workflow using `uses: ./.github/actions/your-action-name`. - -### Special Conditions -You will often see special conditionals used in the pipeline: -- `if: always()`: Ensures a step runs regardless of whether previous steps failed. This is typically used to ensure test results and coverage reports are uploaded even when tests fail. -- `if: failure()`: Used to clean up environments or post error notifications only when a job crashes. - -## Good and Bad Practices - -### Good Practices -- **Timeout restrictions:** Always add `timeout-minutes: 20` (or an appropriate limit) to any step or job that could potentially hang (such as tests or network-dependent operations). This prevents stalled runners from consuming pipeline minutes indefinitely. -- **Explicit Secret Passing:** When calling reusable workflows, explicitly pass required secrets using `secrets: { MY_SECRET: ${{ secrets.MY_SECRET }} }`. Alternatively, use `secrets: inherit` if you trust the downstream workflow. - -### Bad Practices -- **Avoid `toJSON(secrets)`:** Never pass secrets to another workflow by serializing the entire secrets object with `toJSON(secrets)`. This is a security risk as it exposes all environment secrets to potentially untrusted outputs or logs. From aa20a86e7466b4a26846c4a8931be2a8f372a97e Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Mon, 19 Oct 2026 14:33:16 +0200 Subject: [PATCH 03/10] Add missing documentation and fix AI writing --- docs/developer-guide/170-ci-cd-pipeline.md | 33 ++++++++++++++++------ 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/docs/developer-guide/170-ci-cd-pipeline.md b/docs/developer-guide/170-ci-cd-pipeline.md index 5caa0281..e50ebb97 100644 --- a/docs/developer-guide/170-ci-cd-pipeline.md +++ b/docs/developer-guide/170-ci-cd-pipeline.md @@ -72,13 +72,13 @@ The `!failure() && !cancelled()` condition on the calling job is explained in [S ### Sharding UI App Tests -To speed up the UI app tests, `ui-app-tests.yml` splits them into four shards using a matrix job. Each shard runs a part of the tests in parallel using the Playwright `--shard` option and uploads its own test report. `fail-fast: false` lets the other shards finish when one fails, so you see all failing tests at once. +To speed up the UI app tests, `ui-app-tests.yml` splits them into shards using a matrix job (`shardIndex` and `shardCount`). Each shard runs a part of the tests in parallel using the Playwright `--shard` option and uploads its own test report. `fail-fast: false` lets the other shards finish when one fails, so you see all failing tests at once. A matrix job reports a check for each shard, but when the job is skipped it reports a single check without the matrix values. The checks of the individual shards therefore can't be required by branch protection. Instead, the `UI App Tests Status` job runs after the shards with `if: always()` and combines their results: it fails when a shard failed or was cancelled, and succeeds when the shards succeeded or were skipped. Branch protection requires this check instead of the shard checks. ## Deployments -Deployments are done by the reusable `deployment.yml` workflow. It builds the deployment, determines which environments to deploy to and runs the `.ci_cd/deploy.sh` script for each of them. When the workflow is triggered manually, it deploys to the environment given in the `ENVIRONMENT` input. Otherwise, it uses the `deploy` configuration in `.ci_cd/ci_cd.json`. +Deployments are done by the reusable `deployment.yml` workflow. It builds the deployment, determines which environments to deploy to and runs the `.ci_cd/deploy.sh` script for each of them. By default, it checks the type of event that triggered the workflow (for example a `push` to a specific branch) and deploys to the environments configured for that event in the `deploy` configuration in `.ci_cd/ci_cd.json`. When you dispatch a deployment manually, you choose the environment with the `ENVIRONMENT` input instead. In the core repository, `staging.yml` does not deploy. Deployments, for example to test servers, are triggered manually with the `deploy.yml` dispatch workflow. To reduce the deployment time significantly, it only runs the jobs a deployment requires (`install-dist` and `deployment`) and skips the tests. @@ -99,8 +99,8 @@ The custom project workflows differ from the core workflows in the following way - **Build:** They build with `custom-install-dist.yml`, which only runs `./gradlew installDist`. The archive steps in `install-dist.yml` assume the layout of the core repository (e.g. `ui/util`), and no custom project job downloads those artifacts. The build includes the deployment, so a broken deployment build already fails in review. - **Code coverage:** Not run yet, because `code-coverage.yml` requires JaCoCo outputs and the `combinedCoverageReport` Gradle task. - **Manager version:** Deployments use the manager Docker image matching the `openremote` version in `gradle/libs.versions.toml`. Snapshot versions are not supported. -- **Deploy script:** When a custom project has no `.ci_cd/deploy.sh`, `deployment.yml` uses the script of the core repository at the same version. Up to 1.32.0, this script reads the `IS_MAIN_REPO` environment variable and the `openremote/.ci_cd` directory, so `deployment.yml` provides both. -- **Manual deployments:** Custom projects call `custom-deploy.yml` instead of `deploy.yml` or `deployment.yml`, so they don't depend on the inputs of `deployment.yml`, which we plan to change. It must be called from a `workflow_dispatch` workflow: +- **Deploy script:** `deployment.yml` runs the `.ci_cd/deploy.sh` script of the core repository at the same `openremote` version. Up to 1.32.0, this script reads the `IS_MAIN_REPO` environment variable and the `openremote/.ci_cd` directory, so `deployment.yml` provides both. +- **Manual deployments:** Custom projects call `custom-deploy.yml` instead of `deploy.yml` or `deployment.yml`, so they don't depend on the inputs of `deployment.yml`. It must be called from a `workflow_dispatch` workflow: ```yaml name: Deploy @@ -144,19 +144,24 @@ We use a "detect changes" job early in the workflow to determine which files hav - `any_modified`: The build, the manager image and the UI app tests run for any backend or UI change, because the UI app tests run against the whole stack. - `backend_spotless_any_modified` and `ui_spotless_any_modified`: Linting only runs when files checked by Spotless changed. -For example, a PR that only touches backend Java code skips the UI component tests but still runs the UI app tests, and a PR that only changes Markdown files skips the build and all tests. +For example: +- A PR that only touches backend Java code skips the UI component tests, but still runs the UI app tests. +- A PR that only touches UI code skips the backend unit and integration tests, so only the build and the UI related jobs run. +- A PR that only changes Markdown files skips the build and all tests. ### Minimal Dependency Installation -Jobs should only install what they need. For example, `install-browsers.yml` installs just the `chromium-headless-shell` Playwright browser, at the Playwright version from `ui/test/package.json`, and shares it with the UI test jobs as an artifact. Likewise, jobs download the build outputs of `install-dist` instead of building them again. +Installing fewer dependencies is an optimization in itself, because every dependency a job installs adds download and setup time. Jobs should therefore only install what they need. For example, `install-browsers.yml` only installs the `chromium-headless-shell` Playwright browser (at the Playwright version from `ui/test/package.json`) instead of all Playwright browsers. + +Install shared dependencies once, early in the pipeline. `install-browsers.yml` downloads the Playwright browser once and runs in parallel with the build, before the UI component tests and the UI app test shards start. These jobs then get the browser from an artifact instead of each downloading it themselves. The download time of the Playwright browser differed a lot per shard, so downloading only once also made the pipeline more stable. Likewise, jobs download the build outputs of `install-dist` instead of building them again. ### Checking out with Git LFS -Only use `lfs: true` in the `actions/checkout` step if the job actually requires files stored in Git LFS (like the `*.mbtiles` map tiles). Checking out LFS files adds significant time and bandwidth overhead, so it is omitted where unnecessary. +Only use `lfs: true` in the `actions/checkout` step if the job actually requires files stored in Git LFS (like the `*.mbtiles` map tiles). Checking out LFS files adds significant time and bandwidth overhead, so it is omitted where possible. ### File Modified Metadata Tricks -To avoid rebuilding components when passing artifacts between jobs, we update file modified metadata after downloading artifacts. For example, jobs that use the downloaded UI component build run `find . -name "*.tsbuildinfo" -exec touch {} +`. This makes the TypeScript compiler recognize that the downloaded build output is up to date, preventing unnecessary compilations. Run `npx tsc -b --verbose` to see why TypeScript considers a build out of date. +To avoid rebuilding components when passing artifacts between jobs, we update file modified metadata after downloading artifacts. For example, jobs that use the downloaded UI component build run `find . -name "*.tsbuildinfo" -exec touch {} +`. This makes the TypeScript compiler recognize that the downloaded build output is up-to-date, preventing unnecessary compilations. Run `npx tsc -b --verbose` to see why TypeScript considers a build out of date. ## Artifacts: When and What to Include @@ -192,7 +197,17 @@ To write or update a composite action: 2. Define `runs: using: "composite"`. 3. Reference the action in a workflow using `uses: openremote/core/.github/actions/your-action-name@`. -Don't reference actions with a local path like `./.github/actions/your-action-name` in workflows that custom projects also use. A local path is resolved in the checked-out repository, which is the custom project in that case. Because the reference contains a commit SHA, update it in the workflows after changing an action. +Don't reference actions with a local path like `./.github/actions/your-action-name` in workflows that custom projects also use. A local path is resolved in the checked-out repository, which is the custom project in that case. + +:::warning + +After changing a composite action, update the commit SHA in the workflows that use it to the SHA of **the commit that changed the action** on your PR branch. Because PRs are squash merged, this SHA differs from the SHA of the commit that ends up in the history of the `main` branch, so it points to a dangling commit. This could become an issue if the repository is ever garbage collected. However, it is less cumbersome than waiting for the PR to be merged and then creating a follow-up PR to update the commit SHAs, or changing the merge rules to allow merging individual commits. + +If you reference a commit on `main` instead, Dependabot considers the reference to follow the `main` branch. Every time a new commit is merged, it opens a PR to update the SHA to the latest commit on `main`, even though the action didn't change (see [#3067](https://github.com/openremote/core/pull/3067), [#3081](https://github.com/openremote/core/pull/3081) and [#3087](https://github.com/openremote/core/pull/3087), fixed in [#3093](https://github.com/openremote/core/pull/3093)). + +::: + +Dependabot also updates the actions used inside our composite actions. When it does, add a commit to the Dependabot PR that updates the references in the workflows to the SHA of the Dependabot commit, which won't be part of the `main` history either (see [#3301](https://github.com/openremote/core/pull/3301)). ### Special Conditions From 5bf795fbca4f75acf03eefa4b6dbb76d27f04d8b Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Mon, 19 Oct 2026 14:42:28 +0200 Subject: [PATCH 04/10] Use admonition for note on testing dispatchable workflows --- docs/developer-guide/170-ci-cd-pipeline.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/developer-guide/170-ci-cd-pipeline.md b/docs/developer-guide/170-ci-cd-pipeline.md index e50ebb97..61d2ca70 100644 --- a/docs/developer-guide/170-ci-cd-pipeline.md +++ b/docs/developer-guide/170-ci-cd-pipeline.md @@ -14,7 +14,12 @@ To test changes without polluting the main repository, you can: ```bash gh workflow run --ref ``` - * **Note on new dispatchable workflows:** If you are adding a completely new workflow file, it won't be recognized by GitHub at all until it is registered. A common workaround is to temporarily add an `on: [push]` event to the workflow, push your branch so it runs once, and then remove the `push` event. After this, it will be available to trigger via the CLI. + + :::note + + If you are adding a completely new dispatchable workflow file, it won't be recognized by GitHub at all until it is registered. A common workaround is to temporarily add an `on: [push]` event to the workflow, push your branch so it runs once, and then remove the `push` event. After this, it will be available to trigger via the CLI. + + ::: ## "Environment" and "Job" Workflows From b9ec5063067676b40b5d4f437c0abb019d99af01 Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Mon, 19 Oct 2026 14:43:39 +0200 Subject: [PATCH 05/10] Don't repeat composite action updating rule --- docs/developer-guide/170-ci-cd-pipeline.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer-guide/170-ci-cd-pipeline.md b/docs/developer-guide/170-ci-cd-pipeline.md index 61d2ca70..b78bfd33 100644 --- a/docs/developer-guide/170-ci-cd-pipeline.md +++ b/docs/developer-guide/170-ci-cd-pipeline.md @@ -212,7 +212,7 @@ If you reference a commit on `main` instead, Dependabot considers the reference ::: -Dependabot also updates the actions used inside our composite actions. When it does, add a commit to the Dependabot PR that updates the references in the workflows to the SHA of the Dependabot commit, which won't be part of the `main` history either (see [#3301](https://github.com/openremote/core/pull/3301)). +Dependabot also updates the actions used inside our composite actions. When it does, add a commit to the Dependabot PR as described above (see [#3301](https://github.com/openremote/core/pull/3301)). ### Special Conditions From f3f00f52cd6c37fd9b7813231bebc7016855476b Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Mon, 19 Oct 2026 15:30:35 +0200 Subject: [PATCH 06/10] Restructure pipeline documentation page --- docs/developer-guide/170-ci-cd-pipeline.md | 176 +++++++++++---------- 1 file changed, 91 insertions(+), 85 deletions(-) diff --git a/docs/developer-guide/170-ci-cd-pipeline.md b/docs/developer-guide/170-ci-cd-pipeline.md index b78bfd33..df1203e5 100644 --- a/docs/developer-guide/170-ci-cd-pipeline.md +++ b/docs/developer-guide/170-ci-cd-pipeline.md @@ -1,27 +1,8 @@ -# CI/CD Pipeline +# CI/CD pipeline This document describes the continuous integration and continuous deployment (CI/CD) pipeline of the [core repository](https://github.com/openremote/core), introducing the concepts and workflows added in our pipeline migration (introduced in PR [#2696](https://github.com/openremote/core/pull/2696)). The pipeline runs on GitHub Actions and its workflows are in the [`.github/workflows`](https://github.com/openremote/core/tree/main/.github/workflows) directory. -## How to make changes to the pipeline and test them - -When modifying the pipeline, you should generally branch off `main` and test your changes by opening a Pull Request. Since the workflows run automatically for PRs, you can iterate on pipeline changes within the PR itself. - -Changes to `.yml` files do not count as code changes when [detecting changes](#detect-changes), so a PR that only changes workflows skips the build and test jobs. To run the jobs you changed, temporarily include a change to a source file in your PR. - -To test changes without polluting the main repository, you can: -1. Push to your own fork and test against custom branches. -2. Test dispatchable workflows manually. Note that if a workflow file has not been merged to `main` yet, you cannot trigger it from the GitHub Actions UI. Instead, you must use the [GitHub CLI](https://cli.github.com/) to trigger it on your feature branch: - ```bash - gh workflow run --ref - ``` - - :::note - - If you are adding a completely new dispatchable workflow file, it won't be recognized by GitHub at all until it is registered. A common workaround is to temporarily add an `on: [push]` event to the workflow, push your branch so it runs once, and then remove the `push` event. After this, it will be available to trigger via the CLI. - - ::: - -## "Environment" and "Job" Workflows +## Overview of the workflows The pipeline is structured using a separation of concerns between environments and jobs: @@ -34,13 +15,33 @@ The pipeline is structured using a separation of concerns between environments a The previous pipeline, `ci_cd.yml`, is kept for custom projects that have not migrated to the new workflows yet. -### How Reusable Workflows Work +### How reusable workflows work Job workflows are configured as reusable workflows. This allows them to be shared across multiple environment orchestrators. A job that calls a reusable workflow reports a status check for each job in that workflow, named ` / `. For example, the `unit-tests` job in `review.yml` is named `Test` and calls `unit-tests.yml`, whose job is named `Unit Tests`, so the check is `Test / Unit Tests`. Branch protection on `main` requires these checks by name. -### Handling Skipped Jobs +### Composite actions + +Composite actions are defined under `.github/actions/` and act as reusable steps (unlike reusable workflows, which act as reusable jobs). We use them to keep things DRY. + +## How the pipeline works + +### Detect changes + +We use a "detect changes" job early in the workflow to determine which files have been modified. Subsequent jobs use the outputs of this job as their `skip` condition: + +- `backend_any_modified`: Unit and integration tests only run when backend code changed. +- `ui_any_modified`: UI component tests only run when UI code changed. +- `any_modified`: The build, the manager image and the UI app tests run for any backend or UI change, because the UI app tests run against the whole stack. +- `backend_spotless_any_modified` and `ui_spotless_any_modified`: Linting only runs when files checked by Spotless changed. + +For example: +- A PR that only touches backend Java code skips the UI component tests, but still runs the UI app tests. +- A PR that only touches UI code skips the backend unit and integration tests, so only the build and the UI related jobs run. +- A PR that only changes Markdown files skips the build and all tests. + +### Handling skipped jobs If the calling job is skipped, GitHub only reports a check for the calling job (e.g. `Test`), so required checks like `Test / Unit Tests` are never reported and the PR waits for them indefinitely. A skipped check does satisfy branch protection, so you need to handle skipping *inside* the reusable workflow itself rather than skipping the entire `workflow_call`. @@ -73,23 +74,30 @@ jobs: if: ${{ !inputs.skip }} ``` -The `!failure() && !cancelled()` condition on the calling job is explained in [Special Conditions](#special-conditions). +The `!failure() && !cancelled()` condition on the calling job is explained in [Special conditions](#special-conditions). -### Sharding UI App Tests +### Special conditions + +You will often see special conditionals used in the pipeline: +- `if: ${{ !failure() && !cancelled() }}`: Used on most jobs in the environment workflows. By default, a job is skipped when a job it needs was skipped. This condition lets it run anyway, unless a job it needs failed or the workflow run was cancelled. +- `if: always()`: Ensures a step or job runs regardless of whether previous steps or jobs failed. This is typically used to ensure test results and coverage reports are uploaded even when tests fail, and for the `UI App Tests Status` job. +- `if: failure()`: Runs a step only when a previous step in the job failed, or a job only when a job it needs failed. Use it to clean up environments or post error notifications. + +### Sharding UI app tests To speed up the UI app tests, `ui-app-tests.yml` splits them into shards using a matrix job (`shardIndex` and `shardCount`). Each shard runs a part of the tests in parallel using the Playwright `--shard` option and uploads its own test report. `fail-fast: false` lets the other shards finish when one fails, so you see all failing tests at once. A matrix job reports a check for each shard, but when the job is skipped it reports a single check without the matrix values. The checks of the individual shards therefore can't be required by branch protection. Instead, the `UI App Tests Status` job runs after the shards with `if: always()` and combines their results: it fails when a shard failed or was cancelled, and succeeds when the shards succeeded or were skipped. Branch protection requires this check instead of the shard checks. -## Deployments +### Deployments Deployments are done by the reusable `deployment.yml` workflow. It builds the deployment, determines which environments to deploy to and runs the `.ci_cd/deploy.sh` script for each of them. By default, it checks the type of event that triggered the workflow (for example a `push` to a specific branch) and deploys to the environments configured for that event in the `deploy` configuration in `.ci_cd/ci_cd.json`. When you dispatch a deployment manually, you choose the environment with the `ENVIRONMENT` input instead. In the core repository, `staging.yml` does not deploy. Deployments, for example to test servers, are triggered manually with the `deploy.yml` dispatch workflow. To reduce the deployment time significantly, it only runs the jobs a deployment requires (`install-dist` and `deployment`) and skips the tests. -Custom projects deploy with `custom-staging.yml` and `custom-deploy.yml`, see [Custom Projects](#custom-projects). +Custom projects deploy with `custom-staging.yml` and `custom-deploy.yml`, see [Custom projects](#custom-projects). -## Custom Projects +## Custom projects [Custom projects](./040-creating-a-custom-project.md) reuse the pipeline by calling the following workflows of the core repository from their own workflows: @@ -99,13 +107,20 @@ Custom projects deploy with `custom-staging.yml` and `custom-deploy.yml`, see [C | `custom-staging.yml` | Pushes | The same jobs as `custom-review.yml`, followed by a deployment when `.ci_cd/ci_cd.json` configures one for the pushed branch. | | `custom-deploy.yml` | Manual (`workflow_dispatch`) | A deployment using `deployment.yml`. | +Reference the core workflows by commit SHA rather than a branch like `main`, so changes to the core pipeline don't unexpectedly break custom projects. + +### Differences from the core workflows + The custom project workflows differ from the core workflows in the following ways: - **Build:** They build with `custom-install-dist.yml`, which only runs `./gradlew installDist`. The archive steps in `install-dist.yml` assume the layout of the core repository (e.g. `ui/util`), and no custom project job downloads those artifacts. The build includes the deployment, so a broken deployment build already fails in review. - **Code coverage:** Not run yet, because `code-coverage.yml` requires JaCoCo outputs and the `combinedCoverageReport` Gradle task. - **Manager version:** Deployments use the manager Docker image matching the `openremote` version in `gradle/libs.versions.toml`. Snapshot versions are not supported. - **Deploy script:** `deployment.yml` runs the `.ci_cd/deploy.sh` script of the core repository at the same `openremote` version. Up to 1.32.0, this script reads the `IS_MAIN_REPO` environment variable and the `openremote/.ci_cd` directory, so `deployment.yml` provides both. -- **Manual deployments:** Custom projects call `custom-deploy.yml` instead of `deploy.yml` or `deployment.yml`, so they don't depend on the inputs of `deployment.yml`. It must be called from a `workflow_dispatch` workflow: + +### Manual deployments + +Custom projects call `custom-deploy.yml` instead of `deploy.yml` or `deployment.yml`, so they don't depend on the inputs of `deployment.yml`. It must be called from a `workflow_dispatch` workflow: ```yaml name: Deploy @@ -134,51 +149,49 @@ jobs: secrets: inherit ``` -Reference the core workflows by commit SHA rather than a branch like `main`, so changes to the core pipeline don't unexpectedly break custom projects. - -## Optimizing Jobs and Job Dependencies +## Making changes to the pipeline -We utilize several strategies to ensure the pipeline is as fast and efficient as possible: +### Testing changes -### Detect Changes +When modifying the pipeline, you should generally branch off `main` and test your changes by opening a Pull Request. Since the workflows run automatically for PRs, you can iterate on pipeline changes within the PR itself. -We use a "detect changes" job early in the workflow to determine which files have been modified. Subsequent jobs use the outputs of this job as their `skip` condition: +Changes to `.yml` files do not count as code changes when [detecting changes](#detect-changes), so a PR that only changes workflows skips the build and test jobs. To run the jobs you changed, temporarily include a change to a source file in your PR. -- `backend_any_modified`: Unit and integration tests only run when backend code changed. -- `ui_any_modified`: UI component tests only run when UI code changed. -- `any_modified`: The build, the manager image and the UI app tests run for any backend or UI change, because the UI app tests run against the whole stack. -- `backend_spotless_any_modified` and `ui_spotless_any_modified`: Linting only runs when files checked by Spotless changed. +To test changes without polluting the main repository, you can: +1. Push to your own fork and test against custom branches. +2. Test dispatchable workflows manually. Note that if a workflow file has not been merged to `main` yet, you cannot trigger it from the GitHub Actions UI. Instead, you must use the [GitHub CLI](https://cli.github.com/) to trigger it on your feature branch: + ```bash + gh workflow run --ref + ``` -For example: -- A PR that only touches backend Java code skips the UI component tests, but still runs the UI app tests. -- A PR that only touches UI code skips the backend unit and integration tests, so only the build and the UI related jobs run. -- A PR that only changes Markdown files skips the build and all tests. + :::note -### Minimal Dependency Installation + If you are adding a completely new dispatchable workflow file, it won't be recognized by GitHub at all until it is registered. A common workaround is to temporarily add an `on: [push]` event to the workflow, push your branch so it runs once, and then remove the `push` event. After this, it will be available to trigger via the CLI. -Installing fewer dependencies is an optimization in itself, because every dependency a job installs adds download and setup time. Jobs should therefore only install what they need. For example, `install-browsers.yml` only installs the `chromium-headless-shell` Playwright browser (at the Playwright version from `ui/test/package.json`) instead of all Playwright browsers. + ::: -Install shared dependencies once, early in the pipeline. `install-browsers.yml` downloads the Playwright browser once and runs in parallel with the build, before the UI component tests and the UI app test shards start. These jobs then get the browser from an artifact instead of each downloading it themselves. The download time of the Playwright browser differed a lot per shard, so downloading only once also made the pipeline more stable. Likewise, jobs download the build outputs of `install-dist` instead of building them again. +### Writing and updating composite actions -### Checking out with Git LFS +To write or update a composite action: +1. Create an `action.yml` file in a dedicated folder under `.github/actions/`. +2. Define `runs: using: "composite"`. +3. Reference the action in a workflow using `uses: openremote/core/.github/actions/your-action-name@`. -Only use `lfs: true` in the `actions/checkout` step if the job actually requires files stored in Git LFS (like the `*.mbtiles` map tiles). Checking out LFS files adds significant time and bandwidth overhead, so it is omitted where possible. +Don't reference actions with a local path like `./.github/actions/your-action-name` in workflows that custom projects also use. A local path is resolved in the checked-out repository, which is the custom project in that case. -### File Modified Metadata Tricks +:::warning -To avoid rebuilding components when passing artifacts between jobs, we update file modified metadata after downloading artifacts. For example, jobs that use the downloaded UI component build run `find . -name "*.tsbuildinfo" -exec touch {} +`. This makes the TypeScript compiler recognize that the downloaded build output is up-to-date, preventing unnecessary compilations. Run `npx tsc -b --verbose` to see why TypeScript considers a build out of date. +After changing a composite action, update the commit SHA in the workflows that use it to the SHA of **the commit that changed the action** on your PR branch. Because PRs are squash merged, this SHA differs from the SHA of the commit that ends up in the history of the `main` branch, so it points to a dangling commit. This could become an issue if the repository is ever garbage collected. However, it is less cumbersome than waiting for the PR to be merged and then creating a follow-up PR to update the commit SHAs, or changing the merge rules to allow merging individual commits. -## Artifacts: When and What to Include +If you reference a commit on `main` instead, Dependabot considers the reference to follow the `main` branch. Every time a new commit is merged, it opens a PR to update the SHA to the latest commit on `main`, even though the action didn't change (see [#3067](https://github.com/openremote/core/pull/3067), [#3081](https://github.com/openremote/core/pull/3081) and [#3087](https://github.com/openremote/core/pull/3087), fixed in [#3093](https://github.com/openremote/core/pull/3093)). -Artifacts are used to share files between isolated jobs. +::: -**The most important rule for artifacts is to split them where possible and create artifacts containing only the bare necessities for each specific job.** +Dependabot also updates the actions used inside our composite actions. When it does, add a commit to the Dependabot PR as described above (see [#3301](https://github.com/openremote/core/pull/3301)). -- **What to include:** Only the files downstream jobs need, such as build outputs, test reports and coverage files. Select specific build outputs, like `install-dist.yml` does for the UI component build, rather than uploading full `build` folders. Only upload a full folder when downstream jobs need all of it: the `or-manager-build` artifact contains `manager/build`, because the manager image build, the UI app tests and deployments use the manager distribution in it. Never upload `node_modules`. -- **When to use them:** Only pass artifacts when a downstream job strictly requires the output of an upstream job. Artifact upload/download takes time and consumes storage. Anyone with read access to the repository can download its artifacts, so never include secrets or other sensitive data. -- **Optimization through splitting:** Uploading separate, minimal artifacts (e.g., `install-dist.yml` saves the manager build, UI component build and UI app build as separate artifacts rather than one massive archive) ensures downstream jobs only download exactly what they need, significantly speeding up the pipeline. +## Guidelines -## Configuration: Gradle vs. Workflows +### Gradle vs. workflows A good rule of thumb is: - **Use Gradle** to manage the internal build lifecycle, task graphs, dependencies, and code compilation. @@ -186,46 +199,42 @@ A good rule of thumb is: If a task can be optimized or defined within Gradle (such as skipping a task with `-x`), prefer that over complex bash scripts in the workflow. For example, `install-dist.yml` runs `./gradlew installDist -x :deployment:installDist`, because the core repository builds the deployment in `deployment.yml`. However, if skipping a process depends on the GitHub PR context (like changes to specific directories), manage it at the workflow level. -## Caching Strategies +### Minimal dependency installation -We implement several caching strategies to speed up builds: -- **Gradle:** `actions/setup-java` with `cache: "gradle"` restores the Gradle dependencies and wrapper at the beginning of jobs, to skip downloading them again. -- **Docker Images:** Docker layer caching is used to avoid rebuilding unmodified layers. We utilize the GitHub Actions cache backend for Docker buildx (`type=gha`). +Installing fewer dependencies is an optimization in itself, because every dependency a job installs adds download and setup time. Jobs should therefore only install what they need. For example, `install-browsers.yml` only installs the `chromium-headless-shell` Playwright browser (at the Playwright version from `ui/test/package.json`) instead of all Playwright browsers. -## GitHub Concepts +Install shared dependencies once, early in the pipeline. `install-browsers.yml` downloads the Playwright browser once and runs in parallel with the build, before the UI component tests and the UI app test shards start. These jobs then get the browser from an artifact instead of each downloading it themselves. The download time of the Playwright browser differed a lot per shard, so downloading only once also made the pipeline more stable. Likewise, jobs download the build outputs of `install-dist` instead of building them again. -### Composite Actions +### Checking out with Git LFS -Composite actions are defined under `.github/actions/` and act as reusable steps (unlike reusable workflows, which act as reusable jobs). We use them to keep things DRY. -To write or update a composite action: -1. Create an `action.yml` file in a dedicated folder under `.github/actions/`. -2. Define `runs: using: "composite"`. -3. Reference the action in a workflow using `uses: openremote/core/.github/actions/your-action-name@`. +Only use `lfs: true` in the `actions/checkout` step if the job actually requires files stored in Git LFS (like the `*.mbtiles` map tiles). Checking out LFS files adds significant time and bandwidth overhead, so it is omitted where possible. -Don't reference actions with a local path like `./.github/actions/your-action-name` in workflows that custom projects also use. A local path is resolved in the checked-out repository, which is the custom project in that case. +### Artifacts -:::warning +Artifacts are used to share files between isolated jobs. -After changing a composite action, update the commit SHA in the workflows that use it to the SHA of **the commit that changed the action** on your PR branch. Because PRs are squash merged, this SHA differs from the SHA of the commit that ends up in the history of the `main` branch, so it points to a dangling commit. This could become an issue if the repository is ever garbage collected. However, it is less cumbersome than waiting for the PR to be merged and then creating a follow-up PR to update the commit SHAs, or changing the merge rules to allow merging individual commits. +**The most important rule for artifacts is to split them where possible and create artifacts containing only the bare necessities for each specific job.** -If you reference a commit on `main` instead, Dependabot considers the reference to follow the `main` branch. Every time a new commit is merged, it opens a PR to update the SHA to the latest commit on `main`, even though the action didn't change (see [#3067](https://github.com/openremote/core/pull/3067), [#3081](https://github.com/openremote/core/pull/3081) and [#3087](https://github.com/openremote/core/pull/3087), fixed in [#3093](https://github.com/openremote/core/pull/3093)). +- **What to include:** Only the files downstream jobs need, such as build outputs, test reports and coverage files. Select specific build outputs, like `install-dist.yml` does for the UI component build, rather than uploading full `build` folders. Only upload a full folder when downstream jobs need all of it: the `or-manager-build` artifact contains `manager/build`, because the manager image build, the UI app tests and deployments use the manager distribution in it. Never upload `node_modules`. +- **When to use them:** Only pass artifacts when a downstream job strictly requires the output of an upstream job. Artifact upload/download takes time and consumes storage. Anyone with read access to the repository can download its artifacts, so never include secrets or other sensitive data. +- **Optimization through splitting:** Uploading separate, minimal artifacts (e.g., `install-dist.yml` saves the manager build, UI component build and UI app build as separate artifacts rather than one massive archive) ensures downstream jobs only download exactly what they need, significantly speeding up the pipeline. -::: +### File modified metadata tricks -Dependabot also updates the actions used inside our composite actions. When it does, add a commit to the Dependabot PR as described above (see [#3301](https://github.com/openremote/core/pull/3301)). +To avoid rebuilding components when passing artifacts between jobs, we update file modified metadata after downloading artifacts. For example, jobs that use the downloaded UI component build run `find . -name "*.tsbuildinfo" -exec touch {} +`. This makes the TypeScript compiler recognize that the downloaded build output is up-to-date, preventing unnecessary compilations. Run `npx tsc -b --verbose` to see why TypeScript considers a build out of date. -### Special Conditions +### Caching -You will often see special conditionals used in the pipeline: -- `if: ${{ !failure() && !cancelled() }}`: Used on most jobs in the environment workflows. By default, a job is skipped when a job it needs was skipped. This condition lets it run anyway, unless a job it needs failed or the workflow run was cancelled. -- `if: always()`: Ensures a step or job runs regardless of whether previous steps or jobs failed. This is typically used to ensure test results and coverage reports are uploaded even when tests fail, and for the `UI App Tests Status` job. -- `if: failure()`: Runs a step only when a previous step in the job failed, or a job only when a job it needs failed. Use it to clean up environments or post error notifications. +We implement several caching strategies to speed up builds: +- **Gradle:** `actions/setup-java` with `cache: "gradle"` restores the Gradle dependencies and wrapper at the beginning of jobs, to skip downloading them again. +- **Docker Images:** Docker layer caching is used to avoid rebuilding unmodified layers. We utilize the GitHub Actions cache backend for Docker buildx (`type=gha`). -## Good and Bad Practices +### Timeouts -### Good Practices +Always add `timeout-minutes: 20` (or an appropriate limit) to any step or job that could potentially hang (such as tests or network-dependent operations). This prevents stalled runners from consuming pipeline minutes indefinitely. + +### Secrets -- **Timeout restrictions:** Always add `timeout-minutes: 20` (or an appropriate limit) to any step or job that could potentially hang (such as tests or network-dependent operations). This prevents stalled runners from consuming pipeline minutes indefinitely. - **Explicit Secret Passing:** When calling reusable workflows, explicitly pass the required secrets: ```yaml secrets: @@ -233,7 +242,4 @@ You will often see special conditionals used in the pipeline: DOCKERHUB_PASSWORD: ${{ secrets._TEMP_DOCKERHUB_PASSWORD }} ``` Alternatively, use `secrets: inherit` if you trust the downstream workflow and it requires all secrets, like `deployment.yml`. - -### Bad Practices - - **Avoid `toJSON(secrets)`:** Never pass secrets to another workflow by serializing the entire secrets object with `toJSON(secrets)`. This is a security risk as it exposes all environment secrets to potentially untrusted outputs or logs. From 82b296b03ab9665d9f9c99ca33a6043b0ae10a71 Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Mon, 19 Oct 2026 15:31:01 +0200 Subject: [PATCH 07/10] Fix AI writing --- docs/developer-guide/170-ci-cd-pipeline.md | 25 ++++++++++++++-------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/docs/developer-guide/170-ci-cd-pipeline.md b/docs/developer-guide/170-ci-cd-pipeline.md index df1203e5..5d6a8258 100644 --- a/docs/developer-guide/170-ci-cd-pipeline.md +++ b/docs/developer-guide/170-ci-cd-pipeline.md @@ -1,12 +1,12 @@ # CI/CD pipeline -This document describes the continuous integration and continuous deployment (CI/CD) pipeline of the [core repository](https://github.com/openremote/core), introducing the concepts and workflows added in our pipeline migration (introduced in PR [#2696](https://github.com/openremote/core/pull/2696)). The pipeline runs on GitHub Actions and its workflows are in the [`.github/workflows`](https://github.com/openremote/core/tree/main/.github/workflows) directory. +This document describes the continuous integration and continuous deployment (CI/CD) pipeline of the [core repository](https://github.com/openremote/core), introducing the concepts and workflows added in our pipeline migration ([#2696](https://github.com/openremote/core/pull/2696)). The pipeline runs on GitHub Actions and its workflows are in the [`.github/workflows`](https://github.com/openremote/core/tree/main/.github/workflows) directory. ## Overview of the workflows The pipeline is structured using a separation of concerns between environments and jobs: -- **Environment Workflows**: These are entry points that handle specific contexts. They act as the orchestrator for different triggers: +- **Environment Workflows**: These are context specific workflows that act as the orchestrator for different triggers: - `review.yml` for Pull Requests and the merge queue. - `staging.yml` for pushes to `main` and release tags. It builds, tests and publishes the Docker images, Maven artifacts and npm packages. - `deploy.yml` for manual [deployments](#deployments). @@ -17,7 +17,7 @@ The previous pipeline, `ci_cd.yml`, is kept for custom projects that have not mi ### How reusable workflows work -Job workflows are configured as reusable workflows. This allows them to be shared across multiple environment orchestrators. +Job workflows are configured as reusable workflows. This allows them to be shared across multiple environments. A job that calls a reusable workflow reports a status check for each job in that workflow, named ` / `. For example, the `unit-tests` job in `review.yml` is named `Test` and calls `unit-tests.yml`, whose job is named `Unit Tests`, so the check is `Test / Unit Tests`. Branch protection on `main` requires these checks by name. @@ -37,13 +37,14 @@ We use a "detect changes" job early in the workflow to determine which files hav - `backend_spotless_any_modified` and `ui_spotless_any_modified`: Linting only runs when files checked by Spotless changed. For example: + - A PR that only touches backend Java code skips the UI component tests, but still runs the UI app tests. - A PR that only touches UI code skips the backend unit and integration tests, so only the build and the UI related jobs run. - A PR that only changes Markdown files skips the build and all tests. ### Handling skipped jobs -If the calling job is skipped, GitHub only reports a check for the calling job (e.g. `Test`), so required checks like `Test / Unit Tests` are never reported and the PR waits for them indefinitely. A skipped check does satisfy branch protection, so you need to handle skipping *inside* the reusable workflow itself rather than skipping the entire `workflow_call`. +If the calling job is skipped, GitHub only reports a check for the calling job (e.g. `Test`), so required checks like `Test / Unit Tests` are never reported and the PR waits for them indefinitely. A skipped check does satisfy branch protection, so you need to handle skipping _inside_ the reusable workflow itself rather than skipping the entire `workflow_call`. Each job workflow therefore has a `skip` input that its job checks, and the environment workflow passes the skip condition: @@ -79,6 +80,7 @@ The `!failure() && !cancelled()` condition on the calling job is explained in [S ### Special conditions You will often see special conditionals used in the pipeline: + - `if: ${{ !failure() && !cancelled() }}`: Used on most jobs in the environment workflows. By default, a job is skipped when a job it needs was skipped. This condition lets it run anyway, unless a job it needs failed or the workflow run was cancelled. - `if: always()`: Ensures a step or job runs regardless of whether previous steps or jobs failed. This is typically used to ensure test results and coverage reports are uploaded even when tests fail, and for the `UI App Tests Status` job. - `if: failure()`: Runs a step only when a previous step in the job failed, or a job only when a job it needs failed. Use it to clean up environments or post error notifications. @@ -101,11 +103,11 @@ Custom projects deploy with `custom-staging.yml` and `custom-deploy.yml`, see [C [Custom projects](./040-creating-a-custom-project.md) reuse the pipeline by calling the following workflows of the core repository from their own workflows: -| Workflow | Trigger in the custom project | Jobs | -|----------------------|-------------------------------|---------------------------------------------------------------------------------------------------------------------------| -| `custom-review.yml` | Pull Requests | Lint, build, unit tests and integration tests. | -| `custom-staging.yml` | Pushes | The same jobs as `custom-review.yml`, followed by a deployment when `.ci_cd/ci_cd.json` configures one for the pushed branch. | -| `custom-deploy.yml` | Manual (`workflow_dispatch`) | A deployment using `deployment.yml`. | +| Workflow | Trigger in the custom project | Stages | +| -------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------ | +| `custom-review.yml` | Pull Requests | Lint, build, and test. | +| `custom-staging.yml` | Pushes | The same as `custom-review.yml`, followed by a deployment when `.ci_cd/ci_cd.json` configures one for the pushed branch. | +| `custom-deploy.yml` | Manual (`workflow_dispatch`) | A deployment using `deployment.yml`. | Reference the core workflows by commit SHA rather than a branch like `main`, so changes to the core pipeline don't unexpectedly break custom projects. @@ -158,8 +160,10 @@ When modifying the pipeline, you should generally branch off `main` and test you Changes to `.yml` files do not count as code changes when [detecting changes](#detect-changes), so a PR that only changes workflows skips the build and test jobs. To run the jobs you changed, temporarily include a change to a source file in your PR. To test changes without polluting the main repository, you can: + 1. Push to your own fork and test against custom branches. 2. Test dispatchable workflows manually. Note that if a workflow file has not been merged to `main` yet, you cannot trigger it from the GitHub Actions UI. Instead, you must use the [GitHub CLI](https://cli.github.com/) to trigger it on your feature branch: + ```bash gh workflow run --ref ``` @@ -173,6 +177,7 @@ To test changes without polluting the main repository, you can: ### Writing and updating composite actions To write or update a composite action: + 1. Create an `action.yml` file in a dedicated folder under `.github/actions/`. 2. Define `runs: using: "composite"`. 3. Reference the action in a workflow using `uses: openremote/core/.github/actions/your-action-name@`. @@ -194,6 +199,7 @@ Dependabot also updates the actions used inside our composite actions. When it d ### Gradle vs. workflows A good rule of thumb is: + - **Use Gradle** to manage the internal build lifecycle, task graphs, dependencies, and code compilation. - **Use Workflows** for orchestration, conditional job execution (e.g., detect changes), environment provisioning, and deployment. @@ -226,6 +232,7 @@ To avoid rebuilding components when passing artifacts between jobs, we update fi ### Caching We implement several caching strategies to speed up builds: + - **Gradle:** `actions/setup-java` with `cache: "gradle"` restores the Gradle dependencies and wrapper at the beginning of jobs, to skip downloading them again. - **Docker Images:** Docker layer caching is used to avoid rebuilding unmodified layers. We utilize the GitHub Actions cache backend for Docker buildx (`type=gha`). From fc2c6d21d7302f264f506450d8415c1964b744f2 Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Mon, 19 Oct 2026 15:58:26 +0200 Subject: [PATCH 08/10] Address review comments --- docs/developer-guide/170-ci-cd-pipeline.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/developer-guide/170-ci-cd-pipeline.md b/docs/developer-guide/170-ci-cd-pipeline.md index 5d6a8258..a20d7f84 100644 --- a/docs/developer-guide/170-ci-cd-pipeline.md +++ b/docs/developer-guide/170-ci-cd-pipeline.md @@ -34,19 +34,19 @@ We use a "detect changes" job early in the workflow to determine which files hav - `backend_any_modified`: Unit and integration tests only run when backend code changed. - `ui_any_modified`: UI component tests only run when UI code changed. - `any_modified`: The build, the manager image and the UI app tests run for any backend or UI change, because the UI app tests run against the whole stack. -- `backend_spotless_any_modified` and `ui_spotless_any_modified`: Linting only runs when files checked by Spotless changed. +- `backend_spotless_any_modified` and `ui_spotless_any_modified`: The Spotless checks only run when files checked by Spotless changed. For example: - A PR that only touches backend Java code skips the UI component tests, but still runs the UI app tests. - A PR that only touches UI code skips the backend unit and integration tests, so only the build and the UI related jobs run. -- A PR that only changes Markdown files skips the build and all tests. +- A PR that only changes Markdown files skips the build and all tests, but still runs the Spotless checks. ### Handling skipped jobs -If the calling job is skipped, GitHub only reports a check for the calling job (e.g. `Test`), so required checks like `Test / Unit Tests` are never reported and the PR waits for them indefinitely. A skipped check does satisfy branch protection, so you need to handle skipping _inside_ the reusable workflow itself rather than skipping the entire `workflow_call`. +If the calling job is skipped, GitHub only reports a check for the calling job (e.g. `Test`), so required checks like `Test / Unit Tests` are never reported and the PR waits for them indefinitely. A required check that is reported as skipped does satisfy branch protection, so you need to handle skipping _inside_ the reusable workflow itself rather than skipping the entire `workflow_call`. -Each job workflow therefore has a `skip` input that its job checks, and the environment workflow passes the skip condition: +Each job workflow that can be skipped therefore has a `skip` input that its job checks, and the environment workflow passes the skip condition: ```yaml # review.yml @@ -93,7 +93,7 @@ A matrix job reports a check for each shard, but when the job is skipped it repo ### Deployments -Deployments are done by the reusable `deployment.yml` workflow. It builds the deployment, determines which environments to deploy to and runs the `.ci_cd/deploy.sh` script for each of them. By default, it checks the type of event that triggered the workflow (for example a `push` to a specific branch) and deploys to the environments configured for that event in the `deploy` configuration in `.ci_cd/ci_cd.json`. When you dispatch a deployment manually, you choose the environment with the `ENVIRONMENT` input instead. +Deployments are done by the reusable `deployment.yml` workflow. It builds the deployment, determines which environments to deploy to and runs the `.ci_cd/deploy.sh` script for each of them. By default, it checks the type of event that triggered the workflow (for example a `push` to a specific branch) and deploys to the environments configured for that event in the `deploy` configuration in `.ci_cd/ci_cd.json`. When you dispatch a deployment manually on a branch, you choose the environment with the `ENVIRONMENT` input instead. When you dispatch it on a tag, the `workflow_dispatch` deploy entry in `.ci_cd/ci_cd.json` is used instead of the `ENVIRONMENT` and `MANAGER_TAG` inputs. In the core repository, `staging.yml` does not deploy. Deployments, for example to test servers, are triggered manually with the `deploy.yml` dispatch workflow. To reduce the deployment time significantly, it only runs the jobs a deployment requires (`install-dist` and `deployment`) and skips the tests. @@ -111,6 +111,8 @@ Custom projects deploy with `custom-staging.yml` and `custom-deploy.yml`, see [C Reference the core workflows by commit SHA rather than a branch like `main`, so changes to the core pipeline don't unexpectedly break custom projects. +The core workflows use `${{ github.workflow }}-${{ github.ref }}` as their concurrency group, and in a called workflow `github.workflow` is the name of the calling workflow. Don't use the same concurrency group in the calling workflow, because GitHub cancels the run as a deadlock. + ### Differences from the core workflows The custom project workflows differ from the core workflows in the following ways: @@ -151,6 +153,8 @@ jobs: secrets: inherit ``` +The calling workflow must be on the default branch before you can trigger it from the GitHub UI. + ## Making changes to the pipeline ### Testing changes @@ -186,9 +190,9 @@ Don't reference actions with a local path like `./.github/actions/your-action-na :::warning -After changing a composite action, update the commit SHA in the workflows that use it to the SHA of **the commit that changed the action** on your PR branch. Because PRs are squash merged, this SHA differs from the SHA of the commit that ends up in the history of the `main` branch, so it points to a dangling commit. This could become an issue if the repository is ever garbage collected. However, it is less cumbersome than waiting for the PR to be merged and then creating a follow-up PR to update the commit SHAs, or changing the merge rules to allow merging individual commits. +After changing a composite action, update the commit SHA in the workflows that use it to the SHA of **the commit that changed the action** on your PR branch. Because PRs are squash merged, this SHA differs from the SHA of the commit that ends up in the history of the `main` branch, so it points to a dangling commit. This could become an issue if the repository is ever garbage collected, also for the custom projects that call these workflows. We accept this risk, because it is less cumbersome than waiting for the PR to be merged and then creating a follow-up PR to update the commit SHAs, or changing the merge rules to allow merging individual commits. -If you reference a commit on `main` instead, Dependabot considers the reference to follow the `main` branch. Every time a new commit is merged, it opens a PR to update the SHA to the latest commit on `main`, even though the action didn't change (see [#3067](https://github.com/openremote/core/pull/3067), [#3081](https://github.com/openremote/core/pull/3081) and [#3087](https://github.com/openremote/core/pull/3087), fixed in [#3093](https://github.com/openremote/core/pull/3093)). +If you reference a commit on `main` instead, Dependabot considers the reference to follow the `main` branch. Every time a new commit is merged, it opens a PR to update the SHA to the latest commit on `main`, even though the action didn't change (see [#3067](https://github.com/openremote/core/pull/3067), [#3081](https://github.com/openremote/core/pull/3081) and [#3087](https://github.com/openremote/core/pull/3087)). [#3093](https://github.com/openremote/core/pull/3093) stopped these PRs by pointing the references to a commit that is not part of the `main` history. ::: From 0598ddac9c89cf68f5ed5e9612059f74819ee9a9 Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Fri, 23 Oct 2026 14:52:05 +0200 Subject: [PATCH 09/10] Address remaining review comments --- docs/developer-guide/025-code-formatting.md | 18 +++++++++-------- docs/developer-guide/170-ci-cd-pipeline.md | 22 +++++++++++++-------- 2 files changed, 24 insertions(+), 16 deletions(-) diff --git a/docs/developer-guide/025-code-formatting.md b/docs/developer-guide/025-code-formatting.md index 46926ab2..1ce1686f 100644 --- a/docs/developer-guide/025-code-formatting.md +++ b/docs/developer-guide/025-code-formatting.md @@ -615,25 +615,27 @@ Do not use the `-s ours` merge strategy either. It would record a formatting com ### Upgrading an existing custom project to Spotless -Custom projects commonly use the reusable CI/CD workflow from the `main` branch of `openremote/core`: +Custom projects reuse the CI/CD workflows of `openremote/core` (see [Custom projects](./170-ci-cd-pipeline.md#custom-projects)). The `custom-review.yml` and `custom-staging.yml` workflows include the Spotless formatting checks, so add Spotless to a project before you migrate it to these workflows. + +Projects that still use the previous `ci_cd.yml` workflow commonly reference it from the `main` branch: ```yaml uses: openremote/core/.github/workflows/ci_cd.yml@main ``` -Because this follows `main`, new workflow behaviour is inherited automatically. This includes the Spotless formatting checks. +Because this follows `main`, new workflow behaviour is inherited automatically. This includes the Spotless formatting checks, which `ci_cd.yml` runs from release 1.29.0 onwards. #### Temporarily pinning the workflow -When there is not yet time to upgrade a custom project, temporarily pin the reusable workflow to a commit from before the Spotless checks were added: +When there is not yet time to upgrade a custom project, temporarily pin `ci_cd.yml` to release 1.28.1, the last release before the Spotless checks were added: ```yaml -uses: openremote/core/.github/workflows/ci_cd.yml@51f4c3c0c8edd429a65237268d47d615617d4008 +uses: openremote/core/.github/workflows/ci_cd.yml@1.28.1 ``` -Pinning the workflow also prevents the project from receiving other workflow changes made after that commit. Use this only as a temporary measure. +Pinning the workflow also prevents the project from receiving other workflow changes made after that release. Use this only as a temporary measure. -The pinned SHA is the [`51f4c3c0c8edd429a65237268d47d615617d4008`](https://github.com/openremote/core/commit/51f4c3c0c8edd429a65237268d47d615617d4008) commit in `openremote/core`. +Release 1.28.1 is the [`51f4c3c0c8edd429a65237268d47d615617d4008`](https://github.com/openremote/core/commit/51f4c3c0c8edd429a65237268d47d615617d4008) commit, so projects pinned to this SHA use the same workflow. #### Pull request 1: Add Spotless and apply formatting @@ -678,13 +680,13 @@ When other custom-project template updates are also required, compare the existi ##### Commit 1: Add the configuration -Ensure the reusable workflow reference points to `main` before creating the first commit: +If you pinned `ci_cd.yml`, point it back to `main` before creating the first commit: ```yaml uses: openremote/core/.github/workflows/ci_cd.yml@main ``` -Commit the synchronized configuration and workflow change: +Commit the synchronized configuration, including any workflow change: ```shell git add --all diff --git a/docs/developer-guide/170-ci-cd-pipeline.md b/docs/developer-guide/170-ci-cd-pipeline.md index a20d7f84..1d218d30 100644 --- a/docs/developer-guide/170-ci-cd-pipeline.md +++ b/docs/developer-guide/170-ci-cd-pipeline.md @@ -103,13 +103,19 @@ Custom projects deploy with `custom-staging.yml` and `custom-deploy.yml`, see [C [Custom projects](./040-creating-a-custom-project.md) reuse the pipeline by calling the following workflows of the core repository from their own workflows: -| Workflow | Trigger in the custom project | Stages | -| -------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -| `custom-review.yml` | Pull Requests | Lint, build, and test. | -| `custom-staging.yml` | Pushes | The same as `custom-review.yml`, followed by a deployment when `.ci_cd/ci_cd.json` configures one for the pushed branch. | -| `custom-deploy.yml` | Manual (`workflow_dispatch`) | A deployment using `deployment.yml`. | +| Workflow | Trigger in the custom project | Stages | +| -------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| `custom-review.yml` | Pull Requests | Lint, build, and test. | +| `custom-staging.yml` | Pushes | The same as `custom-review.yml`, followed by a deployment when no job failed and `.ci_cd/ci_cd.json` configures one for the pushed branch. | +| `custom-deploy.yml` | Manual (`workflow_dispatch`) | A deployment using `deployment.yml`. | -Reference the core workflows by commit SHA rather than a branch like `main`, so changes to the core pipeline don't unexpectedly break custom projects. +Reference the core workflows by release version rather than a branch like `main`, so changes to the core pipeline don't unexpectedly break custom projects. When no release includes the workflow changes you need, reference a commit SHA instead, with a comment naming the closest release (`git describe --tags --abbrev=0 `). For example, releases up to 1.32.2 don't include these workflows, so you reference the merge commit of [#3441](https://github.com/openremote/core/pull/3441) or later: + +```yaml +uses: openremote/core/.github/workflows/custom-review.yml@b5e84f1b83b64c7808b59823dfd0a9cd3cee1c8f # 1.32.2 +``` + +The exception is `custom-project-test`, which references `main` on purpose to catch issues with changes to the core pipeline early. The core workflows use `${{ github.workflow }}-${{ github.ref }}` as their concurrency group, and in a called workflow `github.workflow` is the name of the calling workflow. Don't use the same concurrency group in the calling workflow, because GitHub cancels the run as a deadlock. @@ -117,7 +123,7 @@ The core workflows use `${{ github.workflow }}-${{ github.ref }}` as their concu The custom project workflows differ from the core workflows in the following ways: -- **Build:** They build with `custom-install-dist.yml`, which only runs `./gradlew installDist`. The archive steps in `install-dist.yml` assume the layout of the core repository (e.g. `ui/util`), and no custom project job downloads those artifacts. The build includes the deployment, so a broken deployment build already fails in review. +- **Build:** They build with `custom-install-dist.yml`, which only runs `./gradlew installDist`. The archive steps in `install-dist.yml` assume the layout of the core repository (e.g. `ui/util`), and no custom project job downloads those artifacts. The build includes the deployment, and changes to the `deployment` directory count as code changes, so a broken deployment build already fails in review. - **Code coverage:** Not run yet, because `code-coverage.yml` requires JaCoCo outputs and the `combinedCoverageReport` Gradle task. - **Manager version:** Deployments use the manager Docker image matching the `openremote` version in `gradle/libs.versions.toml`. Snapshot versions are not supported. - **Deploy script:** `deployment.yml` runs the `.ci_cd/deploy.sh` script of the core repository at the same `openremote` version. Up to 1.32.0, this script reads the `IS_MAIN_REPO` environment variable and the `openremote/.ci_cd` directory, so `deployment.yml` provides both. @@ -147,7 +153,7 @@ on: jobs: deploy: name: Deploy - uses: openremote/core/.github/workflows/custom-deploy.yml@ + uses: openremote/core/.github/workflows/custom-deploy.yml@ with: INPUTS: ${{ toJSON(inputs) }} secrets: inherit From cee0fd66231cdf51c96f2764f38a8a028dbf0b51 Mon Sep 17 00:00:00 2001 From: Ekhorn Date: Fri, 23 Oct 2026 14:54:26 +0200 Subject: [PATCH 10/10] Mention required default branch checks --- docs/developer-guide/170-ci-cd-pipeline.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/developer-guide/170-ci-cd-pipeline.md b/docs/developer-guide/170-ci-cd-pipeline.md index 1d218d30..3bd545c4 100644 --- a/docs/developer-guide/170-ci-cd-pipeline.md +++ b/docs/developer-guide/170-ci-cd-pipeline.md @@ -161,6 +161,17 @@ jobs: The calling workflow must be on the default branch before you can trigger it from the GitHub UI. +### Migrating from `ci_cd.yml` + +The new workflows report different status checks than `ci_cd.yml`. When you migrate a custom project, remove the `CI/CD / CI/CD` check from the required checks in its branch protection and add the checks of `custom-review.yml` instead. Their names start with the name of the job that calls `custom-review.yml` (see [How reusable workflows work](#how-reusable-workflows-work)): + +- ` / Detect whether code changed` +- ` / Lint / Backend Check Spotless` +- ` / Lint / UI Check Spotless` +- ` / Build / Install Dist` +- ` / Test / Unit Tests` +- ` / Test / Integration Tests` + ## Making changes to the pipeline ### Testing changes