Skip to content

Keep CI's apt installs from waiting out a slow Ubuntu mirror #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AndreasBur merged 1 commit into from
Oct 27, 2026
Merged

Keep CI's apt installs from waiting out a slow Ubuntu mirror #83

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions .github/actions/apt-install/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# apt-get install for a CI job, made to survive the Ubuntu mirror having a bad hour.
#
# What it guards against was read out of a job that sat in its install step until the job's
# timeout cancelled it: apt-get update fetched its indices at 13 MB/s, and then the package
# downloads crawled - cmake's 14 MB took five minutes, and gcc-avr's 57 MB was still coming
# thirteen minutes later. A connection that trickles is not a connection that has died, so
# apt never gives up on it, and the step ran into the job's timeout-minutes, which reports as
# a cancellation and says nothing about why.
#
# So, in that order:
#
# - The .deb files are cached, keyed by exactly the versions this run would fetch. A run whose
# packages have not changed downloads nothing from the mirror at all; a new version in the
# archive is a new key, so the cache follows the archive by itself instead of going stale.
# - What is still missing is downloaded with --download-only under a timeout, and retried on a
# fresh connection when the timeout fires. Generous next to a normal run, which fetches all
# of it in seconds.
# - Only then is it installed, from the cache and with no timeout: the downloading is over by
# then, and a timeout that killed dpkg halfway through unpacking would leave the runner
# broken rather than slow.
name: Install Ubuntu packages
description: apt-get install with the downloads cached, and a stalled mirror retried rather than waited out

inputs:
packages:
description: The packages to install, separated by spaces
required: true

runs:
using: composite
steps:
# The index is normally fast even when the archive is not, and the key below needs it. It
# gets the same timeout and retry all the same: ci.yml records an apt-get update that hung.
# --print-uris lists every .deb the install would fetch. Only the file name goes into the
# key: it carries the version, so the key changes exactly when a download would, while the
# URL in front of it names whichever mirror this runner was given.
- name: Work out what would be downloaded
id: plan
shell: bash
run: |
for Attempt in 1 2 3; do
if timeout 120 sudo apt-get update -q; then break; fi
if [ "$Attempt" -eq 3 ]; then
echo "::error::apt-get update did not finish in three attempts"
exit 1
fi
echo "::warning::apt-get update attempt $Attempt stalled or failed; retrying"
done
KEY=$(apt-get install --print-uris -qq --no-install-recommends ${{ inputs.packages }} | cut -d' ' -f2 | sha256sum | cut -c1-16)
echo "key=$KEY" >> "$GITHUB_OUTPUT"

- name: Restore the downloaded packages
uses: actions/cache@v6
with:
path: ~/apt-archives
key: apt-${{ runner.os }}-${{ steps.plan.outputs.key }}
# An older set still holds most of the files; apt fetches only what differs.
restore-keys: apt-${{ runner.os }}-

# apt downloads into ~/apt-archives as root and warns that it cannot hand the job to its
# _apt sandbox user there. That is a warning, the download is the same; the chown
# afterwards is what lets the cache step read the files back to save them.
- name: Download, retrying a stalled mirror
shell: bash
run: |
mkdir -p ~/apt-archives/partial
for Attempt in 1 2 3 4; do
if timeout 240 sudo apt-get -q -o Dir::Cache::Archives="$HOME/apt-archives" -o APT::Keep-Downloaded-Packages=true \
install -y --no-install-recommends --download-only ${{ inputs.packages }}; then
break
fi
if [ "$Attempt" -eq 4 ]; then
echo "::error::apt could not download the packages in four attempts"
exit 1
fi
echo "::warning::apt download attempt $Attempt stalled or failed; retrying on a new connection"
done
sudo chown -R "$(id -u):$(id -g)" ~/apt-archives

- name: Install
shell: bash
run: |
sudo apt-get -q -o Dir::Cache::Archives="$HOME/apt-archives" -o APT::Keep-Downloaded-Packages=true \
install -y --no-install-recommends ${{ inputs.packages }}
sudo chown -R "$(id -u):$(id -g)" ~/apt-archives
17 changes: 9 additions & 8 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@
# README-only commit once hung twice - apt-get update on the simulator job, and the cppcheck
# job before its first step ran at all - and without a timeout the two sat there until
# GitHub's own six-hour limit cancelled them, which reports as a red badge six hours after
# anyone would have looked. The numbers differ per job because the cold-cache cost does: the
# anyone would have looked. The apt installs themselves go through
# .github/actions/apt-install, which caches the packages and retries a mirror that has
# slowed to a trickle, rather than leaving that to these timeouts. The numbers differ per job because the cold-cache cost does: the
# RP2350 firmware clones arduino-pico with the Pico SDK, and that alone is several minutes.
name: CI

Expand Down Expand Up @@ -65,9 +67,9 @@ jobs:
# Pinned to the Ubuntu that carries wxWidgets 3.2, which is what
# find_package(wxWidgets) expects; libwxgtk3.2-dev pulls in the wx-config it needs.
- name: Install the toolchain
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends cmake ninja-build libwxgtk3.2-dev
uses: ./.github/actions/apt-install
with:
packages: cmake ninja-build libwxgtk3.2-dev

# -Werror only here, not in the CMakeLists: a warning should stop a change from
# being merged without stopping somebody in the middle of debugging one.
Expand Down Expand Up @@ -226,10 +228,9 @@ jobs:
- uses: actions/checkout@v7

- name: Install the toolchain
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
cmake ninja-build gcc-avr avr-libc python3
uses: ./.github/actions/apt-install
with:
packages: cmake ninja-build gcc-avr avr-libc python3

# Asked before the build rather than met inside it. Whether an avr-gcc knows this part
# is a property of its device-specs, and a build that walks into a missing one stops
Expand Down
31 changes: 16 additions & 15 deletions .github/workflows/static-analysis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ jobs:
# simulator's compile database is what covers the firmware core, and it does not
# exist without the headers the backend includes.
- name: Install the toolchain
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends cmake ninja-build libwxgtk3.2-dev clang-tidy
uses: ./.github/actions/apt-install
with:
packages: cmake ninja-build libwxgtk3.2-dev clang-tidy

# Configured, not built: clang-tidy compiles each translation unit itself and needs
# nothing from the build but compile_commands.json.
Expand Down Expand Up @@ -80,10 +80,9 @@ jobs:
# The cross compiler is installed although nothing is compiled with it: the tool asks it
# where avr-libc's headers are, and clang reads those instead of the host's.
- name: Install the toolchain
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
cmake ninja-build gcc-avr avr-libc clang-tidy python3
uses: ./.github/actions/apt-install
with:
packages: cmake ninja-build gcc-avr avr-libc clang-tidy python3

# Configured, not built, as above. The toolchain file for the same reason ci.yml names
# it: the database records whichever compiler project() found.
Expand Down Expand Up @@ -127,11 +126,13 @@ jobs:
key: ${{ matrix.cache }}-${{ runner.os }}-${{ hashFiles(format('platform/{0}/platformio.ini', matrix.backend)) }}
restore-keys: ${{ matrix.cache }}-${{ runner.os }}-

- name: Install PlatformIO and clang-tidy
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends clang-tidy
pip install --upgrade platformio
- name: Install clang-tidy
uses: ./.github/actions/apt-install
with:
packages: clang-tidy

- name: Install PlatformIO
run: pip install --upgrade platformio

# compiledb rather than a build: it fetches the platform, runs the pre-scripts - which
# is what writes WebPage.h - and writes the database, and compiles nothing.
Expand All @@ -153,9 +154,9 @@ jobs:
# enough - it reports the same findings with and without compile_commands.json, and
# this way the job still answers when the simulator does not build.
- name: Install cppcheck
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends cppcheck
uses: ./.github/actions/apt-install
with:
packages: cppcheck

# firmware/ only, unlike the clang-tidy job above. Without wxWidgets' own headers
# cppcheck stops at DECLARE_EVENT_TABLE in the simulator's window class and asks to be
Expand Down
Loading