Skip to content

Reducing gha cache size #1608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
sachinpro wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Reducing gha cache size #1608

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4df706f
chore: reducing GHA cache size by caching only release versions
sachinpro Jul 24, 2026
d25903b
chore: fixed issue with reading older driver during perf benchmark
sachinpro Jul 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 2 additions & 18 deletions .github/workflows/windows-benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,51 +33,35 @@
env:
DRIVER_ARCH: x64
BUILD_SHARD: ${{ inputs.build_shard }}
ODBC_GOOGLE_DRIVER_VERSION: 99.99.99
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

Check warning on line 38 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

ref-version-mismatch 

windows-benchmark.yml:38: action's hash pin has mismatched or missing version comment: points to commit 11d5960a3267
with:
ref: ${{ inputs.checkout-ref || github.ref }}

- uses: google-github-actions/auth@v2

Check failure on line 42 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

zizmor/unpinned-uses 

unpinned action reference: action is not pinned to a hash (required by blanket policy)

Check failure on line 42 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

unpinned-uses 

windows-benchmark.yml:42: unpinned action reference: action is not pinned to a hash (required by blanket policy)
with:
create_credentials_file: true
credentials_json: ${{ secrets.BUILD_CACHE_KEY }}

- uses: actions/setup-python@v5

Check failure on line 47 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

zizmor/unpinned-uses 

unpinned action reference: action is not pinned to a hash (required by blanket policy)

Check failure on line 47 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

unpinned-uses 

windows-benchmark.yml:47: unpinned action reference: action is not pinned to a hash (required by blanket policy)
id: py311
with:
python-version: '3.11'

- uses: google-github-actions/setup-gcloud@v2

Check failure on line 52 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

zizmor/unpinned-uses 

unpinned action reference: action is not pinned to a hash (required by blanket policy)

Check failure on line 52 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

unpinned-uses 

windows-benchmark.yml:52: unpinned action reference: action is not pinned to a hash (required by blanket policy)
env:
CLOUDSDK_PYTHON: ${{ steps.py311.outputs.python-path }}

- name: Install CMake
uses: lukka/get-cmake@latest

Check failure on line 57 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

zizmor/unpinned-uses 

unpinned action reference: action is not pinned to a hash (required by blanket policy)

Check failure on line 57 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

unpinned-uses 

windows-benchmark.yml:57: unpinned action reference: action is not pinned to a hash (required by blanket policy)
with:
cmakeVersion: "3.29.0"


- name: Download Installer from GCS
if: env.BUILD_SHARD == 'BqDriver'
shell: bash
run: |
mkdir -p installer_dl
# Sanitize the branch name replacing any character not in [a-zA-Z0-9-] with '_'
SANITIZED_BRANCH=$(echo "${{ github.ref_name }}" | sed -E 's/[^a-zA-Z0-9-]/_/g')
gcloud storage cp gs://odbc-integration-builds/${SANITIZED_BRANCH}/*.msi ./installer_dl/

- name: Install the ODBC Driver
shell: pwsh
run: |
if ($env:BUILD_SHARD -eq 'BqDriver') {
$installer = Get-ChildItem -Path ".\installer_dl\*" -Include "*.msi" | Select-Object -First 1
Write-Output "Executing odbc-driver-install.ps1 with LocalMsiPath: $($installer.FullName)"
.\ci\gha\builds\lib\odbc-driver-install.ps1 -LocalMsiPath $($installer.FullName)
} else {
Write-Output "Executing odbc-driver-install.ps1 for Simba driver (Core)"
.\ci\gha\builds\lib\odbc-driver-install.ps1
}
.\ci\gha\builds\lib\odbc-driver-install.ps1

- name: Configure CMake and Build ONLY performance_test
shell: bash
Expand Down Expand Up @@ -141,7 +125,7 @@
set -e

echo "Uploading results to GCS..."
gcloud storage cp "$RESULTS_FILE" gs://bq-dev-tools-testing-drivers/odbc-perf/${{ github.ref_name }}/results/

Check failure on line 128 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

zizmor/template-injection 

code injection via template expansion: may expand into attacker-controllable code

Check failure on line 128 in .github/workflows/windows-benchmark.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

template-injection 

windows-benchmark.yml:128: code injection via template expansion: may expand into attacker-controllable code

if [ $TEST_EXIT_CODE -ne 0 ]; then
echo "ERROR: Benchmark executable failed with exit code $TEST_EXIT_CODE."
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/windows-cmake.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,18 +30,18 @@
ODBC_GOOGLE_DRIVER_VERSION: 99.99.99
VCPKG_TRIPLET: ${{ matrix.arch }}-windows
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

Check warning on line 33 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

ref-version-mismatch 

windows-cmake.yml:33: action's hash pin has mismatched or missing version comment: points to commit 11d5960a3267
with:
ref: ${{ inputs.checkout-ref }}
- uses: google-github-actions/auth@v2

Check failure on line 36 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

unpinned-uses 

windows-cmake.yml:36: unpinned action reference: action is not pinned to a hash (required by blanket policy)
with:
create_credentials_file: true
credentials_json: ${{ secrets.BUILD_CACHE_KEY }}
- uses: actions/setup-python@v5

Check failure on line 40 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

unpinned-uses 

windows-cmake.yml:40: unpinned action reference: action is not pinned to a hash (required by blanket policy)
id: py311
with:
python-version: '3.11'
- uses: google-github-actions/setup-gcloud@v2

Check failure on line 44 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

unpinned-uses 

windows-cmake.yml:44: unpinned action reference: action is not pinned to a hash (required by blanket policy)
env:
CLOUDSDK_PYTHON: ${{ steps.py311.outputs.python-path }}
# We are deliberately downgrading cmake here because v3.31.0 runs each integration test twice
Expand Down Expand Up @@ -80,7 +80,7 @@
cd "${TEMP}"
mkdir .build
cd .build
git clone --branch ${{ steps.dynamic.outputs.vcpkg-version }} https://github.com/microsoft/vcpkg.git

Check failure on line 83 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

zizmor/template-injection 

code injection via template expansion: may expand into attacker-controllable code
./vcpkg/bootstrap-vcpkg.sh -disableMetrics
- name: Restore vcpkg binaries cache
uses: actions/cache/restore@v4
Expand All @@ -103,14 +103,14 @@
- name: Build cpp-bigquery-odbc
shell: bash
run: |
export VCPKG_ROOT="${TEMP}/.build/vcpkg"
export VCPKG_ROOT=$(cygpath -w "${TEMP}/.build/vcpkg")
export CLOUDSDK_PYTHON="${{ steps.py311.outputs.python-path }}"

Check failure on line 107 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

zizmor/template-injection 

code injection via template expansion: may expand into attacker-controllable code
export CMAKE_OUT="c:/b"



export MSVC_VERSION="${{ matrix.msvc }}"
export BRANCH_IDENTIFIER=$(echo "${{ github.ref_name }}" | tr '[:punct:]' '_')

Check failure on line 113 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

zizmor/template-injection 

code injection via template expansion: may expand into attacker-controllable code

Check failure on line 113 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

template-injection 

windows-cmake.yml:113: code injection via template expansion: may expand into attacker-controllable code

# We don't want to install 'arrow' in x86 because it is not supported
if [ "${{ matrix.arch }}" == "x86" ]; then
Expand Down Expand Up @@ -159,13 +159,13 @@
- name: Running testcases
shell: bash
run: |
export VCPKG_ROOT="${TEMP}/.build/vcpkg"
export VCPKG_ROOT=$(cygpath -w "${TEMP}/.build/vcpkg")
export CLOUDSDK_PYTHON="${{ steps.py311.outputs.python-path }}"

Check failure on line 163 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

zizmor/template-injection 

code injection via template expansion: may expand into attacker-controllable code
# Put the CMake output in a directory with more space and keep it short
# to avoid running into the MSVC limits.
export CMAKE_OUT='c:\b'
export MSVC_VERSION="${{ matrix.msvc }}"
export BRANCH_IDENTIFIER=$(echo "${{ github.ref_name }}" | tr '[:punct:]' '_')

Check failure on line 168 in .github/workflows/windows-cmake.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

template-injection 

windows-cmake.yml:168: code injection via template expansion: may expand into attacker-controllable code
export CPP_BIGQUERY_ODBC_TEST_TABLE_PREFIX="windows_${BRANCH_IDENTIFIER}_${MSVC_VERSION//[-:;.,?\/]/_}_${{ matrix.arch}}_$BUILD_SHARD"

export GOOGLE_APPLICATION_CREDENTIALS="$GOOGLE_GHA_CREDS_PATH"
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/windows-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@
- name: Build cpp-bigquery-odbc
shell: bash
run: |
export VCPKG_ROOT="${TEMP}/.build/vcpkg"
export VCPKG_ROOT=$(cygpath -w "${TEMP}/.build/vcpkg")
export CLOUDSDK_PYTHON="${{ steps.py311.outputs.python-path }}"
export CMAKE_OUT="c:/b"

Expand Down Expand Up @@ -198,7 +198,7 @@
./ci/installer/ODBCDriverForBigQuery/upgrade_code.ps1 -new_version "$wix_version"

- name: Build Windows Installer (WiX)
shell: cmd

Check warning on line 201 in .github/workflows/windows-release.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

misfeature 

windows-release.yml:201: usage of GitHub Actions misfeatures: shell defined here
run: |
echo Locating MSBuild...
for /f "usebackq delims=" %%i in (`"C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -requires Microsoft.Component.MSBuild -find MSBuild\**\Bin\MSBuild.exe`) do (
Expand Down Expand Up @@ -250,7 +250,7 @@
- name: Running testcases
shell: bash
run: |
export VCPKG_ROOT="${TEMP}/.build/vcpkg"
export VCPKG_ROOT=$(cygpath -w "${TEMP}/.build/vcpkg")
export CLOUDSDK_PYTHON="${{ steps.py311.outputs.python-path }}"
export MSVC_VERSION="${{ matrix.msvc }}"
export BRANCH_IDENTIFIER=$(echo "${{ github.ref_name }}" | tr '[:punct:]' '_')
Expand Down Expand Up @@ -292,7 +292,7 @@
subject-path: ci/installer/ODBCDriverForBigQuery/bin/Release/en-us/ODBCDriverforBigQuery_windows_${{ matrix.arch }}_${{ steps.version.outputs.version }}.msi

- name: Release Driver
uses: softprops/action-gh-release@v2

Check notice on line 295 in .github/workflows/windows-release.yml

View workflow job for this annotation

GitHub Actions / zizmor-output

superfluous-actions 

windows-release.yml:295: action functionality is already included by the runner: use `gh release` in a script step
with:
files: ci/installer/ODBCDriverForBigQuery/bin/Release/en-us/*_${{ env.DRIVER_ARCH }}_${{ steps.version.outputs.version }}.msi
overwrite_files: true
Expand Down
3 changes: 3 additions & 0 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ cmake_minimum_required(VERSION 3.14)
if (NOT DEFINED PROJECT_VERSION)
set(PROJECT_VERSION "1.0.0")
endif ()
set(VCPKG_OVERLAY_TRIPLETS
"${CMAKE_CURRENT_LIST_DIR}/ci/etc/triplets"
CACHE PATH "Path to custom triplets")

project(
cpp-bigquery-odbc
Expand Down
5 changes: 5 additions & 0 deletions ci/etc/triplets/arm64-linux.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
set(VCPKG_TARGET_ARCHITECTURE arm64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic)
set(VCPKG_CMAKE_SYSTEM_NAME Linux)
set(VCPKG_BUILD_TYPE release)
4 changes: 4 additions & 0 deletions ci/etc/triplets/arm64-osx.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
set(VCPKG_TARGET_ARCHITECTURE arm64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic)
set(VCPKG_BUILD_TYPE release)
5 changes: 5 additions & 0 deletions ci/etc/triplets/x64-linux.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic)
set(VCPKG_CMAKE_SYSTEM_NAME Linux)
set(VCPKG_BUILD_TYPE release)
4 changes: 4 additions & 0 deletions ci/etc/triplets/x64-osx.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE dynamic)
set(VCPKG_BUILD_TYPE release)
4 changes: 4 additions & 0 deletions ci/etc/triplets/x64-windows-static.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE static)
set(VCPKG_LIBRARY_LINKAGE static)
set(VCPKG_BUILD_TYPE release)
4 changes: 4 additions & 0 deletions ci/etc/triplets/x86-windows-static.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
set(VCPKG_TARGET_ARCHITECTURE x86)
set(VCPKG_CRT_LINKAGE static)
set(VCPKG_LIBRARY_LINKAGE static)
set(VCPKG_BUILD_TYPE release)
Loading