-
Notifications
You must be signed in to change notification settings - Fork 1.1k
[MNT] homogenize CI workflows with GC.OS repositories #702
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
fkiraly
merged 27 commits into
PyPortfolio:main
from
tingiskhan:ci/homogenize-workflows-with-sktime
Mar 8, 2026
Merged
Changes from 11 commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
e0903e3
Initial rework
tingiskhan c5ba221
Adds uv.lock
tingiskhan 04b200b
Use uv
tingiskhan af307d1
Adds comment
tingiskhan aa94384
Use uv here as well
tingiskhan f42002c
Uses dependency groups instead of optionals, and adds requires-python
tingiskhan 5bcb552
Comment fix
tingiskhan 19115cb
Minor fixes
tingiskhan 452beac
Unnecessary line
tingiskhan 95a18d9
Use same as old
tingiskhan bcf8326
Minor cleanup
tingiskhan 3b920b4
Replaces uv sync with `uv venv` and `uv pip install`. Not entirely su…
tingiskhan 1d70991
Merge branch 'main' into ci/homogenize-workflows-with-sktime
tingiskhan 58fffbc
revert rename to track change
fkiraly 54eaf75
As per @tschm's assessment
tingiskhan 1473e63
Removes action.yml and removes unnecessary step
tingiskhan b14483a
Merge branch 'ci/homogenize-workflows-with-sktime' of https://github.…
tingiskhan 5c13445
Adds fix for latest prices containing NaNs by just dropping nans - co…
tingiskhan c9a1b4a
Uses pip
tingiskhan d3c7dbf
Applies "prettier"
tingiskhan 12ac9b2
Rollback
tingiskhan cfcb62d
revert cookbook
fkiraly 748e638
revert to dep groups
fkiraly 60ff9b2
move notebooks last
fkiraly 2ded80e
Update main.yml
fkiraly be33cfd
Update main.yml
fkiraly b4e5730
Update main.yml
fkiraly File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,31 +1,33 @@ | ||
| # This workflow will install Python dependencies, run tests and lint with a variety of Python versions | ||
| # For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python | ||
|
|
||
| name: pytest | ||
| name: Install and test | ||
|
|
||
| on: | ||
| push: | ||
| branches: ["main"] | ||
| branches: | ||
| - main | ||
| pull_request: | ||
| branches: ["main"] | ||
| branches: | ||
| - main | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| code-quality: | ||
| name: code-quality | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - name: repository checkout step | ||
| uses: actions/checkout@v6 | ||
|
|
||
| - uses: actions/setup-python@v6 | ||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v7 | ||
| with: | ||
| python-version: '3.14' | ||
| enable-cache: true | ||
| python-version: "3.14" | ||
|
|
||
| - name: install pre-commit | ||
| run: python3 -m pip install pre-commit | ||
| run: uv pip install pre-commit | ||
|
|
||
| - name: Checkout code | ||
| uses: actions/checkout@v6 | ||
|
|
@@ -40,7 +42,7 @@ jobs: | |
|
|
||
| - name: Print changed files | ||
| run: | | ||
| echo "Changed files: $CHANGED_FILES" | ||
| echo "Changed files:" && echo "$CHANGED_FILES" | tr ' ' '\n' | ||
|
|
||
| - name: Run pre-commit on changed files | ||
| run: | | ||
|
|
@@ -50,17 +52,42 @@ jobs: | |
| echo "No changed files to check." | ||
| fi | ||
|
|
||
| pytest-nosoftdeps: | ||
| detect-notebooks-change: | ||
| needs: code-quality | ||
| name: nosoftdeps (${{ matrix.python-version }}, ${{ matrix.os }}) | ||
| runs-on: ${{ matrix.os }} | ||
| env: | ||
| MPLBACKEND: Agg # https://github.com/orgs/community/discussions/26434 | ||
| name: detect change affecting notebooks | ||
| runs-on: ubuntu-latest | ||
| permissions: | ||
| pull-requests: read | ||
| outputs: | ||
| notebooks: ${{ steps.check.outputs.notebooks }} | ||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Fetch main branch | ||
| run: git fetch origin main | ||
|
|
||
| - name: Check if cookbook, pypfopt or pyproject.toml changed | ||
| id: check | ||
| run: | | ||
| if git diff --quiet origin/main -- cookbook/ pypfopt/ pyproject.toml; then | ||
| echo "No notebook related changes" | ||
| echo "notebooks=false" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "Detected changes in notebooks or pypfopt" | ||
| echo "notebooks=true" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| run-notebook-examples: | ||
| needs: detect-notebooks-change | ||
| if: ${{ needs.detect-notebooks-change.outputs.notebooks == 'true' }} | ||
| runs-on: ubuntu-latest | ||
|
|
||
| strategy: | ||
| matrix: | ||
| os: [ubuntu-latest, macos-latest, windows-latest] | ||
| python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"] | ||
| fail-fast: false # to not fail all combinations if just one fails | ||
| python-version: ['3.10', '3.11', '3.12', '3.13', '3.14'] | ||
| fail-fast: false | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
@@ -69,39 +96,39 @@ jobs: | |
| uses: astral-sh/setup-uv@v7 | ||
| with: | ||
| enable-cache: true | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@v6 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Display Python version | ||
| run: python -c "import sys; print(sys.version)" | ||
|
|
||
| - name: Install dependencies | ||
| shell: bash | ||
| run: uv pip install ".[dev]" --no-cache-dir | ||
| env: | ||
| UV_SYSTEM_PYTHON: 1 | ||
| run: uv sync --extra all_extras --group dev --extra notebook_test --no-cache | ||
|
|
||
| - name: Show dependencies | ||
| run: uv pip list | ||
|
|
||
| - name: Test with pytest | ||
| - name: Collect notebooks | ||
| id: notebooks | ||
| shell: bash | ||
| run: | | ||
| NOTEBOOKS=$(find cookbook -name '*.ipynb' -print0 | xargs -0 echo) | ||
| echo "notebooks=$NOTEBOOKS" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Run notebooks | ||
| shell: bash | ||
| run: | | ||
| pytest ./tests | ||
| uv run pytest --reruns 3 --nbmake --nbmake-timeout=3600 -vv ${{ steps.notebooks.outputs.notebooks }} | ||
|
|
||
| pytest: | ||
| needs: pytest-nosoftdeps | ||
| name: (${{ matrix.python-version }}, ${{ matrix.os }}) | ||
| test-nosoftdeps: | ||
| needs: code-quality | ||
| name: test-nosoftdeps (${{ matrix.python-version }}, ${{ matrix.os }}) | ||
| runs-on: ${{ matrix.os }} | ||
| env: | ||
| MPLBACKEND: Agg # https://github.com/orgs/community/discussions/26434 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| os: [ubuntu-latest, macos-latest, windows-latest] | ||
| python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"] | ||
| fail-fast: false # to not fail all combinations if just one fails | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
@@ -110,34 +137,67 @@ jobs: | |
| uses: astral-sh/setup-uv@v7 | ||
| with: | ||
| enable-cache: true | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Display Python version | ||
| run: python -c "import sys; print(sys.version)" | ||
|
|
||
| - name: Force non-GUI Matplotlib backend (Windows) | ||
| if: ${{ matrix.os == 'windows-latest' }} | ||
| shell: pwsh | ||
| run: echo "MPLBACKEND=Agg" >> $env:GITHUB_ENV | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@v6 | ||
| - name: Install dependencies | ||
| shell: bash | ||
| run: uv sync --group dev --no-cache | ||
|
|
||
| - name: Show dependencies | ||
| run: uv pip list | ||
|
|
||
| - name: Run tests | ||
| run: uv run pytest ./tests | ||
|
|
||
| test-full: | ||
| needs: test-nosoftdeps | ||
| name: test-full (${{ matrix.python-version }}, ${{ matrix.os }}) | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"] | ||
| os: [ubuntu-latest, macos-latest, windows-latest] | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v7 | ||
| with: | ||
| enable-cache: true | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Display Python version | ||
| run: python -c "import sys; print(sys.version)" | ||
|
|
||
| - name: Force non-GUI Matplotlib backend (Windows) | ||
| if: ${{ matrix.os == 'windows-latest' }} | ||
| shell: pwsh | ||
| run: echo "MPLBACKEND=Agg" >> $env:GITHUB_ENV | ||
|
|
||
| - name: Install dependencies | ||
| shell: bash | ||
| run: uv pip install ".[dev,all_extras]" --no-cache-dir | ||
| env: | ||
| UV_SYSTEM_PYTHON: 1 | ||
| run: uv sync --extra all_extras --extra dev --no-cache | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would go for |
||
|
|
||
| - name: Show dependencies | ||
| run: uv pip list | ||
|
|
||
| - name: Test with pytest | ||
| run: | | ||
| pytest ./tests | ||
| - name: Run tests | ||
| run: uv run pytest ./tests | ||
|
|
||
| codecov: | ||
| name: coverage (${{ matrix.python-version }} on ${{ matrix.os }} | ||
| runs-on: ${{ matrix.os }} | ||
| needs: code-quality | ||
| env: | ||
| MPLBACKEND: Agg # https://github.com/orgs/community/discussions/26434 | ||
|
tingiskhan marked this conversation as resolved.
|
||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| matrix: | ||
| os: [ubuntu-latest, macos-latest, windows-latest] | ||
|
|
@@ -150,27 +210,25 @@ jobs: | |
| uses: astral-sh/setup-uv@v7 | ||
| with: | ||
| enable-cache: true | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@v6 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Display Python version | ||
| run: python -c "import sys; print(sys.version)" | ||
|
|
||
| - name: Force non-GUI Matplotlib backend (Windows) | ||
| if: ${{ matrix.os == 'windows-latest' }} | ||
| shell: pwsh | ||
| run: echo "MPLBACKEND=Agg" >> $env:GITHUB_ENV | ||
|
|
||
| - name: Install dependencies | ||
| shell: bash | ||
| run: uv pip install ".[dev,all_extras]" --no-cache-dir | ||
| env: | ||
| UV_SYSTEM_PYTHON: 1 | ||
| run: uv sync --only-group dev --only-group cov | ||
|
|
||
| - name: Show dependencies | ||
| run: uv pip list | ||
|
|
||
| - name: Generate coverage report | ||
| run: | | ||
| pip install pytest pytest-cov | ||
| pytest --cov=./ --cov-report=xml | ||
|
|
||
| - name: Upload coverage to Codecov | ||
|
|
@@ -180,51 +238,3 @@ jobs: | |
| with: | ||
| files: ./coverage.xml | ||
| fail_ci_if_error: true | ||
|
|
||
| notebooks: | ||
| needs: code-quality | ||
| runs-on: ubuntu-latest | ||
|
|
||
| strategy: | ||
| matrix: | ||
| python-version: [ '3.10', '3.11', '3.12', '3.13', '3.14' ] | ||
| fail-fast: false | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v7 | ||
| with: | ||
| enable-cache: true | ||
|
|
||
| - name: Set up Python ${{ matrix.python-version }} | ||
| uses: actions/setup-python@v6 | ||
| with: | ||
| python-version: ${{ matrix.python-version }} | ||
|
|
||
| - name: Display Python version | ||
| run: python -c "import sys; print(sys.version)" | ||
|
|
||
| - name: Install dependencies | ||
| shell: bash | ||
| run: uv pip install ".[dev,all_extras,notebook_test]" --no-cache-dir | ||
| env: | ||
| UV_SYSTEM_PYTHON: 1 | ||
|
|
||
| - name: Show dependencies | ||
| run: uv pip list | ||
|
|
||
| # Discover all notebooks | ||
| - name: Collect notebooks | ||
| id: notebooks | ||
| shell: bash | ||
| run: | | ||
| NOTEBOOKS=$(find cookbook -name '*.ipynb' -print0 | xargs -0 echo) | ||
| echo "notebooks=$NOTEBOOKS" >> $GITHUB_OUTPUT | ||
|
|
||
| # Run all discovered notebooks with nbmake | ||
| - name: Test notebooks | ||
| shell: bash | ||
| run: | | ||
| uv run pytest --reruns 3 --nbmake --nbmake-timeout=3600 -vv ${{ steps.notebooks.outputs.notebooks }} | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.