Skip to content

Commit 599ffb3

Browse files
ybarhate0812ybarhate0812
authored
Merge pull request #106 from SECQUOIA/master
Implement CI/CD pipeline and enhance testing framework for PharmaPy
2 parents 6c422ab + 55eb674 commit 599ffb3

85 files changed

Lines changed: 13150 additions & 5189 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/README.md‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# PharmaPy CI/CD Configuration
2+
3+
This repository uses a consolidated GitHub Actions workflow to ensure efficient testing and reduce CI resource consumption.
4+
5+
## Workflow Structure
6+
7+
### `consolidated-ci.yml`
8+
A single, comprehensive workflow that replaces multiple separate workflows, reducing the total number of CI jobs from 99+ to approximately 12-15.
9+
10+
#### Jobs:
11+
12+
1. **Core Tests** (6 jobs)
13+
- Ubuntu + Windows × Python 3.9, 3.11
14+
- macOS × Python 3.11
15+
- Ubuntu × Python 3.12
16+
- Runs unit and integration tests
17+
- Tests installation methods
18+
19+
2. **Assimulo Integration** (1 job)
20+
- Ubuntu 20.04 × Python 3.9
21+
- Tests scientific computing integration
22+
- Specialized for numerical solver compatibility
23+
24+
3. **Installation Scripts** (2 jobs, conditional)
25+
- Ubuntu + Windows
26+
- Only runs on main branches
27+
- Tests installation scripts
28+
29+
4. **Code Quality & Documentation** (1 job, conditional)
30+
- Only runs on main/master branches
31+
- Linting, coverage, documentation build
32+
33+
5. **Build & Package** (1 job, conditional)
34+
- Only runs on main/master branches or tags
35+
- Creates distributable packages
36+
37+
## Triggers
38+
39+
- **Push**: main, master, develop branches
40+
- **Pull Request**: main, master, develop branches
41+
- **Schedule**: Weekly on Sundays at 3 AM UTC
42+
- **Manual**: Via workflow_dispatch
43+
44+
## Resource Optimization
45+
46+
- **Reduced OS Matrix**: Focus on Ubuntu/Windows, selective macOS
47+
- **Strategic Python Versions**: LTS (3.9) + Current (3.11) + Latest (3.12)
48+
- **Conditional Jobs**: Quality checks and builds only on main branches
49+
- **Smart Matrix Includes**: Additional combinations only where needed
50+
51+
This configuration provides comprehensive testing coverage while using ~85% fewer CI resources than the previous setup.

‎.github/workflows/ci.yml‎

Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,315 @@
1+
name: PharmaPy CI/CD Pipeline
2+
3+
on:
4+
push:
5+
branches: [ main, master, develop ]
6+
pull_request:
7+
branches: [ main, master, develop ]
8+
schedule:
9+
# Run tests weekly on Sunday at 3 AM UTC
10+
- cron: '0 3 * * 0'
11+
workflow_dispatch:
12+
13+
env:
14+
FORCE_COLOR: 1
15+
16+
jobs:
17+
# Core testing matrix - reduced but comprehensive
18+
test-core:
19+
name: Core Tests (${{ matrix.os }}, Python ${{ matrix.python-version }})
20+
runs-on: ${{ matrix.os }}
21+
timeout-minutes: 30 # Prevent long-running jobs
22+
strategy:
23+
fail-fast: false
24+
matrix:
25+
# Reduced OS matrix to essential platforms
26+
os: [ubuntu-latest, windows-latest]
27+
# Reduced Python versions to LTS and latest
28+
python-version: ['3.9', '3.11']
29+
include:
30+
# Add macOS for latest Python only to reduce jobs
31+
- os: macos-latest
32+
python-version: '3.11'
33+
# Add Python 3.12 for Ubuntu only
34+
- os: ubuntu-latest
35+
python-version: '3.12'
36+
37+
steps:
38+
- name: Checkout code
39+
uses: actions/checkout@v4
40+
41+
- name: Set up Miniconda
42+
uses: conda-incubator/setup-miniconda@v3
43+
with:
44+
miniforge-version: latest
45+
activate-environment: pharmapy-ci
46+
environment-file: environment.yml
47+
python-version: ${{ matrix.python-version }}
48+
auto-activate-base: false
49+
auto-update-conda: true
50+
mamba-version: "*"
51+
use-mamba: true
52+
53+
- name: Install system dependencies (Ubuntu)
54+
if: matrix.os == 'ubuntu-latest'
55+
run: |
56+
sudo apt-get update
57+
sudo apt-get install -y build-essential gfortran liblapack-dev libblas-dev
58+
59+
- name: Install system dependencies (macOS)
60+
if: matrix.os == 'macos-latest'
61+
run: |
62+
brew install gcc
63+
64+
- name: Verify conda environment
65+
shell: bash -l {0}
66+
run: |
67+
conda info
68+
conda list
69+
python --version
70+
echo "Environment setup successful"
71+
72+
- name: Install assimulo (optional)
73+
shell: bash -l {0}
74+
continue-on-error: true
75+
run: |
76+
echo "Attempting to install assimulo..."
77+
conda install -c conda-forge assimulo --yes || pip install assimulo || echo "Assimulo installation failed, continuing without it"
78+
79+
- name: Validate environment setup
80+
shell: bash -l {0}
81+
run: |
82+
python -c "import numpy, scipy, matplotlib, pandas; print('Core scientific packages available')"
83+
python -c "import pytest; print('Testing framework available')"
84+
python -c "import importlib.util; print('Assimulo available' if importlib.util.find_spec('assimulo') else 'Assimulo not available')"
85+
86+
- name: Install PharmaPy
87+
shell: bash -l {0}
88+
run: |
89+
pip install -e .
90+
91+
- name: Run unit tests
92+
shell: bash -l {0}
93+
run: |
94+
python -m pytest tests/unit/ -v --tb=short
95+
96+
- name: Run integration tests
97+
shell: bash -l {0}
98+
run: |
99+
python -m pytest tests/integration/ -v --tb=short --run-network-tests
100+
101+
- name: Test installation methods
102+
shell: bash -l {0}
103+
run: |
104+
# Test wheel build and install
105+
python setup.py bdist_wheel
106+
pip install dist/*.whl --force-reinstall
107+
python -c "import PharmaPy; print('Wheel installation successful')"
108+
109+
# Assimulo testing - lightweight focused test
110+
test-assimulo:
111+
name: Assimulo Integration Tests
112+
runs-on: ubuntu-latest # Use latest Ubuntu for speed
113+
timeout-minutes: 15 # Prevent long-running jobs
114+
strategy:
115+
fail-fast: false
116+
matrix:
117+
python-version: ['3.9'] # Focus on 3.9 for assimulo compatibility
118+
119+
steps:
120+
- name: Checkout code
121+
uses: actions/checkout@v4
122+
123+
- name: Set up Miniconda
124+
uses: conda-incubator/setup-miniconda@v3
125+
with:
126+
miniforge-version: latest
127+
activate-environment: pharmapy-assimulo
128+
environment-file: environment.yml
129+
python-version: ${{ matrix.python-version }}
130+
auto-activate-base: false
131+
auto-update-conda: true
132+
mamba-version: "*"
133+
use-mamba: true
134+
135+
- name: Install system dependencies
136+
run: |
137+
sudo apt-get update
138+
sudo apt-get install -y build-essential gfortran liblapack-dev libblas-dev
139+
140+
- name: Install assimulo
141+
shell: bash -l {0}
142+
run: |
143+
conda install -c conda-forge assimulo --yes
144+
145+
- name: Install PharmaPy
146+
shell: bash -l {0}
147+
run: |
148+
pip install -e .
149+
150+
- name: Test assimulo integration (fast tests only)
151+
shell: bash -l {0}
152+
run: |
153+
python -c "import assimulo; print(f'Assimulo version: {assimulo.__version__}')"
154+
python -m pytest tests/integration/test_assimulo.py::TestAssimuloIntegration::test_assimulo_import -v
155+
python -m pytest tests/integration/test_assimulo.py::TestAssimuloIntegration::test_assimulo_solvers_available -v
156+
python -m pytest tests/integration/test_assimulo.py::TestAssimuloIntegration::test_pharmapy_with_assimulo -v
157+
python -m pytest tests/integration/test_assimulo.py::TestAssimuloIntegration::test_assimulo_version_compatibility -v
158+
python -m pytest tests/integration/test_assimulo.py::TestAssimuloIntegration::test_sundials_integration -v
159+
160+
# Installation script testing - runs on all pushes
161+
test-installation-scripts:
162+
name: Installation Scripts
163+
timeout-minutes: 20 # Prevent long-running jobs
164+
if: github.event_name == 'push'
165+
runs-on: ${{ matrix.os }}
166+
strategy:
167+
fail-fast: false
168+
matrix:
169+
os: [ubuntu-latest, windows-latest]
170+
171+
steps:
172+
- name: Checkout code
173+
uses: actions/checkout@v4
174+
175+
- name: Set up Conda
176+
uses: conda-incubator/setup-miniconda@v3
177+
with:
178+
auto-update-conda: true
179+
python-version: '3.9'
180+
channels: conda-forge,defaults
181+
182+
- name: Test Windows installation script
183+
if: matrix.os == 'windows-latest'
184+
shell: cmd
185+
run: |
186+
echo pharmapy-ci-test | InstallOnWindows.bat
187+
call conda activate pharmapy-ci-test
188+
python -c "import PharmaPy; print('Windows script installation successful')"
189+
190+
- name: Test macOS/Linux installation script
191+
if: matrix.os != 'windows-latest'
192+
shell: bash -l {0}
193+
run: |
194+
echo "pharmapy-ci-test" | bash InstallOnMac.sh
195+
conda activate pharmapy-ci-test
196+
python -c "import PharmaPy; print('Unix script installation successful')"
197+
198+
# Code quality and documentation - run on every push
199+
quality-and-docs:
200+
name: Code Quality & Documentation
201+
runs-on: ubuntu-latest
202+
203+
steps:
204+
- name: Checkout code
205+
uses: actions/checkout@v4
206+
207+
- name: Set up Conda
208+
uses: conda-incubator/setup-miniconda@v3
209+
with:
210+
miniforge-version: latest
211+
activate-environment: pharmapy-ci
212+
environment-file: environment.yml
213+
python-version: '3.11'
214+
auto-activate-base: false
215+
216+
- name: Install PharmaPy and dev dependencies
217+
shell: bash -l {0}
218+
run: |
219+
pip install -e .
220+
pip install -r requirements-dev.txt
221+
222+
- name: Install pandoc (for documentation)
223+
shell: bash -l {0}
224+
run: |
225+
conda install -c conda-forge pandoc --yes
226+
227+
- name: Install assimulo (for code quality checks)
228+
shell: bash -l {0}
229+
continue-on-error: true
230+
run: |
231+
conda install -c conda-forge assimulo --yes || pip install assimulo || echo "Assimulo installation failed, but continuing"
232+
233+
- name: Run linting
234+
shell: bash -l {0}
235+
continue-on-error: true
236+
run: |
237+
flake8 PharmaPy/ --count --select=E9,F63,F7,F82 --show-source --statistics
238+
flake8 PharmaPy/ --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
239+
240+
- name: Check code formatting with black
241+
shell: bash -l {0}
242+
continue-on-error: true
243+
run: |
244+
echo "Checking code formatting with black..."
245+
black --check --diff PharmaPy/ tests/ *.py --exclude="/(\.git|\.venv|\.tox|build|dist|\.eggs)/"
246+
247+
- name: Generate coverage report
248+
shell: bash -l {0}
249+
run: |
250+
python -m pytest tests/ --cov=PharmaPy --cov-report=html --cov-report=xml --tb=short
251+
252+
- name: Upload coverage to Codecov
253+
uses: codecov/codecov-action@v3
254+
with:
255+
file: ./coverage.xml
256+
flags: unittests
257+
name: codecov-umbrella
258+
259+
- name: Archive coverage HTML report
260+
uses: actions/upload-artifact@v4
261+
with:
262+
name: coverage-report
263+
path: htmlcov/
264+
265+
- name: Build documentation
266+
shell: bash -l {0}
267+
run: |
268+
cd doc
269+
make html
270+
271+
- name: Archive documentation
272+
uses: actions/upload-artifact@v4
273+
if: success()
274+
with:
275+
name: documentation
276+
path: doc/_build/html/
277+
278+
# Build and packaging - runs on all pushes
279+
build-and-package:
280+
name: Build & Package
281+
runs-on: ubuntu-latest
282+
283+
steps:
284+
- name: Checkout code
285+
uses: actions/checkout@v4
286+
287+
- name: Set up Conda
288+
uses: conda-incubator/setup-miniconda@v3
289+
with:
290+
miniforge-version: latest
291+
activate-environment: pharmapy-build
292+
python-version: '3.11'
293+
auto-activate-base: false
294+
295+
- name: Install build dependencies
296+
shell: bash -l {0}
297+
run: |
298+
pip install build wheel twine
299+
300+
- name: Build source and wheel distributions
301+
shell: bash -l {0}
302+
run: |
303+
python -m build
304+
305+
- name: Verify build
306+
shell: bash -l {0}
307+
run: |
308+
pip install dist/*.whl
309+
python -c "import PharmaPy; print('Package installation successful')"
310+
311+
- name: Archive build artifacts
312+
uses: actions/upload-artifact@v4
313+
with:
314+
name: dist-packages
315+
path: dist/

0 commit comments

Comments
 (0)