Skip to content

Commit 9febc63

Browse files
daniel-crombiedaniel-crombie
committed
Add internal workflow and CI/CD script folders for ongoing development automation
1 parent e392b7b commit 9febc63

13 files changed

Lines changed: 791 additions & 0 deletions

‎.gitea/workflows/README.md‎

Lines changed: 282 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,282 @@
1+
# Gitea Workflows - Internal Development Only
2+
3+
> **Note:** These workflows are for the **primary developer's internal Gitea instance**. The public GitHub repository uses separate workflows in `.github/workflows/`.
4+
5+
---
6+
7+
## Architecture: Modular Workflows
8+
9+
This directory uses a **modular architecture** where jobs are defined once and reused across multiple workflows.
10+
11+
```
12+
.gitea/workflows/
13+
├── jobs/ # Reusable job definitions
14+
│ ├── lint.yml # Linting (flake8, black, isort, pylint)
15+
│ ├── unit-test.yml # Unit tests with coverage
16+
│ ├── integration-test.yml # Integration tests against API
17+
│ └── installation.yml # Installation validation
18+
│
19+
├── quick.yml # Auto: Lint + Unit tests (fast, daily development)
20+
├── integration.yml # Tag integration-*: Integration tests only
21+
├── installation.yml # Tag install-*: Installation validation only
22+
├── full.yml # Tag full-*: Complete validation (all jobs)
23+
│
24+
├── release-test.yml # Tag v*-test*: Release to TestPyPI
25+
└── release-production.yml # Tag v*: Release to PyPI
26+
```
27+
28+
---
29+
30+
## Quick Reference
31+
32+
| Workflow | Trigger | Jobs | Duration |
33+
|----------|---------|------|----------|
34+
| `quick.yml` | Auto (push) | Lint + Unit tests | ~3-5 min |
35+
| `integration.yml` | Tag `integration-*` | Integration tests | ~5-10 min |
36+
| `installation.yml` | Tag `install-*` | Installation validation | ~4-6 min |
37+
| `full.yml` | Tag `full-*` | All jobs (complete) | ~12-18 min |
38+
| `release-test.yml` | Tag `v*-test*` | TestPyPI release | ~5 min |
39+
| `release-production.yml` | Tag `v*` | PyPI release | ~5 min |
40+
41+
---
42+
43+
## Daily Usage
44+
45+
### 1. Normal Development (Automatic - Fast)
46+
```bash
47+
git commit -m "Add watermark feature"
48+
git push
49+
# → quick.yml runs automatically (~3-5 min)
50+
# → Lint + Unit tests (Python 3.10 & 3.14)
51+
```
52+
53+
### 2. Integration Tests Only
54+
```bash
55+
git tag integration-1
56+
git push origin integration-1
57+
# → integration.yml runs (~5-10 min)
58+
# → Integration tests against real API
59+
60+
# Cleanup after:
61+
git push origin --delete integration-1
62+
```
63+
64+
### 3. Installation Validation Only
65+
```bash
66+
git tag install-check
67+
git push origin install-check
68+
# → installation.yml runs (~4-6 min)
69+
# → Validates package installation methods
70+
71+
# Cleanup:
72+
git push origin --delete install-check
73+
```
74+
75+
### 4. Full Validation (Before Release)
76+
```bash
77+
git tag full-v0.2.0-rc1
78+
git push origin full-v0.2.0-rc1
79+
# → full.yml runs (~12-18 min)
80+
# → Lint → Unit → Installation → Integration (all tests)
81+
82+
# Cleanup:
83+
git push origin --delete full-v0.2.0-rc1
84+
```
85+
86+
### 5. Release
87+
```bash
88+
# Test release first
89+
git tag v0.2.0-test1
90+
git push origin v0.2.0-test1
91+
# → Publishes to TestPyPI
92+
93+
# Production release
94+
git tag v0.2.0
95+
git push origin v0.2.0
96+
# → Publishes to PyPI
97+
```
98+
99+
---
100+
101+
## Workflow Details
102+
103+
### `quick.yml` - Fast Daily Development
104+
105+
**Triggers:**
106+
- Automatic on push (only when relevant files change)
107+
- Pull requests
108+
109+
**Runs when these files change:**
110+
- Python files: `*.py`, `ilovepdf/**/*.py`, `tests/**/*.py`, `samples/**/*.py`
111+
- Config files: `pyproject.toml`, `requirements*.txt`, `setup.py`, `setup.cfg`
112+
- Workflow files: `quick.yml`, `jobs/lint.yml`, `jobs/unit-test.yml`
113+
114+
**Skips when only these change:**
115+
- Documentation: `*.md`
116+
- Docker files: `.docker/**`
117+
- Other workflows
118+
119+
**Jobs:**
120+
1. Lint (Python 3.10)
121+
2. Unit Tests (Python 3.10 & 3.14 in parallel)
122+
123+
---
124+
125+
### `integration.yml` - API Integration Tests
126+
127+
**Triggers:**
128+
- Tags matching `integration-*`
129+
130+
**Requirements:**
131+
- `ILOVEPDF_PUBLIC_KEY` secret
132+
- `ILOVEPDF_SECRET_KEY` secret
133+
134+
**Jobs:**
135+
1. Validate API keys
136+
2. Integration tests (Python 3.10 & 3.14)
137+
138+
**Note:** Tests interact with real iLovePDF API
139+
140+
---
141+
142+
### `installation.yml` - Package Installation
143+
144+
**Triggers:**
145+
- Tags matching `install-*`
146+
147+
**Jobs:**
148+
1. Global installation validation
149+
2. User installation validation
150+
3. VCS installation from git
151+
152+
**Tests:**
153+
- Python 3.10 & 3.14
154+
155+
---
156+
157+
### `full.yml` - Complete Validation
158+
159+
**Triggers:**
160+
- Tags matching `full-*`
161+
162+
**Requirements:**
163+
- All secrets must be configured
164+
165+
**Jobs (in order):**
166+
1. Lint
167+
2. Unit Tests (3.10 & 3.14 in parallel)
168+
3. Installation Validation (3.10 & 3.14 in parallel)
169+
4. Validate API Keys
170+
5. Integration Tests (3.10 & 3.14 in parallel)
171+
172+
**Use before:**
173+
- Creating releases
174+
- Major pull requests
175+
- After significant refactoring
176+
177+
---
178+
179+
## Configuration
180+
181+
### Python Versions
182+
- **Min:** 3.10
183+
- **Max:** 3.14
184+
- All workflows test both versions
185+
186+
### Fail-Fast Strategy
187+
- All matrix jobs use `fail-fast: true`
188+
- If one Python version fails, others are cancelled immediately
189+
- Saves time and provides faster feedback
190+
191+
### Required Secrets
192+
Set in Gitea repository settings → Secrets:
193+
- `ILOVEPDF_PUBLIC_KEY` - For integration tests
194+
- `ILOVEPDF_SECRET_KEY` - For integration tests
195+
196+
Get keys at: https://developer.ilovepdf.com/user/projects
197+
198+
---
199+
200+
## Modular Jobs
201+
202+
Each job is defined once in `jobs/` and reused across workflows:
203+
204+
### `jobs/lint.yml`
205+
- Flake8 style checking
206+
- Black formatting verification
207+
- isort import sorting
208+
- Pylint code quality (min score: 8.0)
209+
210+
### `jobs/unit-test.yml`
211+
- Pytest unit tests
212+
- Code coverage (min: 85%)
213+
- Timeout: 30 seconds per test
214+
- Coverage reports (XML + terminal)
215+
216+
### `jobs/integration-test.yml`
217+
- Integration tests against iLovePDF API
218+
- Requires API credentials
219+
- Timeout: 60 seconds per test
220+
- Real API interaction
221+
222+
### `jobs/installation.yml`
223+
- Global installation (`pip install`)
224+
- User installation (`pip install --user`)
225+
- VCS installation (`pip install git+...`)
226+
- Import verification
227+
228+
---
229+
230+
## Troubleshooting
231+
232+
| Issue | Cause | Solution |
233+
|-------|-------|----------|
234+
| `quick.yml` doesn't run | Only docs changed | Expected behavior (docs don't trigger CI) |
235+
| Integration tests skip | Tag doesn't match pattern | Use `integration-*` pattern |
236+
| Integration tests fail immediately | Missing secrets | Configure `ILOVEPDF_PUBLIC_KEY` and `ILOVEPDF_SECRET_KEY` |
237+
| One Python version fails, other cancelled | `fail-fast: true` | Fix the issue and re-run the workflow |
238+
| Installation tests fail | Script path issue | Check `.github/workflows/scripts/install_and_verify.sh` exists |
239+
| Full workflow takes too long | All jobs run sequentially | Expected (~12-18 min for complete validation) |
240+
241+
---
242+
243+
## Best Practices
244+
245+
1. **Daily development:** Push frequently, let `quick.yml` catch issues early (~3-5 min)
246+
2. **Before PRs:** Run `full-*` tag to validate everything (~12-18 min)
247+
3. **API changes:** Run `integration-*` tag to test against real API (~5-10 min)
248+
4. **Installation changes:** Run `install-*` tag to validate package installation (~4-6 min)
249+
5. **Clean up tags:** Delete test tags after use to keep repository clean
250+
6. **Monitor times:** If workflows slow down, optimize or parallelize jobs
251+
252+
---
253+
254+
## Comparison: Modular vs Monolithic
255+
256+
**Before (Monolithic):**
257+
- Single `ci.yml` with all jobs inline
258+
- Changes require editing large file
259+
- Can't run individual validations
260+
- Code duplication across workflows
261+
262+
**After (Modular):**
263+
- Jobs defined once in `jobs/`
264+
- Workflows compose jobs as needed
265+
- Run exactly what you need (granular control)
266+
- No code duplication
267+
- Easy to maintain and extend
268+
269+
---
270+
271+
## For Contributors
272+
273+
These internal workflows **do not affect public contributions**.
274+
275+
Focus on the public GitHub repository workflows in `.github/workflows/`.
276+
277+
Your PR must pass the public CI/CD pipeline.
278+
279+
---
280+
281+
**Maintainer:** Internal use only for primary development
282+
**Last Updated:** 2025-01-11

‎.gitea/workflows/full.yml‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
name: Full Validation
2+
3+
on:
4+
push:
5+
tags:
6+
- "full-*"
7+
8+
jobs:
9+
lint:
10+
name: Lint
11+
uses: ./.gitea/workflows/jobs/lint.yml
12+
with:
13+
python-version: "3.10"
14+
15+
unit_test:
16+
name: Unit Tests
17+
needs: lint
18+
strategy:
19+
fail-fast: true
20+
matrix:
21+
python-version: ["3.10", "3.14"]
22+
uses: ./.gitea/workflows/jobs/unit-test.yml
23+
with:
24+
python-version: ${{ matrix.python-version }}
25+
26+
installation:
27+
name: Installation Validation
28+
needs: unit_test
29+
strategy:
30+
fail-fast: true
31+
matrix:
32+
python-version: ["3.10", "3.14"]
33+
uses: ./.gitea/workflows/jobs/installation.yml
34+
with:
35+
python-version: ${{ matrix.python-version }}
36+
37+
validate_secrets:
38+
name: Validate API Keys
39+
needs: installation
40+
runs-on: ubuntu-latest
41+
container:
42+
image: ghcr.io/catthehacker/ubuntu:act-latest
43+
outputs:
44+
secrets_available: ${{ steps.check.outputs.available }}
45+
steps:
46+
- name: Check if secrets are set
47+
id: check
48+
run: |
49+
if [ -n "${{ secrets.ILOVEPDF_PUBLIC_KEY }}" ] && [ -n "${{ secrets.ILOVEPDF_SECRET_KEY }}" ]; then
50+
echo "available=true" >> $GITHUB_OUTPUT
51+
echo "✓ API keys are configured"
52+
else
53+
echo "available=false" >> $GITHUB_OUTPUT
54+
echo "✗ API keys are missing"
55+
echo "Please configure ILOVEPDF_PUBLIC_KEY and ILOVEPDF_SECRET_KEY in repository secrets"
56+
exit 1
57+
fi
58+
59+
integration_test:
60+
name: Integration Tests
61+
needs: validate_secrets
62+
if: needs.validate_secrets.outputs.secrets_available == 'true'
63+
strategy:
64+
fail-fast: true
65+
matrix:
66+
python-version: ["3.10", "3.14"]
67+
uses: ./.gitea/workflows/jobs/integration-test.yml
68+
with:
69+
python-version: ${{ matrix.python-version }}
70+
secrets:
71+
ILOVEPDF_PUBLIC_KEY: ${{ secrets.ILOVEPDF_PUBLIC_KEY }}
72+
ILOVEPDF_SECRET_KEY: ${{ secrets.ILOVEPDF_SECRET_KEY }}

‎.gitea/workflows/installation.yml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: Installation Validation
2+
3+
on:
4+
push:
5+
tags:
6+
- "install-*"
7+
8+
jobs:
9+
installation:
10+
name: Installation Validation
11+
strategy:
12+
fail-fast: true
13+
matrix:
14+
python-version: ["3.10", "3.14"]
15+
uses: ./.gitea/workflows/jobs/installation.yml
16+
with:
17+
python-version: ${{ matrix.python-version }}

0 commit comments

Comments
 (0)