Skip to content

Commit d0c10e6

Browse files
committed
refactor: Overhaul Python SDK CI/CD with a dedicated release workflow, broadened CI triggers, and updated build configurations.
1 parent 25ead3e commit d0c10e6

9 files changed

Lines changed: 973 additions & 194 deletions

File tree

.coverage

52 KB
Binary file not shown.

.github/WORKFLOW.md

Lines changed: 100 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,16 @@
22

33
## Overview
44

5-
The Python SDK has its own self-contained CI/CD workflow located in `sdk/python/.github/workflows/`. This allows the SDK to be managed independently and potentially moved to its own repository in the future.
5+
The LNMP Python SDK has two main GitHub Actions workflows for continuous integration and release automation.
66

77
## Workflow Structure
88

9-
### 1. **ci.yml** - Main CI/CD Pipeline
9+
### 1. **ci.yml** - Continuous Integration
1010

1111
**Triggers:**
12-
- Push to `main` or `develop` branches (when SDK files change)
13-
- Pull requests to `main` (when SDK files change)
14-
- Release events
12+
- Push to `main` or `develop` branches
13+
- Pull requests to `main`
14+
- Manual workflow dispatch
1515

1616
**Jobs:**
1717

@@ -25,16 +25,16 @@ The Python SDK has its own self-contained CI/CD workflow located in `sdk/python/
2525
- Runs pytest with coverage reporting
2626
- Uploads coverage to Codecov
2727

28-
#### Build Wheels
29-
- Only runs on release events
28+
#### Build Wheels (Manual Only)
29+
- Triggered via `workflow_dispatch`
3030
- Builds native wheels for:
31-
- Linux (x86_64, aarch64)
31+
- Linux (x86_64)
3232
- macOS (x86_64, arm64)
3333
- Windows (x86_64)
3434
- Uploads wheels as artifacts
3535

36-
#### Publish to PyPI
37-
- Only runs for non-prerelease releases
36+
#### Publish to PyPI (Manual Only)
37+
- Triggered via `workflow_dispatch`
3838
- Downloads all platform wheels
3939
- Publishes to PyPI using trusted publisher (OIDC)
4040

@@ -43,45 +43,107 @@ The Python SDK has its own self-contained CI/CD workflow located in `sdk/python/
4343
- Executes performance benchmarks
4444
- Uploads results as artifacts
4545

46-
## Path Filtering
46+
---
4747

48-
All jobs use path filtering to only run when SDK-specific files change:
49-
```yaml
50-
paths:
51-
- 'sdk/python/**'
52-
- '.github/workflows/python-ci.yml'
53-
```
48+
### 2. **release.yml** - Automated Releases
49+
50+
**Triggers:**
51+
- Push tags matching `v*` (e.g., `v0.5.7`)
52+
53+
**Jobs:**
5454

55-
## Independence from Main Repo
55+
#### Test
56+
- Matrix testing across Python 3.9-3.12 and all platforms
57+
- Ensures quality before release
5658

57-
This structure allows the SDK to:
58-
1. Have its own release cycle
59-
2. Be tested independently
60-
3. Maintain separate versioning
61-
4. Eventually move to its own repository with minimal changes
59+
#### Build Wheels
60+
- Builds wheels for all supported platforms:
61+
- Linux x86_64
62+
- macOS universal2 (x86_64 + arm64)
63+
- Windows x86_64
64+
- Uploads as artifacts
65+
66+
#### Build Source Distribution
67+
- Creates source distribution (sdist)
68+
- Uploads as artifact
6269

63-
## Future Migration
70+
#### Publish to PyPI
71+
- Downloads all wheels and sdist
72+
- Publishes to PyPI using trusted publisher
73+
- Only runs for version tags (not pre-releases)
6474

65-
To move the SDK to a separate repository:
66-
1. Copy `sdk/python/` to new repo root
67-
2. Update path filters in workflow (remove `sdk/python/` prefix)
68-
3. Update checkout paths
69-
4. Set up PyPI trusted publisher for new repo
70-
5. Update documentation links
75+
#### Create GitHub Release
76+
- Creates GitHub release with changelog
77+
- Attaches all wheels and sdist
78+
- Marks pre-releases appropriately
7179

7280
## Environment Variables
7381

74-
The workflow uses:
82+
Both workflows use:
7583
- `CARGO_TERM_COLOR=always` - Colored Cargo output
7684
- `RUST_BACKTRACE=1` - Full backtraces on errors
85+
- `PYTHONPATH=$PWD:$PYTHONPATH` - For local package imports
86+
87+
## PyPI Publishing Setup
7788

78-
## Secrets Required
89+
### Trusted Publisher (Recommended) ✅
7990

80-
For PyPI publishing, configure:
81-
- **Trusted Publisher (Recommended)**: Configure at https://pypi.org/manage/account/publishing/
82-
- Owner: `lnmplang`
83-
- Repository: `lnmp-protocol` (or new SDK repo name)
84-
- Workflow: `ci.yml`
85-
- Environment: `pypi`
91+
Configure at https://pypi.org/manage/account/publishing/
92+
93+
**Settings:**
94+
- **Owner**: Your GitHub username or organization
95+
- **Repository**: `lnmp-sdk-python`
96+
- **Workflow**: `release.yml`
97+
- **Environment**: `pypi`
8698

8799
No manual tokens required with trusted publishers!
100+
101+
### Alternative: Manual Token
102+
103+
If trusted publisher isn't available:
104+
1. Generate PyPI API token at https://pypi.org/manage/account/token/
105+
2. Add as GitHub secret: `PYPI_TOKEN`
106+
3. Update workflow to use token-based authentication
107+
108+
## Release Process
109+
110+
### Creating a Release
111+
112+
```bash
113+
# 1. Update version in pyproject.toml and Cargo.toml
114+
# 2. Commit changes
115+
git add pyproject.toml Cargo.toml
116+
git commit -m "chore: bump version to 0.5.7"
117+
118+
# 3. Create and push tag
119+
git tag v0.5.7
120+
git push origin main
121+
git push origin v0.5.7
122+
123+
# 4. GitHub Actions automatically:
124+
# - Runs tests
125+
# - Builds wheels for all platforms
126+
# - Publishes to PyPI
127+
# - Creates GitHub Release
128+
```
129+
130+
### Pre-release
131+
132+
```bash
133+
# Use pre-release tag format
134+
git tag v0.5.7-beta.1
135+
git push origin v0.5.7-beta.1
136+
137+
# Workflow will create GitHub release but skip PyPI
138+
```
139+
140+
## Required GitHub Secrets
141+
142+
- None required if using PyPI trusted publisher
143+
- `PYPI_TOKEN` - Only if using manual token authentication
144+
145+
## Permissions
146+
147+
Workflows require:
148+
- `contents: write` - For creating GitHub releases
149+
- `id-token: write` - For PyPI trusted publishing

.github/workflows/ci.yml

Lines changed: 11 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -3,15 +3,9 @@ name: Python SDK CI/CD
33
on:
44
push:
55
branches: [ main, develop ]
6-
paths:
7-
- 'sdk/python/**'
8-
- '.github/workflows/python-ci.yml'
96
pull_request:
107
branches: [ main ]
11-
paths:
12-
- 'sdk/python/**'
13-
release:
14-
types: [published]
8+
workflow_dispatch:
159

1610
env:
1711
CARGO_TERM_COLOR: always
@@ -24,9 +18,6 @@ jobs:
2418
quality:
2519
name: Code Quality (Rust)
2620
runs-on: ubuntu-latest
27-
defaults:
28-
run:
29-
working-directory: sdk/python
3021

3122
steps:
3223
- uses: actions/checkout@v4
@@ -44,8 +35,8 @@ jobs:
4435
~/.cargo/registry/index/
4536
~/.cargo/registry/cache/
4637
~/.cargo/git/db/
47-
sdk/python/target/
48-
key: ${{ runner.os }}-cargo-${{ hashFiles('sdk/python/Cargo.lock') }}
38+
target/
39+
key: ${{ runner.os }}-cargo-${{ hashFiles('Cargo.lock') }}
4940

5041
- name: Run rustfmt
5142
run: cargo fmt --all -- --check
@@ -69,10 +60,6 @@ jobs:
6960
os: [ubuntu-latest, macos-latest, windows-latest]
7061
python-version: ['3.9', '3.10', '3.11', '3.12']
7162

72-
defaults:
73-
run:
74-
working-directory: sdk/python
75-
7663
steps:
7764
- uses: actions/checkout@v4
7865

@@ -104,13 +91,15 @@ jobs:
10491
run: maturin develop --release
10592

10693
- name: Run Python Tests
107-
run: pytest tests/ -v --cov=lnmp --cov-report=xml
94+
run: |
95+
export PYTHONPATH=$PWD:$PYTHONPATH
96+
pytest tests/ -v --cov=lnmp --cov-report=xml
10897
10998
- name: Upload Coverage
11099
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.11'
111100
uses: codecov/codecov-action@v3
112101
with:
113-
file: ./sdk/python/coverage.xml
102+
file: ./coverage.xml
114103
flags: python-sdk
115104

116105
# ============================================
@@ -120,7 +109,7 @@ jobs:
120109
name: Build Wheels (${{ matrix.target }})
121110
runs-on: ${{ matrix.os }}
122111
needs: test
123-
if: github.event_name == 'release'
112+
if: github.event_name == 'workflow_dispatch'
124113
strategy:
125114
matrix:
126115
include:
@@ -144,11 +133,6 @@ jobs:
144133
- os: windows-latest
145134
target: x86_64-pc-windows-msvc
146135
name: windows-x86_64
147-
148-
defaults:
149-
run:
150-
working-directory: sdk/python
151-
152136
steps:
153137
- uses: actions/checkout@v4
154138

@@ -172,7 +156,7 @@ jobs:
172156
uses: actions/upload-artifact@v3
173157
with:
174158
name: wheels-${{ matrix.name }}
175-
path: sdk/python/dist/*.whl
159+
path: dist/*.whl
176160

177161
# ============================================
178162
# Job 4: Publish to PyPI
@@ -181,7 +165,7 @@ jobs:
181165
name: Publish to PyPI
182166
runs-on: ubuntu-latest
183167
needs: build
184-
if: github.event_name == 'release' && !github.event.release.prerelease
168+
if: github.event_name == 'workflow_dispatch'
185169
environment:
186170
name: pypi
187171
url: https://pypi.org/p/lnmp
@@ -213,9 +197,6 @@ jobs:
213197
runs-on: ubuntu-latest
214198
needs: quality
215199
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
216-
defaults:
217-
run:
218-
working-directory: sdk/python
219200

220201
steps:
221202
- uses: actions/checkout@v4
@@ -240,4 +221,4 @@ jobs:
240221
uses: actions/upload-artifact@v3
241222
with:
242223
name: benchmarks
243-
path: sdk/python/BENCHMARKS.md
224+
path: BENCHMARKS.md

0 commit comments

Comments
 (0)