Skip to content

Commit 4b9893c

Browse files
mivertowskiclaude
andcommitted
feat: add Python bindings for batch kernel execution via PyO3
Introduce `rustkernel-python` crate providing native Python access to all 106 batch kernels. Users can `pip install rustkernels` and call kernels directly from Python/Jupyter without a REST/gRPC server. - PyKernelRegistry with discovery (by_domain, by_mode, search) and execute - Module-level convenience functions (execute, list_domains, enabled_domains) - Exception hierarchy mapping all KernelError variants - GIL-releasing async bridge (py.allow_threads + internal tokio runtime) - Feature pass-through mirroring the facade crate (--features full) - Python package README, CHANGELOG, publish-python.sh script - CI job for Python 3.10/3.12 matrix (maturin develop + pytest) - Crate excluded from default-members (requires Python dev headers) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 9150697 commit 4b9893c

21 files changed

Lines changed: 1691 additions & 6 deletions

.github/workflows/ci.yml

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -258,3 +258,39 @@ jobs:
258258
- name: Run security audit
259259
working-directory: RustKernels/RustKernels
260260
run: cargo audit
261+
262+
python-bindings:
263+
name: Python Bindings
264+
runs-on: ubuntu-latest
265+
strategy:
266+
matrix:
267+
python-version: ["3.10", "3.12"]
268+
steps:
269+
- uses: actions/checkout@v4
270+
with:
271+
path: RustKernels/RustKernels
272+
273+
- name: Checkout RustCompute
274+
uses: actions/checkout@v4
275+
with:
276+
repository: mivertowski/RustCompute
277+
path: RustCompute/RustCompute
278+
279+
- name: Install Rust toolchain
280+
uses: dtolnay/rust-toolchain@stable
281+
282+
- name: Set up Python ${{ matrix.python-version }}
283+
uses: actions/setup-python@v5
284+
with:
285+
python-version: ${{ matrix.python-version }}
286+
287+
- name: Install maturin and pytest
288+
run: pip install maturin pytest
289+
290+
- name: Build and install Python package
291+
working-directory: RustKernels/RustKernels/crates/rustkernel-python
292+
run: maturin develop --features full
293+
294+
- name: Run Python tests
295+
working-directory: RustKernels/RustKernels/crates/rustkernel-python
296+
run: pytest tests/ -v

.gitignore

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,17 @@ target
2020
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
2121
#.idea/
2222

23+
# Python artifacts
24+
__pycache__/
25+
*.pyc
26+
*.pyo
27+
*.egg-info/
28+
.venv/
29+
dist/
30+
*.whl
31+
*.so
32+
*.pyd
33+
2334
# Generated documentation
2435
docs/book/
2536
docs/rustkernels-executive-overview.aux

CHANGELOG.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Changelog
2+
3+
All notable changes to this project will be documented in this file.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## [0.4.0] - 2025-05-15
9+
10+
### Added
11+
12+
- **Python bindings** (`rustkernel-python`) — native Python access to all 106 batch kernels via PyO3
13+
- `pip install rustkernels` for Python 3.10+
14+
- `KernelRegistry` class with discovery (`kernel_ids`, `by_domain`, `search`) and `execute()` for batch kernels
15+
- Module-level `execute()` convenience function with cached default registry
16+
- Catalog API: `list_domains()`, `total_kernel_count()`, `enabled_domains()`
17+
- Full exception hierarchy mapping all `KernelError` variants
18+
- GIL-releasing async bridge (`py.allow_threads()` + internal tokio runtime)
19+
- Feature pass-through: `--features full` enables all 14 domains
20+
- `scripts/publish-python.sh` for PyPI publishing via maturin
21+
- **Deep RingKernel 0.4.2 integration** across all crates
22+
- Bidirectional `Domain` conversion between RustKernels and RingKernel
23+
- Re-exports: `ControlBlock`, `Backend`, `KernelStatus`, `RuntimeMetrics`, `K2KConfig`, `Priority`
24+
- Submodule re-exports: `checkpoint`, `dispatcher`, `health`, `pubsub`
25+
- **Production-ready kernel execution** — all 106 kernels fully implemented with real dispatch (no stubs)
26+
- **Enterprise modules** in `rustkernel-core`:
27+
- Security: JWT/API key auth, RBAC, multi-tenancy, secrets management
28+
- Observability: Prometheus metrics, OTLP tracing, structured logging, SLO alerting
29+
- Resilience: circuit breakers, exponential retry, timeout propagation, health probes
30+
- Runtime: lifecycle state machine, graceful shutdown, production config presets
31+
- Memory: size-stratified pools, pressure handling, inter-phase reductions
32+
- **Ecosystem service layer** (`rustkernel-ecosystem`):
33+
- Axum REST API (`KernelRouter`)
34+
- Tower middleware (timeout, rate limiting)
35+
- Tonic gRPC server (`KernelGrpcServer`)
36+
- Actix actor integration (`KernelActor`)
37+
- **New kernels**:
38+
- Graph: `GNNInference`, `GraphAttention`
39+
- ML: `EmbeddingGeneration`, `SemanticSimilarity`, `SecureAggregation`, `DrugInteractionPrediction`, `ClinicalPathwayConformance`, `StreamingIsolationForest`, `AdaptiveThreshold`, `SHAPValues`, `FeatureImportance`
40+
- Process Intelligence: `DigitalTwin`, `NextActivityPrediction`, `EventLogImputation`
41+
- **K2K coordination** patterns: `IterativeState`, `ScatterGatherState`, `FanOutTracker`, `PipelineTracker`
42+
- **Type-erased batch execution** via `KernelRegistry::execute_batch(kernel_id, json_bytes)`
43+
- **CLI tool** (`rustkernel-cli`) for kernel management
44+
45+
### Changed
46+
47+
- Workspace expanded to 20 crates (added `rustkernel-python`)
48+
- `rustkernel-python` excluded from `default-members` (requires Python dev headers)
49+
- `pyo3 = "0.23"` added to workspace dependencies
50+
51+
## [0.3.0] - 2025-03-01
52+
53+
### Added
54+
55+
- Initial 14-domain workspace with 106 kernel stubs
56+
- Core traits: `GpuKernel`, `BatchKernel`, `RingKernelHandler`, `IterativeKernel`
57+
- Kernel registry with typed factory registration
58+
- Domain-based licensing system
59+
- Proc macro derive: `#[gpu_kernel]`, `#[derive(KernelMessage)]`
60+
61+
[0.4.0]: https://github.com/mivertowski/RustKernels/compare/v0.3.0...v0.4.0
62+
[0.3.0]: https://github.com/mivertowski/RustKernels/releases/tag/v0.3.0

Cargo.lock

Lines changed: 104 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,30 @@ members = [
2020
"crates/rustkernel-audit",
2121
"crates/rustkernel-cli",
2222
"crates/rustkernel-ecosystem",
23+
"crates/rustkernel-python",
24+
]
25+
26+
# Exclude rustkernel-python from default builds (requires Python dev headers + maturin)
27+
default-members = [
28+
"crates/rustkernels",
29+
"crates/rustkernel-core",
30+
"crates/rustkernel-derive",
31+
"crates/rustkernel-graph",
32+
"crates/rustkernel-ml",
33+
"crates/rustkernel-compliance",
34+
"crates/rustkernel-temporal",
35+
"crates/rustkernel-risk",
36+
"crates/rustkernel-banking",
37+
"crates/rustkernel-behavioral",
38+
"crates/rustkernel-orderbook",
39+
"crates/rustkernel-procint",
40+
"crates/rustkernel-clearing",
41+
"crates/rustkernel-treasury",
42+
"crates/rustkernel-accounting",
43+
"crates/rustkernel-payments",
44+
"crates/rustkernel-audit",
45+
"crates/rustkernel-cli",
46+
"crates/rustkernel-ecosystem",
2347
]
2448

2549
[workspace.package]
@@ -149,6 +173,9 @@ quote = "1.0"
149173
proc-macro2 = "1.0"
150174
darling = "0.20"
151175

176+
# Python bindings
177+
pyo3 = { version = "0.23", features = ["extension-module"] }
178+
152179
# Runtime registration
153180
inventory = "0.3"
154181

0 commit comments

Comments
 (0)