|
1 | 1 | # RustKernels |
2 | 2 |
|
3 | | -GPU kernel library for financial services and analytics. Ported from the DotCompute C# implementation to Rust, using the RingKernel framework. |
| 3 | +**High-performance GPU kernel library for financial services, compliance, and enterprise analytics.** |
4 | 4 |
|
5 | | -**Version**: 0.1.0 |
6 | | -**Author**: Michael Ivertowski |
7 | | -**License**: Apache-2.0 |
| 5 | +[](LICENSE) |
| 6 | +[](https://www.rust-lang.org) |
| 7 | + |
| 8 | +--- |
| 9 | + |
| 10 | +## Overview |
| 11 | + |
| 12 | +RustKernels provides **106 GPU-accelerated algorithms** across **14 domain-specific crates**, purpose-built for financial institutions, compliance teams, and enterprise analytics platforms. The library delivers consistent, production-grade implementations of algorithms previously developed in C# against Orleans grains, now ported to Rust with the RingKernel framework for maximum performance. |
8 | 13 |
|
9 | | -## What This Is |
| 14 | +This is not a general-purpose compute library. RustKernels exists to provide a unified Rust interface to specialized algorithms for graph analytics, machine learning, AML/compliance, risk calculations, process mining, and financial operations. |
10 | 15 |
|
11 | | -A collection of GPU-accelerated algorithms organized into 14 domain-specific crates. The kernels cover graph analytics, machine learning, compliance/AML, risk calculations, and various financial operations. |
| 16 | +## Key Capabilities |
12 | 17 |
|
13 | | -This is not a general-purpose compute library. It exists to provide a consistent Rust interface to algorithms that were previously implemented in C# against Orleans grains. |
| 18 | +| Capability | Description | |
| 19 | +|------------|-------------| |
| 20 | +| **14 Domain Categories** | Graph analytics, ML/AI, compliance, risk, treasury, process intelligence, healthcare, and more | |
| 21 | +| **106 Production Kernels** | Comprehensive coverage from PageRank to drug interaction prediction | |
| 22 | +| **Dual Execution Modes** | Batch (CPU-orchestrated) and Ring (GPU-persistent actors) | |
| 23 | +| **Sub-millisecond Latency** | Ring mode delivers 100-500ns message handling | |
| 24 | +| **K2K Coordination** | Built-in patterns for cross-kernel workflows | |
| 25 | +| **Enterprise Licensing** | Domain-based feature gating for commercial deployment | |
| 26 | + |
| 27 | +## Domain Coverage |
| 28 | + |
| 29 | +| Domain | Crate | Kernels | Highlights | |
| 30 | +|--------|-------|---------|------------| |
| 31 | +| **Graph Analytics** | `rustkernel-graph` | 28 | PageRank, community detection, GNN inference, graph attention networks | |
| 32 | +| **Statistical ML** | `rustkernel-ml` | 17 | Clustering, anomaly detection, NLP embeddings, federated learning, healthcare analytics | |
| 33 | +| **Compliance** | `rustkernel-compliance` | 11 | AML pattern detection, KYC scoring, sanctions screening, entity resolution | |
| 34 | +| **Temporal Analysis** | `rustkernel-temporal` | 7 | ARIMA forecasting, seasonal decomposition, change point detection | |
| 35 | +| **Risk Analytics** | `rustkernel-risk` | 5 | Monte Carlo VaR, credit scoring, stress testing, real-time correlation | |
| 36 | +| **Process Intelligence** | `rustkernel-procint` | 7 | DFG construction, conformance checking, digital twin simulation | |
| 37 | +| **Behavioral Analytics** | `rustkernel-behavioral` | 6 | Profiling, forensic queries, causal graph analysis | |
| 38 | +| **Banking** | `rustkernel-banking` | 1 | Fraud pattern matching | |
| 39 | +| **Order Matching** | `rustkernel-orderbook` | 1 | High-frequency order book engine | |
| 40 | +| **Clearing** | `rustkernel-clearing` | 5 | Netting calculation, DVP matching, settlement execution | |
| 41 | +| **Treasury** | `rustkernel-treasury` | 5 | Liquidity optimization, FX hedging, cash flow forecasting | |
| 42 | +| **Accounting** | `rustkernel-accounting` | 9 | Network generation, GL reconciliation, GAAP violation detection | |
| 43 | +| **Payments** | `rustkernel-payments` | 2 | Payment processing, flow analysis | |
| 44 | +| **Audit** | `rustkernel-audit` | 2 | Feature extraction, hypergraph construction | |
14 | 45 |
|
15 | 46 | ## Execution Modes |
16 | 47 |
|
17 | | -Kernels operate in one of two modes: |
| 48 | +Kernels operate in one of two modes, selected based on latency requirements and workload characteristics: |
| 49 | + |
| 50 | +### Batch Mode |
| 51 | +- **Latency**: 10-50μs launch overhead |
| 52 | +- **State**: CPU memory, transferred to GPU per execution |
| 53 | +- **Use Case**: Heavy periodic computation, large batch processing |
| 54 | +- **Example**: End-of-day risk aggregation, batch AML screening |
| 55 | + |
| 56 | +### Ring Mode |
| 57 | +- **Latency**: 100-500ns per message |
| 58 | +- **State**: Persistent in GPU memory |
| 59 | +- **Use Case**: High-frequency streaming, real-time analytics |
| 60 | +- **Example**: Order book matching, real-time fraud scoring |
| 61 | + |
| 62 | +Most kernels support both modes. The `KernelMetadata` struct indicates supported modes for each kernel. |
18 | 63 |
|
19 | | -- **Batch**: CPU-orchestrated execution. Higher launch overhead (10-50μs), but simpler to reason about. State lives in CPU memory. |
20 | | -- **Ring**: GPU-persistent actors via RingKernel. Lower latency (100-500ns per message), but requires understanding the actor model. State remains on GPU. |
| 64 | +## Recent Additions |
21 | 65 |
|
22 | | -Most kernels support both modes. Choose based on your latency requirements. |
| 66 | +The latest release includes several innovative kernel categories: |
23 | 67 |
|
24 | | -## Domains |
| 68 | +- **Graph Neural Networks**: `GNNInference` and `GraphAttention` for node classification and link prediction using message passing and multi-head attention |
| 69 | +- **NLP/LLM Integration**: `EmbeddingGeneration` and `SemanticSimilarity` for text processing, document similarity, and semantic search |
| 70 | +- **Federated Learning**: `SecureAggregation` with differential privacy for privacy-preserving distributed model training |
| 71 | +- **Healthcare Analytics**: `DrugInteractionPrediction` and `ClinicalPathwayConformance` for clinical decision support |
| 72 | +- **Process Simulation**: `DigitalTwin` for Monte Carlo process simulation and what-if analysis |
| 73 | +- **Streaming ML**: `StreamingIsolationForest` and `AdaptiveThreshold` for online anomaly detection |
| 74 | +- **Explainability**: `SHAPValues` and `FeatureImportance` for model interpretability |
25 | 75 |
|
26 | | -| Domain | Crate | Kernels | |
27 | | -|--------|-------|---------| |
28 | | -| Graph Analytics | `rustkernel-graph` | 28 | |
29 | | -| Statistical ML | `rustkernel-ml` | 17 | |
30 | | -| Compliance | `rustkernel-compliance` | 11 | |
31 | | -| Temporal Analysis | `rustkernel-temporal` | 7 | |
32 | | -| Risk Analytics | `rustkernel-risk` | 5 | |
33 | | -| Banking | `rustkernel-banking` | 1 | |
34 | | -| Behavioral Analytics | `rustkernel-behavioral` | 6 | |
35 | | -| Order Matching | `rustkernel-orderbook` | 1 | |
36 | | -| Process Intelligence | `rustkernel-procint` | 7 | |
37 | | -| Clearing | `rustkernel-clearing` | 5 | |
38 | | -| Treasury | `rustkernel-treasury` | 5 | |
39 | | -| Accounting | `rustkernel-accounting` | 9 | |
40 | | -| Payments | `rustkernel-payments` | 2 | |
41 | | -| Audit | `rustkernel-audit` | 2 | |
| 76 | +## Installation |
42 | 77 |
|
43 | | -## Usage |
| 78 | +Add RustKernels to your `Cargo.toml`: |
44 | 79 |
|
45 | 80 | ```toml |
46 | 81 | [dependencies] |
47 | 82 | rustkernel = "0.1.0" |
48 | 83 | ``` |
49 | 84 |
|
50 | | -```rust |
51 | | -use rustkernel::prelude::*; |
52 | | -use rustkernel::graph::centrality::PageRank; |
| 85 | +### Feature Flags |
53 | 86 |
|
54 | | -let kernel = PageRank::new(); |
55 | | -let metadata = kernel.metadata(); |
56 | | -``` |
57 | | - |
58 | | -Feature flags control which domains are compiled: |
| 87 | +Control which domains are compiled to optimize binary size: |
59 | 88 |
|
60 | 89 | ```toml |
61 | | -# Only what you need |
62 | | -rustkernel = { version = "0.1.0", features = ["graph", "risk"] } |
| 90 | +# Default features (graph, ml, compliance, temporal, risk) |
| 91 | +rustkernel = "0.1.0" |
| 92 | + |
| 93 | +# Selective domain inclusion |
| 94 | +rustkernel = { version = "0.1.0", features = ["graph", "risk", "procint"] } |
63 | 95 |
|
64 | | -# Everything |
| 96 | +# All domains |
65 | 97 | rustkernel = { version = "0.1.0", features = ["full"] } |
66 | 98 | ``` |
67 | 99 |
|
68 | | -Default features include: `graph`, `ml`, `compliance`, `temporal`, `risk`. |
| 100 | +## Quick Start |
| 101 | + |
| 102 | +```rust |
| 103 | +use rustkernel::prelude::*; |
| 104 | +use rustkernel::graph::centrality::PageRank; |
| 105 | + |
| 106 | +// Create kernel instance |
| 107 | +let kernel = PageRank::new(); |
| 108 | + |
| 109 | +// Access metadata |
| 110 | +let metadata = kernel.metadata(); |
| 111 | +println!("Kernel: {}", metadata.id); |
| 112 | +println!("Domain: {:?}", metadata.domain); |
| 113 | + |
| 114 | +// Execute in batch mode |
| 115 | +let input = PageRankInput { |
| 116 | + num_nodes: 1000, |
| 117 | + edges: edges, |
| 118 | + damping_factor: 0.85, |
| 119 | + max_iterations: 100, |
| 120 | + tolerance: 1e-6, |
| 121 | +}; |
| 122 | + |
| 123 | +let result = kernel.execute(input).await?; |
| 124 | +println!("Converged in {} iterations", result.iterations); |
| 125 | +``` |
69 | 126 |
|
70 | 127 | ## Requirements |
71 | 128 |
|
72 | | -- Rust 1.85 or later |
73 | | -- RustCompute (RingKernel) — expected at `../../RustCompute/RustCompute/` relative to this workspace |
74 | | -- CUDA toolkit if you want actual GPU execution; otherwise falls back to CPU |
| 129 | +| Requirement | Version | Notes | |
| 130 | +|-------------|---------|-------| |
| 131 | +| Rust | 1.85+ | Edition 2024 features required | |
| 132 | +| RustCompute | Latest | RingKernel framework (expected at `../../RustCompute/RustCompute/`) | |
| 133 | +| CUDA Toolkit | 12.0+ | Optional; falls back to CPU execution if unavailable | |
75 | 134 |
|
76 | 135 | ## Building and Testing |
77 | 136 |
|
78 | 137 | ```bash |
| 138 | +# Build entire workspace |
79 | 139 | cargo build --workspace |
| 140 | + |
| 141 | +# Run all tests |
80 | 142 | cargo test --workspace |
81 | | -cargo test --package rustkernel-graph # single domain |
| 143 | + |
| 144 | +# Test specific domain |
| 145 | +cargo test --package rustkernel-graph |
| 146 | +cargo test --package rustkernel-ml |
| 147 | + |
| 148 | +# Run benchmarks |
| 149 | +cargo bench --package rustkernel |
| 150 | + |
| 151 | +# Generate documentation |
| 152 | +cargo doc --workspace --no-deps --open |
| 153 | + |
| 154 | +# Lint |
| 155 | +cargo clippy --all-targets --all-features -- -D warnings |
82 | 156 | ``` |
83 | 157 |
|
84 | 158 | ## Project Structure |
85 | 159 |
|
86 | 160 | ``` |
87 | 161 | crates/ |
88 | | -├── rustkernel/ # Facade, re-exports domains |
89 | | -├── rustkernel-core/ # Traits, registry, licensing |
90 | | -├── rustkernel-derive/ # Proc macros |
91 | | -├── rustkernel-cli/ # Command-line tool |
92 | | -└── rustkernel-{domain}/ # 14 domain crates |
| 162 | +├── rustkernel/ # Facade crate, re-exports all domains |
| 163 | +├── rustkernel-core/ # Core traits, registry, licensing, K2K coordination |
| 164 | +├── rustkernel-derive/ # Procedural macros (#[gpu_kernel], #[derive(KernelMessage)]) |
| 165 | +├── rustkernel-cli/ # Command-line interface for kernel management |
| 166 | +└── rustkernel-{domain}/ # 14 domain-specific implementation crates |
| 167 | + ├── src/ |
| 168 | + │ ├── lib.rs # Module exports and registration |
| 169 | + │ ├── messages.rs # Batch kernel I/O types |
| 170 | + │ ├── ring_messages.rs # Ring message types |
| 171 | + │ ├── types.rs # Common domain types |
| 172 | + │ └── {feature}.rs # Kernel implementations |
| 173 | + └── Cargo.toml |
93 | 174 | ``` |
94 | 175 |
|
95 | | -## Status |
| 176 | +## Documentation |
96 | 177 |
|
97 | | -The port is functionally complete. All 106 kernels have been implemented with both BatchKernel and RingKernelHandler traits. K2K (kernel-to-kernel) messaging is in place for cross-kernel coordination patterns. |
| 178 | +- **[Online Documentation](https://mivertowski.github.io/RustKernels/)** - Comprehensive guides and API reference |
| 179 | +- **[Kernel Catalogue](https://mivertowski.github.io/RustKernels/domains/)** - Complete listing of all 106 kernels |
| 180 | +- **[Architecture Guide](https://mivertowski.github.io/RustKernels/architecture/overview.html)** - System design and patterns |
98 | 181 |
|
99 | | -Recent additions include: |
100 | | -- **Graph Neural Networks**: GNNInference, GraphAttention (GAT) |
101 | | -- **NLP/LLM Integration**: EmbeddingGeneration, SemanticSimilarity |
102 | | -- **Federated Learning**: SecureAggregation with differential privacy |
103 | | -- **Healthcare Analytics**: DrugInteractionPrediction, ClinicalPathwayConformance |
104 | | -- **Process Simulation**: DigitalTwin for what-if analysis |
| 182 | +## Contributing |
105 | 183 |
|
106 | | -Test coverage exists for all domains. Some edge cases in the behavioral analytics causal graph module remain flaky. |
| 184 | +Contributions are welcome. Please see [CONTRIBUTING.md](docs/src/appendix/contributing.md) for guidelines. |
107 | 185 |
|
108 | 186 | ## License |
109 | 187 |
|
110 | | -Licensed under Apache-2.0. See [LICENSE](LICENSE) for details. |
| 188 | +Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for details. |
| 189 | + |
| 190 | +--- |
| 191 | + |
| 192 | +**Author**: Michael Ivertowski |
| 193 | +**Version**: 0.1.0 |
0 commit comments