|
| 1 | +# SpConv - Spatial Sparse Convolution Library |
| 2 | + |
| 3 | +## Overview |
| 4 | +SpConv (v2.3.8) implements high-performance sparse convolution operations for 1D/2D/3D/4D data, commonly used in 3D point cloud processing (e.g., autonomous driving). Depends on cumm for GEMM/convolution kernels. Uses PCCM for C++ code generation and pybind11 for bindings. |
| 5 | + |
| 6 | +## Build System |
| 7 | + |
| 8 | +### Build Command (pre-compiled wheel) |
| 9 | +```bash |
| 10 | +export SPCONV_DISABLE_JIT=1 |
| 11 | +export CUMM_CUDA_ARCH_LIST=all |
| 12 | +export CUMM_CUDA_VERSION=12.8 |
| 13 | +export BOOST_ROOT=/path/to/boost_1_77_0 |
| 14 | +pip wheel . --no-deps -w dist/ |
| 15 | +``` |
| 16 | + |
| 17 | +### Key Environment Variables |
| 18 | +| Variable | Purpose | Example | |
| 19 | +|---|---|---| |
| 20 | +| `CUMM_CUDA_VERSION` | Target CUDA version | `"12.8"`, `""` (CPU) | |
| 21 | +| `SPCONV_DISABLE_JIT` | `"1"` for pre-compiled wheels | `"1"` | |
| 22 | +| `CUMM_CUDA_ARCH_LIST` | GPU architectures | `"all"`, `"8.6"` | |
| 23 | +| `BOOST_ROOT` | Boost 1.77.0 headers path | `/path/to/boost_1_77_0` | |
| 24 | +| `SPCONV_PYTHON_LIST` | Python versions for build script | `"3.10;3.11;3.12;3.13"` | |
| 25 | +| `SPCONV_VERSION_SUFFIX` | Dev version suffix | `"1.0"` → `2.3.8.dev1000` | |
| 26 | + |
| 27 | +### Build Dependencies |
| 28 | +- **Python**: pccm>=0.4.16, ccimport>=0.4.4, pybind11>=2.6.0, fire, numpy |
| 29 | +- **Critical dep**: cumm>=0.8.2 (must be pre-installed for wheel builds) |
| 30 | +- **C++**: NVIDIA CUDA Toolkit, C++ compiler |
| 31 | +- **Headers**: Boost 1.77.0 (header-only, geometry module required) |
| 32 | + |
| 33 | +### Docker Build (Linux manylinux wheels) |
| 34 | +```bash |
| 35 | +# Download Boost first |
| 36 | +mkdir -p third_party |
| 37 | +wget https://boostorg.jfrog.io/artifactory/main/release/1.77.0/source/boost_1_77_0.zip -O third_party/boost.zip |
| 38 | +unzip third_party/boost.zip -d third_party/boost |
| 39 | + |
| 40 | +docker run --rm \ |
| 41 | + -e PLAT=manylinux_2_28_x86_64 \ |
| 42 | + -e CUMM_CUDA_VERSION=12.8 \ |
| 43 | + -e SPCONV_PYTHON_LIST="3.10;3.11;3.12;3.13" \ |
| 44 | + -e BOOST_ROOT=/io/third_party/boost/boost_1_77_0 \ |
| 45 | + -v $(pwd):/io \ |
| 46 | + scrin/manylinux2014-cuda:cu128-devel-1.0.0 \ |
| 47 | + bash -c "source /etc/bashrc && /io/tools/build-wheels.sh" |
| 48 | +``` |
| 49 | + |
| 50 | +### Build Order |
| 51 | +cumm must be built and installed BEFORE building spconv (spconv imports from cumm at build time). |
| 52 | + |
| 53 | +### Platform Tags |
| 54 | +- CUDA < 12.4: `manylinux2014_x86_64` |
| 55 | +- CUDA ≥ 12.4: `manylinux_2_28_x86_64` |
| 56 | + |
| 57 | +## Project Structure |
| 58 | +- `spconv/` - Python package |
| 59 | + - `csrc/` - C++ source definitions via PCCM |
| 60 | + - `sparse/all.py` (99KB) - SpconvOps main binding |
| 61 | + - `sparse/convops.py` (99KB) - GemmTuner, ConvTuner, ConvGemmOps |
| 62 | + - `sparse/indices.py` (84KB) - Sparse index pair generation |
| 63 | + - `sparse/alloc.py` - Memory allocation (thrust, external) |
| 64 | + - `pytorch/` - PyTorch integration, quantization |
| 65 | + - `core.py` - Kernel parameter definitions (SIMT, Volta, Turing, Ampere) |
| 66 | + - `constants.py` - Package constants, Boost path, JIT settings |
| 67 | + - `core_cc/` - Generated C++ extensions (build output) |
| 68 | +- `test/` - Test suite |
| 69 | +- `example/` - MNIST, sparse conv examples |
| 70 | +- `tools/` - Build scripts |
| 71 | +- `docs/` - API, performance, quantization guides |
| 72 | + |
| 73 | +## Key Files |
| 74 | +- `setup.py` - Package build, cumm dependency constraint, kernel compilation |
| 75 | +- `spconv/core.py` - **Critical**: GEMM/conv kernel parameters for all GPU architectures |
| 76 | +- `spconv/constants.py` - Boost path, JIT settings, weight layout |
| 77 | +- `tools/build-wheels.sh` - Linux wheel build script (uses SPCONV_PYTHON_LIST) |
| 78 | + |
| 79 | +## Kernel Architecture Support (core.py) |
| 80 | +- `SHUFFLE_SIMT_PARAMS` - f32/f16 kernels for all GPUs (fallback) |
| 81 | +- `SHUFFLE_VOLTA_PARAMS` - Volta tensor core (sm_70) |
| 82 | +- `SHUFFLE_TURING_PARAMS` - Turing tensor core (sm_75) |
| 83 | +- `SHUFFLE_AMPERE_PARAMS` - Ampere (currently empty, uses NVRTC) |
| 84 | +- `IMPLGEMM_*_PARAMS` - Implicit GEMM variants for each arch |
| 85 | + |
| 86 | +## Package Naming |
| 87 | +- CPU: `spconv` |
| 88 | +- CUDA: `spconv-cu{VER}` (e.g., `spconv-cu128`) |
0 commit comments