Skip to content

Commit e4e2dc7

Browse files
Gofingeclaude
andcommitted
Update cumm dependency to >=0.8.2, add CLAUDE.md and custom build script
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 263d6b4 commit e4e2dc7

5 files changed

Lines changed: 159 additions & 4 deletions

File tree

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,4 +116,4 @@ example/libspconv/cumm
116116
example/libspconv/spconv/include
117117
example/libspconv/spconv/src
118118

119-
third_party/boost
119+
third_party/

CLAUDE.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# SpConv - Spatial Sparse Convolution Library
2+
3+
## Overview
4+
SpConv (v2.3.8) implements high-performance sparse convolution operations for 1D/2D/3D/4D data, commonly used in 3D point cloud processing (e.g., autonomous driving). Depends on cumm for GEMM/convolution kernels. Uses PCCM for C++ code generation and pybind11 for bindings.
5+
6+
## Build System
7+
8+
### Build Command (pre-compiled wheel)
9+
```bash
10+
export SPCONV_DISABLE_JIT=1
11+
export CUMM_CUDA_ARCH_LIST=all
12+
export CUMM_CUDA_VERSION=12.8
13+
export BOOST_ROOT=/path/to/boost_1_77_0
14+
pip wheel . --no-deps -w dist/
15+
```
16+
17+
### Key Environment Variables
18+
| Variable | Purpose | Example |
19+
|---|---|---|
20+
| `CUMM_CUDA_VERSION` | Target CUDA version | `"12.8"`, `""` (CPU) |
21+
| `SPCONV_DISABLE_JIT` | `"1"` for pre-compiled wheels | `"1"` |
22+
| `CUMM_CUDA_ARCH_LIST` | GPU architectures | `"all"`, `"8.6"` |
23+
| `BOOST_ROOT` | Boost 1.77.0 headers path | `/path/to/boost_1_77_0` |
24+
| `SPCONV_PYTHON_LIST` | Python versions for build script | `"3.10;3.11;3.12;3.13"` |
25+
| `SPCONV_VERSION_SUFFIX` | Dev version suffix | `"1.0"``2.3.8.dev1000` |
26+
27+
### Build Dependencies
28+
- **Python**: pccm>=0.4.16, ccimport>=0.4.4, pybind11>=2.6.0, fire, numpy
29+
- **Critical dep**: cumm>=0.8.2 (must be pre-installed for wheel builds)
30+
- **C++**: NVIDIA CUDA Toolkit, C++ compiler
31+
- **Headers**: Boost 1.77.0 (header-only, geometry module required)
32+
33+
### Docker Build (Linux manylinux wheels)
34+
```bash
35+
# Download Boost first
36+
mkdir -p third_party
37+
wget https://boostorg.jfrog.io/artifactory/main/release/1.77.0/source/boost_1_77_0.zip -O third_party/boost.zip
38+
unzip third_party/boost.zip -d third_party/boost
39+
40+
docker run --rm \
41+
-e PLAT=manylinux_2_28_x86_64 \
42+
-e CUMM_CUDA_VERSION=12.8 \
43+
-e SPCONV_PYTHON_LIST="3.10;3.11;3.12;3.13" \
44+
-e BOOST_ROOT=/io/third_party/boost/boost_1_77_0 \
45+
-v $(pwd):/io \
46+
scrin/manylinux2014-cuda:cu128-devel-1.0.0 \
47+
bash -c "source /etc/bashrc && /io/tools/build-wheels.sh"
48+
```
49+
50+
### Build Order
51+
cumm must be built and installed BEFORE building spconv (spconv imports from cumm at build time).
52+
53+
### Platform Tags
54+
- CUDA < 12.4: `manylinux2014_x86_64`
55+
- CUDA ≥ 12.4: `manylinux_2_28_x86_64`
56+
57+
## Project Structure
58+
- `spconv/` - Python package
59+
- `csrc/` - C++ source definitions via PCCM
60+
- `sparse/all.py` (99KB) - SpconvOps main binding
61+
- `sparse/convops.py` (99KB) - GemmTuner, ConvTuner, ConvGemmOps
62+
- `sparse/indices.py` (84KB) - Sparse index pair generation
63+
- `sparse/alloc.py` - Memory allocation (thrust, external)
64+
- `pytorch/` - PyTorch integration, quantization
65+
- `core.py` - Kernel parameter definitions (SIMT, Volta, Turing, Ampere)
66+
- `constants.py` - Package constants, Boost path, JIT settings
67+
- `core_cc/` - Generated C++ extensions (build output)
68+
- `test/` - Test suite
69+
- `example/` - MNIST, sparse conv examples
70+
- `tools/` - Build scripts
71+
- `docs/` - API, performance, quantization guides
72+
73+
## Key Files
74+
- `setup.py` - Package build, cumm dependency constraint, kernel compilation
75+
- `spconv/core.py` - **Critical**: GEMM/conv kernel parameters for all GPU architectures
76+
- `spconv/constants.py` - Boost path, JIT settings, weight layout
77+
- `tools/build-wheels.sh` - Linux wheel build script (uses SPCONV_PYTHON_LIST)
78+
79+
## Kernel Architecture Support (core.py)
80+
- `SHUFFLE_SIMT_PARAMS` - f32/f16 kernels for all GPUs (fallback)
81+
- `SHUFFLE_VOLTA_PARAMS` - Volta tensor core (sm_70)
82+
- `SHUFFLE_TURING_PARAMS` - Turing tensor core (sm_75)
83+
- `SHUFFLE_AMPERE_PARAMS` - Ampere (currently empty, uses NVRTC)
84+
- `IMPLGEMM_*_PARAMS` - Implicit GEMM variants for each arch
85+
86+
## Package Naming
87+
- CPU: `spconv`
88+
- CUDA: `spconv-cu{VER}` (e.g., `spconv-cu128`)

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[build-system]
2-
requires = ["setuptools>=41.0", "wheel", "pccm>=0.4.16", "cumm>=0.7.11"]
2+
requires = ["setuptools>=41.0", "wheel", "pccm>=0.4.16", "cumm>=0.8.2"]
33
# requires = ["setuptools>=41.0", "wheel", "pccm>=0.4.0", "cumm @ file:///io/dist/cumm_cu120-0.4.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"]
44
# requires = ["setuptools>=41.0", "wheel", "pccm>=0.4.0", "cumm-cu126 @ file:///io/dist/cumm_cu126-0.7.3-cp313-cp313-manylinux_2_28_x86_64.whl"]
55
build-backend = "setuptools.build_meta"

setup.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,9 @@
3939
cuda_ver_str = cuda_ver.replace(".", "") # 10.2 to 102
4040

4141
RELEASE_NAME += "-cu{}".format(cuda_ver_str)
42-
deps = ["cumm-cu{}>=0.7.11, <0.8.0".format(cuda_ver_str)]
42+
deps = ["cumm-cu{}>=0.8.2".format(cuda_ver_str)]
4343
else:
44-
deps = ["cumm>=0.7.11, <0.8.0"]
44+
deps = ["cumm>=0.8.2"]
4545

4646

4747

tools/build-wheels-custom.sh

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
#!/bin/bash
2+
# Parameterized spconv wheel builder
3+
# Accepts SPCONV_PYTHON_LIST env var (e.g., "3.10;3.11;3.12;3.13")
4+
# Expects cumm wheels at /io/cumm_dist/ for pre-installation
5+
set -e -u -x
6+
7+
function repair_wheel {
8+
wheel="$1"
9+
outpath="$2"
10+
if ! auditwheel show "$wheel"; then
11+
echo "Skipping non-platform wheel $wheel"
12+
else
13+
auditwheel repair "$wheel" --plat "$PLAT" --only-plat -w "$outpath"
14+
fi
15+
}
16+
17+
gcc -v
18+
export SPCONV_DISABLE_JIT="1"
19+
export CUMM_CUDA_ARCH_LIST="all"
20+
21+
# Default to 3.10-3.13 if not specified
22+
SPCONV_PYTHON_LIST="${SPCONV_PYTHON_LIST:-3.10;3.11;3.12;3.13}"
23+
24+
# Derive CUDA version short string for wheel matching
25+
CUDA_VER_SHORT=$(echo "${CUMM_CUDA_VERSION}" | sed 's/\.//')
26+
27+
# Clean up previous build artifacts (may be owned by root from prior Docker runs)
28+
rm -rf /io/build /io/*.egg-info /io/wheelhouse_tmp
29+
mkdir -p /io/wheelhouse_tmp /io/dist
30+
31+
for PYVER in ${SPCONV_PYTHON_LIST//;/ }; do
32+
PYVER2=$(echo "$PYVER" | sed 's/\.//')
33+
PYVER_CP="cp${PYVER2}-cp${PYVER2}"
34+
PYTHON="/opt/python/${PYVER_CP}/bin/python"
35+
PIP="/opt/python/${PYVER_CP}/bin/pip"
36+
37+
echo "=== Building spconv for Python ${PYVER} ==="
38+
39+
# Install build dependencies
40+
"${PIP}" install pccm pybind11 ccimport
41+
42+
# Install pre-built cumm wheel
43+
if [ -d /io/cumm_dist ]; then
44+
CUMM_WHL=$(ls /io/cumm_dist/cumm_cu${CUDA_VER_SHORT}-*-${PYVER_CP}-*.whl 2>/dev/null | head -1)
45+
if [ -n "$CUMM_WHL" ]; then
46+
echo "Installing cumm from: ${CUMM_WHL}"
47+
"${PIP}" install "$CUMM_WHL"
48+
else
49+
echo "WARNING: No cumm wheel found for cu${CUDA_VER_SHORT} ${PYVER_CP}, trying pip install"
50+
"${PIP}" install "cumm-cu${CUDA_VER_SHORT}>=0.8.2"
51+
fi
52+
else
53+
echo "WARNING: /io/cumm_dist not found, trying pip install"
54+
"${PIP}" install "cumm-cu${CUDA_VER_SHORT}>=0.8.2"
55+
fi
56+
57+
"${PIP}" wheel /io/ -v --no-deps -w /io/wheelhouse_tmp
58+
done
59+
60+
# Bundle external shared libraries into the wheels
61+
for whl in /io/wheelhouse_tmp/*.whl; do
62+
repair_wheel "$whl" /io/dist
63+
done
64+
65+
rm -rf /io/wheelhouse_tmp
66+
echo "=== spconv wheels built successfully ==="
67+
ls -la /io/dist/

0 commit comments

Comments
 (0)