Skip to content

Commit 13377c0

Browse files
committed
Update sglang-integration.md
1 parent a872571 commit 13377c0

1 file changed

Lines changed: 22 additions & 12 deletions

File tree

docs/sglang-integration.md

Lines changed: 22 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -81,25 +81,35 @@ this provides significant speedups.
8181

8282
## Installation
8383

84-
**CRITICAL**: SGLang and vLLM have conflicting PyTorch dependencies. You MUST use
85-
separate virtual environments.
86-
87-
### vLLM Environment (Default)
84+
**CRITICAL**: SGLang requires a TWO-environment architecture due to torchao version conflicts.
8885

86+
### Quick Setup (Recommended)
8987
```bash
90-
python -m venv .venv-vllm
91-
source .venv-vllm/bin/activate
92-
pip install openpipe-art[backend]
88+
# Run the setup script (creates both environments)
89+
chmod +x scripts/setup_sglang.sh
90+
./scripts/setup_sglang.sh
9391
```
9492

95-
### SGLang Environment
96-
93+
### Manual Setup
9794
```bash
98-
python -m venv .venv-sglang
99-
source .venv-sglang/bin/activate
100-
pip install openpipe-art[sglang]
95+
# 1. Main training environment (ART + Unsloth)
96+
python3.11 -m venv .venv
97+
source .venv/bin/activate
98+
pip install -e ".[sglang]"
99+
deactivate
100+
101+
# 2. SGLang server environment (ISOLATED - no ART)
102+
python3.11 -m venv .venv-sglang-server
103+
source .venv-sglang-server/bin/activate
104+
pip install "sglang[srt]>=0.5.5"
105+
deactivate
106+
107+
# 3. Activate main env to run training
108+
source .venv/bin/activate
101109
```
102110

111+
The SGLang backend automatically detects `.venv-sglang-server` and uses it for the inference server subprocess.
112+
103113
## Usage
104114

105115
### Basic Usage (Auto-detect GPUs)

0 commit comments

Comments
 (0)