@@ -81,25 +81,35 @@ this provides significant speedups.
8181
8282## Installation
8383
84- ** CRITICAL** : SGLang and vLLM have conflicting PyTorch dependencies. You MUST use
85- separate virtual environments.
86-
87- ### vLLM Environment (Default)
84+ ** CRITICAL** : SGLang requires a TWO-environment architecture due to torchao version conflicts.
8885
86+ ### Quick Setup (Recommended)
8987``` bash
90- python -m venv .venv-vllm
91- source .venv-vllm/bin/activate
92- pip install openpipe-art[backend]
88+ # Run the setup script (creates both environments)
89+ chmod +x scripts/setup_sglang.sh
90+ ./scripts/setup_sglang.sh
9391```
9492
95- ### SGLang Environment
96-
93+ ### Manual Setup
9794``` bash
98- python -m venv .venv-sglang
99- source .venv-sglang/bin/activate
100- pip install openpipe-art[sglang]
95+ # 1. Main training environment (ART + Unsloth)
96+ python3.11 -m venv .venv
97+ source .venv/bin/activate
98+ pip install -e " .[sglang]"
99+ deactivate
100+
101+ # 2. SGLang server environment (ISOLATED - no ART)
102+ python3.11 -m venv .venv-sglang-server
103+ source .venv-sglang-server/bin/activate
104+ pip install " sglang[srt]>=0.5.5"
105+ deactivate
106+
107+ # 3. Activate main env to run training
108+ source .venv/bin/activate
101109```
102110
111+ The SGLang backend automatically detects ` .venv-sglang-server ` and uses it for the inference server subprocess.
112+
103113## Usage
104114
105115### Basic Usage (Auto-detect GPUs)
0 commit comments