Skip to content

Commit 2da5cfa

Browse files
feat: make Llama.cpp context size configurable via environment
- Add LLAMA_ARG_CTX_SIZE environment variable support in llamacpp service - Increase default context size from 4096 to 8192 tokens for better performance - Allow overriding via .env file for different deployment scenarios - Maintains backward compatibility with existing setups
1 parent c8aadbc commit 2da5cfa

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

docker-compose.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -412,7 +412,7 @@ services:
412412
container_name: llama-decoder-dev-remote
413413
environment:
414414
- LLAMA_ARG_MODEL=/models/model.gguf
415-
- LLAMA_ARG_CTX_SIZE=4096
415+
- LLAMA_ARG_CTX_SIZE=${LLAMA_ARG_CTX_SIZE:-8192}
416416
- LLAMA_ARG_HOST=0.0.0.0
417417
- LLAMA_ARG_PORT=8080
418418
ports:

0 commit comments

Comments
 (0)