Feature Area
Agent capabilities
Is your feature request related to an existing bug?
NA — this is a new capability, not a bug fix.
Describe the solution you'd like
Summary
Add an optional, lightweight pre-inference layer that detects the language of an agent's system prompt and automatically translates it to the language the target model handles best — before the prompt ever reaches the LLM. The user writes prompts in whatever language they're comfortable with; CrewAI handles the optimization transparently.
Motivation (with public sources)
1. Instruction-following quality varies by language.
The Multi-IF benchmark (Meta, arXiv:2410.15553) tested 14 LLMs on multi-turn multilingual instruction following. It found that non-Latin scripts — including Chinese, Hindi, and Russian — exhibit systematically higher instruction-following error rates compared to English. o1-preview dropped from 87.7% accuracy at turn 1 to 70.7% at turn 3 on these languages, while English held up better. For Agent frameworks where system prompts encode critical constraints (role definitions, output formats, safety rules), this quality gap directly impacts reliability.
2. Tokenizer efficiency varies dramatically by model family — and by language.
Independent measurements (PromptQuorum, 2026-05; Presenc AI tokenizer benchmark, 2026-05; Mark Huang's controlled token-count experiment, 2026-04) converge on the same picture:
| Model family |
English tokens/word |
Chinese tokens/char |
English advantage |
| GPT-4o / GPT-5.x |
~1.0–1.3 |
~1.5–2.5 |
English ~20–50% fewer tokens |
| Claude Opus 4.7 |
~1.0–1.1 |
~1.1–1.3 |
English ~5–15% fewer tokens |
| LLaMA 3 (8B/70B) |
~1.0–1.2 |
~2.0–3.5 |
English ~50–65% fewer tokens |
| DeepSeek V4 |
~1.0–1.1 |
~1.0–1.1 |
Near parity |
| Qwen 3.5 |
~1.0–1.1 |
~1.0–1.05 |
Near parity, Chinese slightly cheaper |
The practical implication: a 2000-character Chinese system prompt costs ~3500 tokens on LLaMA 3 but only ~2000 tokens on DeepSeek V4. The same prompt translated to English costs ~1500 tokens on LLaMA 3. For Agent workflows that reuse the same system prompt across many turns, this compounds quickly.
3. The best practice exists but requires manual effort.
PromptQuorum (google-research, 2026-05) documents the optimal pattern for DeepSeek-family models: "English system prompt + Chinese user query". But this pattern differs per model family — Qwen 3.5 handles pure-Chinese prompts natively; LLaMA 3 struggles with any Chinese at all. Today, developers must manually research and maintain per-model, per-language prompt variants. CrewAI already has an i18n prompt infrastructure (src/crewai/translations/en.json) — this proposal extends that from static translations to model-aware dynamic routing.
4. Nobody has built this yet.
A search across LangChain, AutoGPT, CrewAI, Microsoft Agent Framework, and the academic literature (RouteLLM, PickLLM, CLPS) confirms: existing routers optimize for which model to use (cost/quality/latency tradeoffs). None optimize for which language to present to a given model. This is an orthogonal dimension that can stack on top of existing routing.
Proposed Design
A lightweight decision layer inserted between prompt construction and LLM invocation:
User writes system prompt (any language)
│
▼
┌─────────────────────────────────────┐
│ 1. MODEL PROFILE LOOKUP │ ← static JSON config, <1KB per model
│ - preferred language │
│ - tokenizer efficiency ratios │
│ - bilingual capability flag │
├─────────────────────────────────────┤
│ 2. TOKEN COST ESTIMATION │ ← tiktoken / HuggingFace tokenizers
│ - Count tokens in current lang │ microsecond-level, no LLM call
│ - Count tokens if translated │
├─────────────────────────────────────┤
│ 3. UNTRANSLATABLE TERM CHECK │ ← glossary whitelist lookup
│ - Domain terms (API keys, codes) │ "if term in glossary → keep original"
│ - Proper nouns, named entities │
├─────────────────────────────────────┤
│ 4. TRANSLATE (if beneficial) │ ← cached result, one-time cost
│ - Code blocks / JSON → skip │
│ - Structured data → skip │
│ - Natural language → translate │
└─────────────────────────────────────┘
│
▼
Optimized system prompt → LLM
Key design decisions:
- User-overridable:
Agent(..., auto_translate_prompt=False) disables it. Power users who have manually tuned prompts in a specific language keep full control.
- System prompt only: User messages are NOT translated — altering user intent is risky and ethically questionable.
- No output back-translation: Models naturally follow the user's language for responses. A Chinese user asking a question gets a Chinese answer regardless of the system prompt's language. The
"Respond in {user_lang}" instruction can be appended to the translated system prompt as a safety net (~3 tokens).
- Translation caching: System prompts change infrequently. Cache the translated version; re-translate only when the prompt text changes.
- Boundary detection: Code blocks, JSON schemas, API keys, URLs are detected via fence markers and excluded from translation.
What this is NOT
- NOT model-capability routing (RouteLLM, PickLLM already do that; this is complementary)
- NOT cross-lingual prompt optimization (arXiv:2512.02841 optimizes one prompt for all languages; this translates per language instead)
- NOT automatic user-message translation (too risky, alters user intent)
Describe alternatives you've considered
Alternative 1: Documentation-only approach.
Tell users "write system prompts in English for best results." This costs nothing to implement, but shifts the cognitive burden to every user. Most non-English-speaking users won't know this advice exists, and those who do must manually maintain bilingual prompt variants.
Alternative 2: LLM-based language detection + translation at runtime.
Use a small model to detect language and translate. This adds meaningful latency (~200–500ms per request) and cost (extra inference) that partially cancels the token savings. The static-profile approach avoids this entirely — model preferences don't change at runtime.
Alternative 3: Cross-lingual prompt optimization (arXiv:2512.02841 approach).
Train prompts to work equally well in all languages. Promising research direction but not production-ready — the paper reports 5–10% improvement through optimization, not parity. The translate-and-route approach works today with zero training.
Additional context
Scope and non-goals
- Scope: system prompts and tool/function descriptions (the "rules layer" of an agent)
- Non-scope: user messages, creative outputs, conversational turns
- Models that benefit most: English-pretrained models running non-English system prompts (LLaMA 3, Mistral, GPT-4o with Chinese prompts)
- Models that benefit least: Chinese-native models (DeepSeek V4, Qwen 3.5) where tokenizer efficiency is already near parity
- Risk to acknowledge: translation-induced semantic drift — a carefully worded system prompt might lose nuance when machine-translated. Mitigation: opt-out flag, glossary preservation, back-translation consistency check as optional validation step.
Prior art / related work
| Source |
What it covers |
Relationship to this proposal |
| Multi-IF (arXiv:2410.15553) |
Multilingual instruction-following benchmark |
Empirical evidence for the problem |
| PromptQuorum (2026-05) |
Per-model prompting best practices |
Documents the "English SP + native UP" pattern |
| Presenc AI tokenizer benchmark (2026-05) |
Tokenizer efficiency by model family |
Quantifies the token cost asymmetry |
| Mark Huang token experiment (2026-04) |
Cross-tokenizer Chinese vs English token counts |
Independent verification of efficiency claims |
| RouteLLM (arXiv:2406.18665) |
Model-capability routing |
Orthogonal — routes between models, not languages within a model |
| Cross-Lingual Prompt Steerability (arXiv:2512.02841) |
Optimizing one prompt for all languages |
Orthogonal — optimizes prompt content; this proposal changes prompt language |
Integration with CrewAI's existing i18n
CrewAI already has src/crewai/translations/en.json for static prompt templates. This proposal extends that infrastructure from "static JSON translations" to "runtime language routing with model awareness." The existing translation system handles the "what to say"; this proposal handles "what language to say it in for this specific model."
MVP scope estimate
A minimal viable implementation (model profiles for ~10 popular models + token counting + glossary check + translation cache + CrewAI Agent integration) would be roughly 800–1100 lines of Python. The core logic is a pure function with no framework coupling, making it testable in isolation before CrewAI integration.
Willingness to Contribute
I could provide more detailed specifications
Feature Area
Agent capabilities
Is your feature request related to an existing bug?
NA — this is a new capability, not a bug fix.
Describe the solution you'd like
Summary
Add an optional, lightweight pre-inference layer that detects the language of an agent's system prompt and automatically translates it to the language the target model handles best — before the prompt ever reaches the LLM. The user writes prompts in whatever language they're comfortable with; CrewAI handles the optimization transparently.
Motivation (with public sources)
1. Instruction-following quality varies by language.
The Multi-IF benchmark (Meta, arXiv:2410.15553) tested 14 LLMs on multi-turn multilingual instruction following. It found that non-Latin scripts — including Chinese, Hindi, and Russian — exhibit systematically higher instruction-following error rates compared to English. o1-preview dropped from 87.7% accuracy at turn 1 to 70.7% at turn 3 on these languages, while English held up better. For Agent frameworks where system prompts encode critical constraints (role definitions, output formats, safety rules), this quality gap directly impacts reliability.
2. Tokenizer efficiency varies dramatically by model family — and by language.
Independent measurements (PromptQuorum, 2026-05; Presenc AI tokenizer benchmark, 2026-05; Mark Huang's controlled token-count experiment, 2026-04) converge on the same picture:
The practical implication: a 2000-character Chinese system prompt costs ~3500 tokens on LLaMA 3 but only ~2000 tokens on DeepSeek V4. The same prompt translated to English costs ~1500 tokens on LLaMA 3. For Agent workflows that reuse the same system prompt across many turns, this compounds quickly.
3. The best practice exists but requires manual effort.
PromptQuorum (google-research, 2026-05) documents the optimal pattern for DeepSeek-family models: "English system prompt + Chinese user query". But this pattern differs per model family — Qwen 3.5 handles pure-Chinese prompts natively; LLaMA 3 struggles with any Chinese at all. Today, developers must manually research and maintain per-model, per-language prompt variants. CrewAI already has an i18n prompt infrastructure (
src/crewai/translations/en.json) — this proposal extends that from static translations to model-aware dynamic routing.4. Nobody has built this yet.
A search across LangChain, AutoGPT, CrewAI, Microsoft Agent Framework, and the academic literature (RouteLLM, PickLLM, CLPS) confirms: existing routers optimize for which model to use (cost/quality/latency tradeoffs). None optimize for which language to present to a given model. This is an orthogonal dimension that can stack on top of existing routing.
Proposed Design
A lightweight decision layer inserted between prompt construction and LLM invocation:
Key design decisions:
Agent(..., auto_translate_prompt=False)disables it. Power users who have manually tuned prompts in a specific language keep full control."Respond in {user_lang}"instruction can be appended to the translated system prompt as a safety net (~3 tokens).What this is NOT
Describe alternatives you've considered
Alternative 1: Documentation-only approach.
Tell users "write system prompts in English for best results." This costs nothing to implement, but shifts the cognitive burden to every user. Most non-English-speaking users won't know this advice exists, and those who do must manually maintain bilingual prompt variants.
Alternative 2: LLM-based language detection + translation at runtime.
Use a small model to detect language and translate. This adds meaningful latency (~200–500ms per request) and cost (extra inference) that partially cancels the token savings. The static-profile approach avoids this entirely — model preferences don't change at runtime.
Alternative 3: Cross-lingual prompt optimization (arXiv:2512.02841 approach).
Train prompts to work equally well in all languages. Promising research direction but not production-ready — the paper reports 5–10% improvement through optimization, not parity. The translate-and-route approach works today with zero training.
Additional context
Scope and non-goals
Prior art / related work
Integration with CrewAI's existing i18n
CrewAI already has
src/crewai/translations/en.jsonfor static prompt templates. This proposal extends that infrastructure from "static JSON translations" to "runtime language routing with model awareness." The existing translation system handles the "what to say"; this proposal handles "what language to say it in for this specific model."MVP scope estimate
A minimal viable implementation (model profiles for ~10 popular models + token counting + glossary check + translation cache + CrewAI Agent integration) would be roughly 800–1100 lines of Python. The core logic is a pure function with no framework coupling, making it testable in isolation before CrewAI integration.
Willingness to Contribute
I could provide more detailed specifications