[Batianxieshen1]

Summary

truncate_embedding_input bounds embedding input by estimating tokens as chars / 4 (with CJK runs costing 4 units each). For structured text a real BPE tokenizer produces up to ~3x more tokens than that estimate, so the guard lets requests through that the provider then rejects with exceed_context_size_error. The reporter measured it on a real directory overview:

text	guard estimate	real tokens	ratio
full .overview.md (28,152 chars)	7,038	20,530	2.92x
truncated to "2,408 estimated" (9,629 chars)	2,408	7,101	2.95x

Result: directory-level overview embeddings fail every time with HTTP 400, so the affected write is reported as failed even though the memory items it summarizes landed. The current workaround is lowering embedding.max_input_tokens globally, which also truncates legitimate long memory items.

Refs #6418. This implements the fail-safe option (option 4) rather than changing the global heuristic — the estimate is still used as a cheap pre-check, but the provider now gets the last word when it disagrees.

What changed

When a provider rejects the request and reports the window it enforces, the embedder re-truncates the batch to that window and retries exactly once.

• _context_window_from_error reads n_ctx / context_length / max_context_length from the error body. llama-server returns n_ctx; the aliases cover other OpenAI-compatible servers.
• _retry_input_within_context re-bounds the strings and returns None — meaning "do not retry" — when:
  • the error carries no window (a plain 400/401/429 is not a length problem),
  • the input is multimodal content parts (no comparable text budget),
  • re-truncating would not change the payload (the window is already wide enough, so resending would fail identically).

Both embed and embed_async use the same path, so sync and async behave alike. Everything else is unchanged: no new dependencies, no config keys, no change to the token estimate itself, and a failed retry surfaces the retry's error rather than masking it.

Validation

• Extracted the two helpers into a harness and drove them with the exact error shape from the report ({'error': {'code': 400, ..., 'n_ctx': 2560}}, max_input_tokens=2400, 31,200-char overview):
  • retries with the input re-bounded to 10,269 chars,
  • no n_ctx → no retry,
  • n_ctx wider than the text → no retry (payload would be unchanged),
  • multimodal input → no retry,
  • context_length alias → retry,
  • non-dict body → no retry.
• Four unit tests added to tests/unit/test_openai_embedder.py covering: retry succeeds, no-window → single attempt + original error, unchanged payload → single attempt, retry failure → second error surfaces.
• python -c "ast.parse(...)" on both changed files.

Not run: the repo's full pytest tests/unit — the openviking package is not installed in this environment (pip install -e . was not possible here), so the new tests have not executed against the real OpenAIDenseEmbedder constructor. CI should be treated as the first real run of the test file; the helper logic itself is covered by the harness above.

Note on the estimate itself

I deliberately did not change chars / 4. Making the heuristic conservative would truncate text that currently fits, which is a behaviour change for every embedding caller. If maintainers prefer to also fix the estimate (option 2 in the report), that can be a follow-up with its own measurements — the retry added here stays correct either way.