[Bug] Anthropic / Claude prompt caching hit rate is much lower than OpenAI under similar AstrBot usage

[911218sky]

What happened / 发生了什么

When I use Claude in AstrBot, prompt cache hits are much lower than expected.

Under a similar setup, GPT models can hit cache much more often, but Claude often creates cache again instead of reading from cache.

From my testing, this seems to be related to how AstrBot sends Anthropic requests, especially when tools/plugins are enabled.

Reproduce / 如何复现？

1. Use a Claude model in AstrBot.
2. Enable tools or plugins.
3. Send several similar messages in the same conversation.
4. Check the provider-side usage stats.
5. Claude often does not hit cache well, while GPT performs much better in a similar setup.

AstrBot version, deployment method (e.g., Windows Docker Desktop deployment), provider used, and messaging platform used. / AstrBot 版本、部署方式（如 Windows Docker Desktop 部署）、使用的提供商、使用的消息平台适配器

AstrBot v4.26.5
Docker deployment
Claude model via Anthropic-compatible provider path (claude-sonnet-5)
Telegram adapter

OS

Linux

Logs / 报错日志

There may be no Python exception for this issue.

The main problem is visible in provider-side usage stats:

• Claude often creates cache again instead of reading cache.
• GPT cache hit rate is much better under similar usage.

I also tested a local patch and the behavior improved after:

• keeping tool order stable
• adding Anthropic cache-control markers on stable request parts

If needed, I can provide screenshots showing the before/after difference.

Are you willing to submit a PR? / 你愿意提交 PR 吗？

Yes!

[dosubot]

Thanks for the detailed report and for offering to submit a PR!

After looking through the codebase, I can confirm your findings: AstrBot currently does not use Anthropic's explicit cache_control API feature — the string cache_control doesn't appear anywhere in the codebase [1]. The current approach optimizes caching indirectly through:

1. Deterministic tool ordering — FunctionTool.normalize() sorts tools by name for stable serialization [2]
2. Temporary content injection instead of mutating system_prompt — e.g., group chat ICL and KB results are injected as temp user content to avoid invalidating the prefix cache [3]
3. System prompt format — wrapped as [{"type": "text", "text": ...}] for Anthropic compatibility [4]

However, as you identified, without explicit cache_control={"type": "ephemeral"} breakpoints on stable prefixes (system, tools), Anthropic's caching engine may not reliably identify what should be cached, leading to lower hit rates compared to OpenAI which handles prefix caching more automatically.

Your proposed approach (explicit cache breakpoints on system/tools + ensuring stable tool order) aligns well with Anthropic's prompt caching docs and the existing architectural direction of the project. The tool ordering part is already handled by PR #6282, so the main gap is adding the cache_control annotations on the Anthropic request path in astrbot/core/provider/sources/anthropic_source.py [1].

Looking forward to your PR! 🎉

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.