fix(usage): expose cache write accounting semantics#4193
Conversation
Read cache_write_tokens from prompt token details before falling back to response input details. Preserve generic OpenAI usage handling across non-streaming and streaming responses.
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for parsing prompt_tokens_details.cache_write_tokens in OpenAI-style usage metrics, falling back to input_tokens_details.cache_write_tokens if not present, and updates the corresponding unit tests. Feedback on the changes points out that renaming TestParseOpenAIStreamUsageResponsesFields to TestParseOpenAIStreamUsageChatCompletionsFields and changing its payload results in a loss of test coverage for older Responses/Codex-style fields in streaming responses. It is recommended to keep both tests to ensure both formats are covered.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Restore the Responses-style streaming test alongside the new Chat Completions coverage. Prevents regressions in input_tokens detail parsing while addressing the review feedback.
Add an optional cache input mode to the core usage detail contract. The field states whether cache read and creation tokens are already included in input tokens. Its zero value remains unknown so existing plugins and consumers stay backward compatible.
Mark OpenAI, Gemini, and Interactions usage as included-input accounting and Claude usage as separate-input accounting. This removes ambiguity for consumers while preserving all existing token fields and parser behavior. Parser regression tests cover both included and separate cache semantics.
Include the optional cache_input_mode field in usage queue token payloads. Consumers can use the explicit mode when available and continue inferring semantics when older producers omit it. Queue tests verify the new metadata without changing existing JSON fields.
seakee
changed the title
1 participant
Summary
Parse OpenAI Chat Completions cache creation tokens and expose optional cache-input accounting semantics to usage consumers.
The accounting metadata is backward compatible: consumers can use
cache_input_modewhen present and continue inferring provider semantics when older queue producers omit it.Changes
prompt_tokens_details.cache_write_tokensand retain the Responses API fallback.tokens.cache_input_modefield through the Redis usage queue.Testing
go test -race ./internal/runtime/executor/helps ./internal/redisqueue ./sdk/cliproxy/usagego build -o /tmp/cpa-cache-accounting-server ./cmd/servergo test ./...— the unchanged catalog test currently failsTestModelsWithClientVersionReturnsCodexCatalogwithcustom priority = 143, want 129; the modified packages pass.Compatibility
Related
Refs #4180.
Refs seakee/CPA-Manager-Plus#338.
This PR does not modify
internal/translator/**; translator propagation remains maintainer-owned.