fix(usage): align cache accounting and GPT-5.6 billing#338
Merged
Conversation
Compute cache-hit numerator and denominator using resolved billing models before aggregating aliases. Expose normalized cache metrics across summaries, timelines, and model breakdown responses. Clamp malformed ratios without changing raw usage storage or requiring a migration.
Consume normalized cache metrics across model, provider, account, credential, and API-key analytics. Render legacy C and fine-grained CR/CW on a dedicated line while hiding zero values. Preserve compatibility fallbacks and add accessible labels and regression coverage.
Add persisted cache-accounting mode and normalized token buckets for existing and new usage events. Backfill legacy rows transactionally and rebuild rollups so pricing and cache-hit analytics use one canonical input total. Migration rollback coverage ensures partial failures can be retried without leaving inconsistent checkpoints.
Normalize included and separate cache token accounting from current CPA payloads without requiring new upstream metadata. Apply response-tier precedence, GPT-5.4 through GPT-5.6 long-context pricing, and distinct cache read and creation rates. Update repositories, rollups, hit-rate calculations, and regression coverage while preserving compatibility with older CPA versions.
Mirror backend cache normalization and service-tier pricing in frontend usage calculations. Keep CR and CW distinct, use normalized total input for cache-hit rates, and handle GPT-5.6 cache creation pricing. Regression coverage includes legacy CPA payloads, long-context boundaries, Flex pricing, and explicit zero prices.
seakee
changed the title
3 tasks
18 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Normalize cache accounting and pricing across current CPA payloads without requiring unreleased upstream metadata.
The change fixes GPT-5.6 cache read/write visibility, cache-hit denominators, long-context billing, and actual response-tier pricing.
Scope
Changes
User Impact
Cache reads and writes now appear consistently in monitoring and analytics. Cost estimates no longer double-count or omit cache tokens, and GPT-5.6 requests use the documented cache, long-context, and service-tier rules.
Compatibility / Runtime Notes
cache_input_modeis optional and provider/model inference is used when it is absent.Data / Security Notes
The migration adds nullable accounting and service-tier columns to
usage_events, backfills them transactionally, and resets rollup checkpoints for recalculation. Raw usage fields, authentication data, key handling, log sanitization, and exports remain unchanged.Risk / Rollback
Risk level: Medium
Rollback notes:
Verification
Commands / evidence:
Frontend result: 86 test files, 740 tests passed.
Manager Server full and race suites passed.
Screenshots / Recordings
N/A. The existing realtime cache-label change is covered by frontend regression tests; no additional visual layout was introduced in the follow-up commits.
Docs
Related
Closes #333
Closes #336
Refs #326
Refs router-for-me/CLIProxyAPI#4193