Skip to content

fix: normalize multimodal usage details#6248

Open
fengnian2025 wants to merge 5 commits into
QuantumNous:mainfrom
fengnian2025:codex/fix-multimodal-usage-details
Open

fix: normalize multimodal usage details#6248
fengnian2025 wants to merge 5 commits into
QuantumNous:mainfrom
fengnian2025:codex/fix-multimodal-usage-details

Conversation

@fengnian2025

@fengnian2025 fengnian2025 commented Jul 16, 2026

Copy link
Copy Markdown

⚠️ 提交说明 / PR Notice

Important

  • 请提供人工撰写的简洁摘要,避免直接粘贴未经整理的 AI 输出。

📝 变更描述 / Description

完善多模态 usage 明细的归一化与协议透传:Gemini 在 modality 明细不完整时,将 aggregate token 与已知图片、音频和 reasoning 明细之间的正差额归入文本 token,因为发现as资源可能不会直接返回textToken;同时补充 OpenAI output_tokens_details 的数据结构否则会丢失明细、深拷贝及 Responses/Chat 转换支持。这样结算使用的 token 分类与上游 aggregate usage 保持一致,并保留原生输出明细。

本提交由 AI 辅助实现和测试;当前 Git 提交者不是该仓库历史核心维护者。

🚀 变更类型 / Type of change

  • 🐛 Bug 修复 (Bug fix) - 请关联对应 Issue,避免将设计取舍、理解偏差或预期不一致直接归类为 bug
  • ✨ 新功能 (New feature) - 重大特性建议先通过 Issue 沟通
  • ⚡ 性能优化 / 重构 (Refactor)
  • 📝 文档更新 (Documentation)

🔗 关联任务 / Related Issue

✅ 提交前检查项 / Checklist

  • 人工确认: 我已亲自整理并撰写此描述,没有直接粘贴未经处理的 AI 输出。
  • 非重复提交: 我已搜索现有的 IssuesPRs,确认不是重复提交。
  • Bug fix 说明: 若此 PR 标记为 Bug fix,我已提交或关联对应 Issue,且不会将设计取舍、预期不一致或理解偏差直接归类为 bug。
  • 变更理解: 我已理解这些更改的工作原理及可能影响。
  • 范围聚焦: 本 PR 未包含任何与当前任务无关的代码改动。
  • 本地验证: 已在本地运行并通过测试或手动验证,维护者可以据此复核结果。
  • 安全合规: 代码中无敏感凭据,且符合项目代码规范。

📸 运行证明 / Proof of Work

go test ./...
PASS

Summary by CodeRabbit

  • New Features
    • Usage details now separately show image input vs image output token breakdowns.
    • Billing, quota, and usage conversion now include output token details when available.
    • Pricing/billing UI now reflects image input tokens and displays image output token counts.
  • Bug Fixes
    • Improved reconciliation when token totals/details are missing or inconsistent across conversions.
    • Prevented negative completion-token values in reported usage.
    • Ensured output token details are properly mapped and deep-copied during conversions.
  • Documentation
    • Added the missing “Image output” i18n label across supported locales.

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4544ce2e-93a8-4a90-a585-ecb5aefdf6c8

📥 Commits

Reviewing files that changed from the base of the PR and between cea1782 and c8e96e8.

📒 Files selected for processing (1)
  • relay/channel/openai/image_stream_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • relay/channel/openai/image_stream_test.go

Walkthrough

Changes

Usage token accounting

Layer / File(s) Summary
OpenAI usage contract and cloning
dto/openai_response.go, dto/billing_usage.go, dto/billing_usage_test.go
Adds, detects, serializes, and deep-copies OutputTokensDetails.
OpenAI usage propagation
relay/channel/openai/..., service/relayconvert/internal/oai_*/...
Maps output details into completion usage across normalization and response conversion paths.
Gemini usage reconciliation
service/billing_usage.go, service/billing_usage_test.go, service/relayconvert/internal/gemini_chat/...
Reconciles token remainders, derives totals, includes reasoning tokens, and clamps negative completion counts.
Image usage reporting and display
service/text_quota.go, service/text_quota_test.go, web/classic/..., web/default/...
Separates image input and output usage in logs, pricing, breakdowns, types, and translations.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Possibly related PRs

Poem

I’m a rabbit counting tokens bright,
Output details hop into sight.
Gemini finds text in the hay,
Negative totals bound away.
Input and output each chart—
Cloned carrots stay worlds apart!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 4.17% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: normalizing multimodal usage details across billing and relay conversion paths.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@relay/channel/openai/image_stream_test.go`:
- Around line 39-58: Update
TestNormalizeOpenAIImageUsagePreservesProvidedOutputDetails and
TestNormalizeOpenAIImageUsageCopiesNativeOutputDetails to use testify/assert for
all non-fatal value checks, and add the assert import to the file. Keep the
existing test setup and expectations unchanged.

In `@service/billing_usage_test.go`:
- Around line 7-39: Replace testify/require with testify/assert in
service/billing_usage_test.go lines 7-39 and
service/relayconvert/internal/gemini_chat/to_oai_chat_resp_test.go lines 7-32,
updating every require.Equal and require.Zero assertion in the affected tests to
the corresponding assert method; no other test behavior needs changing.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0da8649f-3ce3-48ea-bd01-0a41db157e8a

📥 Commits

Reviewing files that changed from the base of the PR and between a63364d and 17a10f5.

📒 Files selected for processing (12)
  • dto/billing_usage.go
  • dto/billing_usage_test.go
  • dto/openai_response.go
  • relay/channel/openai/image_stream_test.go
  • relay/channel/openai/relay_image.go
  • service/billing_usage.go
  • service/billing_usage_test.go
  • service/relayconvert/internal/gemini_chat/to_oai_chat_resp.go
  • service/relayconvert/internal/gemini_chat/to_oai_chat_resp_test.go
  • service/relayconvert/internal/oai_chat/to_oai_responses_resp.go
  • service/relayconvert/internal/oai_responses/to_oai_chat_resp.go
  • service/relayconvert/internal/oai_responses/to_oai_chat_resp_test.go

Comment thread relay/channel/openai/image_stream_test.go
Comment thread service/billing_usage_test.go Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@web/classic/src/helpers/render.jsx`:
- Around line 1635-1638: The image token handling must preserve legacy usage
logs in both affected sites. In web/classic/src/helpers/render.jsx lines
1635-1638, remove imageInputTokenCount from the destructuring defaults and
compute it afterward using the legacy condition image_input == null &&
image_ratio != null, mapping legacy image_output values to input pricing while
preserving current behavior. In
web/default/src/features/usage-logs/components/dialogs/details-dialog.tsx lines
436-445, apply the same heuristic so legacy image_output values are labeled
“Image input”; no other sites require changes.

In `@web/default/src/i18n/locales/fr.json`:
- Line 2275: Update the French translation for the “Image output” key in fr.json
to the idiomatic “Sortie d’image”, matching the apostrophe style used by the
adjacent translation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 774ba158-8cbf-4a88-8334-ba547efd59fd

📥 Commits

Reviewing files that changed from the base of the PR and between 17a10f5 and 1777c3c.

📒 Files selected for processing (12)
  • service/text_quota.go
  • service/text_quota_test.go
  • web/classic/src/helpers/render.jsx
  • web/default/src/features/usage-logs/components/dialogs/details-dialog.tsx
  • web/default/src/features/usage-logs/types.ts
  • web/default/src/i18n/locales/en.json
  • web/default/src/i18n/locales/fr.json
  • web/default/src/i18n/locales/ja.json
  • web/default/src/i18n/locales/ru.json
  • web/default/src/i18n/locales/vi.json
  • web/default/src/i18n/locales/zh-TW.json
  • web/default/src/i18n/locales/zh.json

Comment thread web/classic/src/helpers/render.jsx
Comment thread web/default/src/i18n/locales/fr.json Outdated
@fengnian2025

fengnian2025 commented Jul 16, 2026

Copy link
Copy Markdown
Author

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4544ce2e-93a8-4a90-a585-ecb5aefdf6c8

📥 Commits
Reviewing files that changed from the base of the PR and between cea1782 and c8e96e8.

📒 Files selected for processing (1)

  • relay/channel/openai/image_stream_test.go

🚧 Files skipped from review as they are similar to previous changes (1)

  • relay/channel/openai/image_stream_test.go

Walkthrough

Changes

Usage token accounting

Layer / File(s) Summary
OpenAI usage contract and cloning
dto/openai_response.go, dto/billing_usage.go, dto/billing_usage_test.go Adds, detects, serializes, and deep-copies OutputTokensDetails.
OpenAI usage propagation
relay/channel/openai/..., service/relayconvert/internal/oai_*/... Maps output details into completion usage across normalization and response conversion paths.
Gemini usage reconciliation
service/billing_usage.go, service/billing_usage_test.go, service/relayconvert/internal/gemini_chat/... Reconciles token remainders, derives totals, includes reasoning tokens, and clamps negative completion counts.
Image usage reporting and display
service/text_quota.go, service/text_quota_test.go, web/classic/..., web/default/... Separates image input and output usage in logs, pricing, breakdowns, types, and translations.
Estimated code review effort: 3 (Moderate) | ~30 minutes

Possibly related PRs

Poem

I’m a rabbit counting tokens bright,
Output details hop into sight.
Gemini finds text in the hay,
Negative totals bound away.
Input and output each chart—
Cloned carrots stay worlds apart!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 4.17% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: normalizing multimodal usage details across billing and relay conversion paths.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)

  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4544ce2e-93a8-4a90-a585-ecb5aefdf6c8

📥 Commits
Reviewing files that changed from the base of the PR and between cea1782 and c8e96e8.

📒 Files selected for processing (1)

  • relay/channel/openai/image_stream_test.go

🚧 Files skipped from review as they are similar to previous changes (1)

  • relay/channel/openai/image_stream_test.go

Walkthrough

Changes

Usage token accounting

Layer / File(s) Summary
OpenAI usage contract and cloning
dto/openai_response.go, dto/billing_usage.go, dto/billing_usage_test.go Adds, detects, serializes, and deep-copies OutputTokensDetails.
OpenAI usage propagation
relay/channel/openai/..., service/relayconvert/internal/oai_*/... Maps output details into completion usage across normalization and response conversion paths.
Gemini usage reconciliation
service/billing_usage.go, service/billing_usage_test.go, service/relayconvert/internal/gemini_chat/... Reconciles token remainders, derives totals, includes reasoning tokens, and clamps negative completion counts.
Image usage reporting and display
service/text_quota.go, service/text_quota_test.go, web/classic/..., web/default/... Separates image input and output usage in logs, pricing, breakdowns, types, and translations.
Estimated code review effort: 3 (Moderate) | ~30 minutes

Possibly related PRs

Poem

I’m a rabbit counting tokens bright,
Output details hop into sight.
Gemini finds text in the hay,
Negative totals bound away.
Input and output each chart—
Cloned carrots stay worlds apart!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 4.17% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: normalizing multimodal usage details across billing and relay conversion paths.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)

  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

@fengnian2025 The quoted status indicates that reviews are currently paused and the last run found no actionable comments.

When you are ready, comment @coderabbitai resume to re-enable automatic reviews, or @coderabbitai review to run a one-off review.

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

@fengnian2025 The status is unchanged: reviews for #6248 are paused, and the last review reported no actionable findings.

When ready, use @coderabbitai resume to restore automatic reviews or @coderabbitai review for a one-time review.

@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

ast-grep outline \
  dto/billing_usage_test.go \
  relay/channel/openai/image_stream_test.go \
  service/billing_usage_test.go \
  service/relayconvert/internal/gemini_chat/to_oai_chat_resp_test.go \
  service/relayconvert/internal/oai_responses/to_oai_chat_resp_test.go \
  --items all

rg -n -C 3 \
  'OutputTokensDetails|UsageFromChatUsage|UsageFromResponsesUsage|normalizeOpenAIUsage|UsageFromGeminiMetadata|usageFromGeminiBillingUsage' \
  dto relay/channel/openai service

Length of output: 34558


🏁 Script executed:

#!/bin/bash
set -euo pipefail

ast-grep outline service/relayconvert/internal/oai_chat --items all

sed -n '105,165p' service/relayconvert/internal/oai_chat/to_oai_responses_resp.go

fd --type f --extension go . service/relayconvert/internal/oai_chat | sort | while read -r file; do
  case "$file" in
    *_test.go)
      echo "===== $file ====="
      ast-grep outline "$file" --items all
      ;;
  esac
done

Length of output: 11171


🏁 Script executed:

#!/bin/bash
set -euo pipefail

sed -n '1,155p' service/relayconvert/internal/oai_chat/to_oai_responses_resp_test.go

Length of output: 5799


🏁 Script executed:

go test ./service/relayconvert/internal/oai_chat

Length of output: 3489


@fengnian2025 The focused oai_chat package suite passes:

ok github.com/QuantumNous/new-api/service/relayconvert/internal/oai_chat 0.016s

This confirms the current baseline is green. The proposed OutputTokensDetails Chat → Responses regression test still needs to be added to validate the newly identified passthrough and deep-copy behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants