Skip to content

feat: add adaptive Codex quota cooldown#4399

Draft
936220035 wants to merge 6 commits into
router-for-me:devfrom
936220035:codex/cpa-adaptive-cooldown-upstream-20260717
Draft

feat: add adaptive Codex quota cooldown#4399
936220035 wants to merge 6 commits into
router-for-me:devfrom
936220035:codex/cpa-adaptive-cooldown-upstream-20260717

Conversation

@936220035

Copy link
Copy Markdown

Summary

  • classify Codex HTTP 429 responses into transient throttling and explicit usage-limit exhaustion
  • use bounded adaptive backoff for transient throttling, while preserving provider reset times for usage limits
  • add non-consuming WHAM usage probes with shared quota-bucket deduplication and strict recovery checks
  • persist the optional next probe time and migrate legacy cooldown state without breaking old files
  • keep the adaptive policy disabled by default
  • include prerequisite Go 1.26 race/vet fixes for async test cleanup, watcher synchronization, and immutable returned stream headers

Safety

  • no active model requests are introduced by the probe path
  • failed, incomplete, or ambiguous usage responses never unlock credentials
  • returned stream headers are frozen before the API returns
  • the Seoul-specific three-hour compatibility patch is not included as an upstream behavior

Verification

  • Windows: focused affected packages x20, full go test -count=1 ./..., and server build
  • NAS Linux Go 1.26.1 with vendored dependencies and GOMAXPROCS=2:
    • full go vet ./...
    • five critical adaptive cooldown race tests x100 each
    • full go test -race -count=1 -p 2 ./...
    • linux/amd64 CGO build
  • NAS source archive SHA-256: d0e328e020d1cbe6b4b508919293fc4f75ed2e9158d45b12d52e8d239af1ba28
  • NAS binary SHA-256: f70e47ea2e11158d2033ee4893a1aa4494dd3e5022a3503e93c5a7d909938cea

@github-actions
github-actions Bot changed the base branch from main to dev July 17, 2026 14:47
@github-actions

Copy link
Copy Markdown

This pull request targeted main.

The base branch has been automatically changed to dev.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces adaptive Codex cooldowns and a non-consuming quota-status probe loop that queries the WHAM usage endpoint to check for limit resets. It also refactors several tests to eliminate race conditions and arbitrary sleeps, fixes context cancellation ownership in stream callbacks, and updates FileBodySource.WriteTo to report written bytes. The review feedback highlights a potential infinite loop and log spam in the quota probe loop if ResetQuota fails, suggesting a backoff schedule for failed resets.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +357 to +365
if outcome.recovered && outcome.err == nil {
ids := m.codexQuotaAuthIDsForBucket(bucketKey)
for _, authID := range ids {
if _, _, errReset := m.ResetQuota(ctx, authID); errReset != nil {
log.Warnf("failed to reset recovered Codex quota state for %s: %v", authID, errReset)
}
}
return
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If ResetQuota fails, the quota state remains exceeded and NextProbeAt is not updated. In the next iteration of the probe loop, this auth will be collected as due again immediately, leading to an infinite loop of rapid probes and log spam. To prevent this, we should back off the probe schedule for any auths that failed to reset.

	if outcome.recovered && outcome.err == nil {
		ids := m.codexQuotaAuthIDsForBucket(bucketKey)
		var failedIDs []string
		for _, authID := range ids {
			if _, _, errReset := m.ResetQuota(ctx, authID); errReset != nil {
				log.Warnf("failed to reset recovered Codex quota state for %s: %v", authID, errReset)
				failedIDs = append(failedIDs, authID)
			}
		}
		if len(failedIDs) > 0 {
			m.mu.Lock()
			for _, authID := range failedIDs {
				if auth := m.auths[authID]; auth != nil {
					auth.Quota.NextProbeAt = now.Add(quotaProbeNearInterval)
					for _, state := range auth.ModelStates {
						if state != nil {
							state.Quota.NextProbeAt = now.Add(quotaProbeNearInterval)
						}
					}
				}
			}
			m.mu.Unlock()
		}
		return
	}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant