ci(models): payload-aware model selection for GitHub Models

[google-labs-jules]

This PR introduces payload-aware model selection for GitHub Models. By estimating the input token count (using a simple length / 4 heuristic) and checking it against the max_input_tokens limit of available models, the system now avoids selecting models with insufficient context windows for large diffs or DOM changes.

Key changes:

• scripts/lib/modelPicker.ts: Added token limit filtering logic.
• scripts/clients/githubModelsCodeReviewClient.ts: Calculates estimate from diff context.
• scripts/clients/githubModelsVisualReviewClient.ts: Calculates estimate from DOM diff file.

Fixes #2584

---

Merged with similiar prs

Closes #2591
Closes #2592

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[github-actions]

🚀 Deployment Details (Last updated: Jun 19, 2026, 12:37 PM PST)

🚀 Pushed to gh-pages; publish in progress

• Preview URL: https://arii.github.io/tech-dancer/ci-models-token-aware-3998645635553103582/
• Preview Dashboard: https://arii.github.io/tech-dancer/previews/index.html
• Commit SHA: 35580fc
• Marker File: verify here

[github-actions]

🐙 GitHub Models Code Review

Powered by GitHub Models

Reviewing: PR #2587

Code Review Feedback

HIGH SEVERITY REVIEW

---

1. Blocking Bug: Incorrect pickOptimalModel Call Arity

Evidence:

• In githubModelsCodeReviewClient.ts:

  const modelName = await pickOptimalModel(apiKey, fallback, false, estimatedInputTokens);

• In githubModelsVisualReviewClient.ts:

  const modelName = await pickOptimalModel(apiKey, fallback, true, estimatedInputTokens);

• In modelPicker.ts, the new signature is:

  export async function pickOptimalModel(
    token: string,
    fallback: string = 'gpt-4o-mini',
    needsVision: boolean = false,
    estimatedInputTokens: number = 0
  ): Promise<string> {

BUT: The call sites in both clients are correct and match the new signature.
NO ARITY BUG FOUND.

---

2. Blocking Bug: Type Error in GitHubModel.limits.max_input_tokens

Evidence:

• In modelPicker.ts:

  const rawMaxIn = m.limits?.max_input_tokens;
  if (rawMaxIn === undefined || rawMaxIn === null) return false;
  
  const maxIn = typeof rawMaxIn === 'number' ? rawMaxIn : Number(rawMaxIn);
  if (isNaN(maxIn)) return false;

• GitHubModel.limits.max_input_tokens is defined as number | undefined.

Analysis:

• If max_input_tokens is a number, typeof rawMaxIn === 'number' is true.
• If it's a string, Number(rawMaxIn) is used.
• If it's undefined/null, filtered out.

NO TYPE ERROR FOUND.

---

3. Blocking Bug: Incorrect Token Estimation Logic

Evidence:

• In githubModelsCodeReviewClient.ts:

  const estimatedInputTokens = Math.ceil(summary.diffContext.length / 4);

• In githubModelsVisualReviewClient.ts:

  domDiffLength = Math.min(stats.size, 3000);
  const estimatedInputTokens = Math.ceil(domDiffLength / 4);

• In both cases, the value passed is a number.

NO TYPE ERROR FOUND.

---

4. Blocking Bug: Incorrect Use of fs.statSync for Token Estimation

Evidence:

• In githubModelsVisualReviewClient.ts:

  const stats = fs.statSync(domDiffPath);
  domDiffLength = Math.min(stats.size, 3000);

• stats.size is the file size in bytes, not characters.

Analysis:

• If the file is UTF-8 encoded and contains multi-byte characters, stats.size may not equal the number of characters.
• The token estimation is based on character count, not byte count.
• This can lead to severe underestimation or overestimation of input tokens, especially for non-ASCII content.

This is a blocking bug.
The correct approach is to read the file contents and use .length on the string, not stats.size.

Example fix:

const domDiffContent = fs.readFileSync(domDiffPath, 'utf8');
domDiffLength = Math.min(domDiffContent.length, 3000);

---

5. Blocking Bug: Missing Type for estimatedInputTokens Parameter

Evidence:

• In createModel functions:

  async function createModel(estimatedInputTokens: number = 0): Promise<ChatOpenAI> {

• In pickOptimalModel:

  estimatedInputTokens: number = 0

Analysis:

• Types are present and correct.

NO TYPE ERROR FOUND.

---

6. Blocking Bug: Untrusted Input Path

Evidence:

• No new untrusted input path is introduced. All inputs are from environment variables or local files.

NO SECURITY ISSUE FOUND.

---

Summary of Blocking Issues

Blocking Issue:

• In githubModelsVisualReviewClient.ts, the token estimation uses fs.statSync(...).size (bytes), but the heuristic expects character count. This can cause incorrect model selection, leading to runtime errors or model invocation failures for large diffs.

---

Actionable Feedback

Fix the token estimation in githubModelsVisualReviewClient.ts:

• Replace:

  const stats = fs.statSync(domDiffPath);
  domDiffLength = Math.min(stats.size, 3000);

• With:

  const domDiffContent = fs.readFileSync(domDiffPath, 'utf8');
  domDiffLength = Math.min(domDiffContent.length, 3000

---

Generated by github-models-code-review

[arii]

Hi! This PR has merge conflicts with the base branch.

To fix this, please follow these steps:

1. Ensure your local main branch is up to date.
2. Switch back to your feature branch.
3. Merge main into your branch.
4. Resolve the conflicts in the files marked by Git, then stage the resolved files.
5. Complete the merge commit.
6. Push the resolved branch to origin.

Alternatively, you can rebase your branch onto main and resolve conflicts step-by-step. Let us know if you need any help!

[arii]

ANTI-AI-SLOP\n\n\n## FINDINGS\n

\n\n## FINAL RECOMMENDATION\n<Approved | Approved with Minor Changes | Not Approved>\n\n

Inline Comments (Fallback due to Github line resolution errors)

• :1:

[arii]

ANTI-AI-SLOP\n\n\n## FINDINGS\n

\n\n## FINAL RECOMMENDATION\n<Approved | Approved with Minor Changes | Not Approved>\n\n

Inline Comments (Fallback due to Github line resolution errors)

• :1:

[arii]

Automated Review for PR #2587

CI Status: All checks passing.

FINAL RECOMMENDATION

Approved

[arii]

Automated Review for PR #2587

CI Status: All checks passing.

Recommendation: Everything looks good from a CI perspective. Ready for manual review/merge if no other concerns.

FINAL RECOMMENDATION

Approved

[arii]

Comprehensive Review for PR #2587

CI Status: All checks passing.

Recommendation: Everything looks good from a CI perspective. All tests and linters pass. Ready for manual review/merge if no other concerns.

FINAL RECOMMENDATION

Approved