Skip to content

th-1cc9fa: model-output ceiling clamp + raise defaults (Python server → main)#197

Merged
brentrager merged 1 commit into
mainfrom
th-1cc9fa-py-ceiling-main
Jul 8, 2026
Merged

th-1cc9fa: model-output ceiling clamp + raise defaults (Python server → main)#197
brentrager merged 1 commit into
mainfrom
th-1cc9fa-py-ceiling-main

Conversation

@brentrager

Copy link
Copy Markdown
Contributor

Problem

The Python WebSocket server (python/server/) shipped with chat-widget turn sizing (max_tokens=512, max_iterations=6) that starves reasoning models — they exhaust the budget on reasoning and return empty replies / truncated tool loops. It also had no per-model output ceiling, so it couldn't safely raise max_tokens. This is the Python-server parity for the Rust/Go server hardening already on main (#193).

The change lived only on the stale localflavor-hardening-based branch th-1cc9fa-py-ceiling; this brings it to main additively.

Solution

Cherry-picked the clamp commit (f556862) and reconciled with main's evolved turn_runner (kept main's extension-host wiring; added the raised defaults + per-turn ceiling lookup):

  • Raise anti-starvation sizing: DEFAULT_MAX_TOKENS 512→8192, DEFAULT_MAX_ITERATIONS 6→20 (mirrors the Rust/Go server constants).
  • New model_info.py: best-effort per-model output ceiling from the gateway's /model/info (map_model_info + model_output_ceiling), cached once per process, stdlib fetch, no extra deps. No key / any error / unknown model → None → unclamped.
  • Thread the ceiling into AgentOptions.model_max_output so the engine clamps max_tokens down to what the model can physically emit. Feature-detected against the pinned core.
  • Bump the core pin to published PyPI smooai-smooth-operator-core>=1.3.2 and drop the git-rev source — 1.3.2 ships both the clamp field (model_max_output) and the SEP extension host the git rev was pinned for, so the clamp is now LIVE (_CORE_SUPPORTS_CEILING → True), not dormant.

Verification

  • uv sync resolves 1.3.2 from PyPI cleanly.
  • uv run pytest138 passed, 0 skipped (the forward-compat clamp test now runs against the clamp-capable 1.3.2).
  • uv run ruff check . + ruff format --check . → clean.
  • Confirmed live: _CORE_SUPPORTS_CEILING=True, DEFAULT_MAX_TOKENS=8192, DEFAULT_MAX_ITERATIONS=20.

🤖 Generated with Claude Code

…thon server)

Additive onto main: cherry-picked the Python-server clamp from the stale
localflavor-hardening branch (f556862) and reconciled with main's evolved
turn_runner (kept main's extension-host wiring, added the raised defaults +
per-turn ceiling lookup).

- Raise anti-starvation turn sizing: max_tokens 512 -> 8192 (DEFAULT_MAX_TOKENS),
  max_iterations 6 -> 20 (DEFAULT_MAX_ITERATIONS). The old chat-widget sizing
  starves reasoning models (empty replies / truncated tool loops).
- New model_info.py: best-effort per-model output ceiling from the gateway's
  /model/info (map_model_info + model_output_ceiling), cached once per process,
  stdlib fetch, no extra deps. No key / any error / unknown model -> None -> unclamped.
- Thread the resolved ceiling into AgentOptions.model_max_output so the engine
  clamps max_tokens to what the model can emit. Feature-detected against the core.
- Bump the core pin to the published PyPI smooai-smooth-operator-core>=1.3.2 and
  drop the git-rev source (1.3.2 ships BOTH the clamp field and the SEP extension
  host the git rev was pinned for), so the clamp is LIVE (_CORE_SUPPORTS_CEILING
  now True), not dormant.

Tests: 138 passed, 0 skipped (the forward-compat clamp test now runs against the
clamp-capable 1.3.2). ruff check + format clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Jul 8, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: a9d3b90

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@brentrager
brentrager merged commit b693a78 into main Jul 8, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant