Skip to content

runner/coop: coerce message timestamps to float before sorting#49

Merged
akhatua2 merged 1 commit into
mainfrom
fix/coop-timestamp-sort-typeerror
May 14, 2026
Merged

runner/coop: coerce message timestamps to float before sorting#49
akhatua2 merged 1 commit into
mainfrom
fix/coop-timestamp-sort-typeerror

Conversation

@ProKil
Copy link
Copy Markdown
Member

@ProKil ProKil commented May 14, 2026

What

execute_coop crashes mid-rollout with
TypeError: '<' not supported between instances of 'int' and 'str'
when one agent adapter reports numeric timestamps and another reports ISO strings:

sent_msgs.sort(key=lambda x: x.get("timestamp") or 0)
# src/cooperbench/runner/coop.py:156
  • mini_swe_agent emits time.time() floats.
  • The OpenHands SDK adapter can emit ISO strings.
  • Some events have no timestamp at all.

The sort fires before agent{fid}_traj.json is written, so callers that rely on the structured trajectory output get nothing — even though the underlying rollout ran to completion. I hit this from CooperTrain's plan-first eval (cooperbench/CooperTrain#30): both held-out rollouts ran the full 100 steps and produced readable_*.json but no agent1_traj.json/agent2_traj.json, so the eval saw "missing assistant turns" everywhere.

Fix

Extract the timestamp key as a module-level _message_timestamp_key helper that best-effort coerces to float (None / non-numeric → 0.0), then have execute_coop's sort call it. Behavior preserved when all timestamps are numeric (and matches the prior or 0 intent for missing ones); previously-crashing mixed-type runs now sort cleanly with non-numeric entries grouped at the front.

Tests

New tests/runner/test_coop.py (4 tests):

  • _extract_conversation happy-path (sent + received).
  • Mixed int / float / ISO-string / None / missing timestamps all sort without raising.
  • Received messages stay filtered out before the sort.

All 31 runner tests pass (pytest tests/runner/).

Risk

Local — touches one sort key and adds a pure helper. The semantics of the sort are unchanged for any input that previously succeeded (numeric-only timestamps); only previously-crashing inputs change behavior, and they now sort instead of raising.

🤖 Generated with Claude Code

execute_coop crashes mid-rollout with
``TypeError: '<' not supported between instances of 'int' and 'str'``
when one agent adapter reports numeric timestamps (mini_swe_agent uses
time.time() floats) and another reports ISO strings (OpenHands SDK).
The sort fires before agent{fid}_traj.json is written, so callers that
rely on the structured trajectory output get nothing — even though the
underlying rollout ran to completion.

Extract the timestamp key as a module-level
``_message_timestamp_key`` helper that best-effort coerces to float
(None / non-numeric → 0.0), then have execute_coop's sort call it.

Adds tests/runner/test_coop.py with regression coverage:
  - mixed int / float / ISO-string / None / missing timestamps all sort
    without raising
  - received messages stay filtered out before the sort

All 31 runner tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ProKil ProKil requested a review from akhatua2 May 14, 2026 01:22
Copy link
Copy Markdown
Collaborator

@akhatua2 akhatua2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@akhatua2 akhatua2 merged commit 01f4273 into main May 14, 2026
3 checks passed
@akhatua2 akhatua2 deleted the fix/coop-timestamp-sort-typeerror branch May 14, 2026 03:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants