recorder misses all but the first step in agent tool-loop causing infinite loops

[jantimon]

I am using LangGraph OpenAI Agents SDK and running in endless loops with aimock:

sequenceDiagram
    participant C as Client
    participant R as Recorder
    participant U as Upstream
    C->>R: step 0 - user msg
    R->>U: miss
    U-->>R: response A (tool_calls)
    R-->>C: A (saved as fixture)
    C->>R: step 1 - user msg + assistant + tool result
    R-->>C: A (HIT - same userMessage)
    C->>R: step 2 ...
    R-->>C: A (HIT again)
    Note over C,R: infinite loop - upstream never hit again

Loading

root cause

buildFixtureMatch (src/recorder.ts:1057) only writes { userMessage inputText model endpoint }

your matcher in src/router.ts already supports turnIndex and hasToolResult (tests in src/__tests__/turn-index.test.ts)

unfortunately they just never get written by the recorder so step 1+ always matches the step 0 fixture

suggested fix

in buildFixtureMatch always write both:

const messages = request.messages ?? []
match.turnIndex = messages.filter(({role}) => role === "assistant").length
if (messages.some(({role}) => role === "tool")) {
  match.hasToolResult = true
}

it is important to record turnIndex: 0 explicitly because skipping it when the count is 0 leaves the fixture unconstrained and it'll match every later step

validated locally on 1.19.1

I patched aimock locally and deleted the old fixtures and re-ran against real upstream and it resolved the problem for me

possible test

I asked claude which unit taset change might be best for aimock to test it and it proposed to change src/__tests__/snapshot-recording.test.ts

it("records distinct fixtures per turn when userMessage repeats (agent tool-loop)", async () => {                                                                                                                         
  // Upstream returns different responses based on turnIndex — same pattern as                                                                                                                                            
  // the matcher tests in turn-index.test.ts, just exercised through the                                                                                                                                                  
  // record-and-replay path.                                                                                                                                                                                              
  const { recorderUrl, fixturePath } = await setupUpstreamAndRecorder([                                                                                                                                                   
    {                                                                                                                                                                                                                     
      match: { userMessage: "plan a trip", turnIndex: 0 },           
      response: {                                                                                                                                                                                                         
        toolCalls: [{ id: "c1", name: "book_flight", arguments: "{}" }],
      },                                                                                                                                                                                                                  
    },                                                               
    {                                                                                                                                                                                                                     
      match: { userMessage: "plan a trip", turnIndex: 1, hasToolResult: true },
      response: { content: "Booked!" },                                                                                                                                                                                   
    },
  ]);                                                                                                                                                                                                                     
                                                                      
  const testId = "agent loop";                                                                                                                                                                                            
  
  // Step 0                                                                                                                                                                                                               
  await post(                                                        
    `${recorderUrl}/v1/chat/completions`,    
    {
      model: "gpt-4",
      messages: [{ role: "user", content: "plan a trip" }],                                                                                                                                                               
    },
    { "x-test-id": testId },                                                                                                                                                                                              
  );                                                                 
                                              
  // Step 1 — same user message, conversation grew by one assistant + one tool                                                                                                                                            
  await post(
    `${recorderUrl}/v1/chat/completions`,                                                                                                                                                                                 
    {                                                                                                                                                                                                                     
      model: "gpt-4",                        
      messages: [                                                                                                                                                                                                         
        { role: "user", content: "plan a trip" },                    
        {                                    
          role: "assistant",                                                                                                                                                                                              
          content: null,
          tool_calls: [                                                                                                                                                                                                   
            { id: "c1", type: "function", function: { name: "book_flight", arguments: "{}" } },
          ],                                                                                                                                                                                                              
        },
        { role: "tool", tool_call_id: "c1", content: '{"ok":true}' },                                                                                                                                                     
      ],                                                             
    },                                                                                                                                                                                                                    
    { "x-test-id": testId },
  );                                                                                                                                                                                                                      
                                                                      
  const content = JSON.parse(                
    fs.readFileSync(path.join(fixturePath, "agent-loop", "openai.json"), "utf-8"),
  ) as FixtureFile;                                                                                                                                                                                                       
  
  // Today this fails: file.fixtures.length === 1 (step-0 response served                                                                                                                                                 
  // for step 1 too, recorder never reached). After the fix: 2.      
  expect(content.fixtures).toHaveLength(2);                                                                                                                                                                               
  expect(content.fixtures[0].match.turnIndex).toBe(0);               
  expect(content.fixtures[1].match.turnIndex).toBe(1);                                                                                                                                                                    
  expect(content.fixtures[1].match.hasToolResult).toBe(true);        
}); 

[jpr5]

Fixed in v1.19.3 (PR #166) — buildFixtureMatch now writes turnIndex and hasToolResult so multi-turn agent tool loops produce distinct fixtures per turn.

[jantimon]

🚀 amazing fast
thank you