[Engine] 1 <tool_call> block(s) in the answer did not read as a call of a declared tool #4071

Activity

1. github-actions

   added

   needs-formatTitle or template does not follow CONTRIBUTING.md yetTitle or template does not follow CONTRIBUTING.md yet

   on Oct 25, 2026
2. github-actions commented on Oct 25, 2026

   

   github-actionsBot

   on Oct 25, 2026 – with GitHub Actions

   More actions

   Thanks for the issue. To keep the tracker readable, every item needs an area prefix in the title and the parts of the issue form filled in. This one is missing:

   • Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other], then a short description. Example: "[Engine] decode stops after 2 minutes".

   Edit the issue (title and description) and this note goes away on its own. If it is still missing parts in 5 days it will be closed, which is not a rejection: fix it and comment /reopen.

   Details: CONTRIBUTING.md
3. galmok

   changed the title [-]1 <tool_call> block(s) in the answer did not read as a call of a declared tool[/-] [+][Engine] 1 <tool_call> block(s) in the answer did not read as a call of a declared tool[/+] on Oct 25, 2026
4. github-actions

   removed

   needs-formatTitle or template does not follow CONTRIBUTING.md yetTitle or template does not follow CONTRIBUTING.md yet

   on Oct 25, 2026
5. github-actions commented on Oct 25, 2026

   

   github-actionsBot

   on Oct 25, 2026 – with GitHub Actions

   More actions

   Thanks, this follows the format now.
6. ackalker commented on Oct 25, 2026

   

   ackalker

   on Oct 25, 2026

   More actions

   Same failure from the Pi + llama-swap side, with a client-side record of one turn - plus one practical obstacle to ever reading your own warning line.

   The witnessed case. 2026-10-25, model qwen3.8-flash-next-iq3_s (IQ3 quant) behind llama-swap v266, Pi as the agent harness. One turn (10.9 s, finish_reason: stop, 1042 output tokens, 0 reasoning tokens) came back as an ordinary answer: its text block ends with a complete, well-formed call block - call opener, function name AskUserQuestion, one questions parameter, and all three closers in order - and the response carried no tool_calls at all. AskUserQuestion was not in the request's tools list; it is a tool name from a different harness's list, which the model wrote out of its training traces. Pi stored the markup as assistant text and the loop ended. That is the "tool calls ending up in regular assistant output, breaking the agent loop" report, seen from the client's side. Nothing was cut mid-block and no reasoning was involved, so for this turn it is the undeclared-name path, not a truncated block and not a thinking leak.

   Why nobody can confirm it after the fact. That diagnostic (the sentence in this issue's title) is printed by the serve process to stdout. Two consequences behind a proxy:

   • it never reaches the engine's own log file - strata-*.log only holds the engine's token/cache stats, so grepping the logs finds nothing;
   • llama-swap keeps upstream stdout in a small in-memory ring. The copy I could grab was 102,401 bytes / 1163 lines = 131 request completions, about 1.5 h of agent use, with nothing older. By the time markup is noticed in the terminal, the line that named the undeclared tool has rolled away - which is exactly why the OP here says the problem cannot be triggered at will.

   What would make this diagnosable, smallest first:

   1. Also emit that warning to stderr / the engine's log file, not only stdout. Users running the server under a proxy have no way to see stdout at all.
   2. Make the fact visible in the response, so the client can react without server-side access - a counter in usage or an extra field (something like "1 undeclared call block: AskUserQuestion"), or a response header. [Engine] Agent recovery when undeclared tool calls are returned as text #3796 is about agent recovery; recovery starts with the client knowing the call was dropped rather than never attempted.
   3. If the "a complete block for a name the request did not declare stays text" behaviour is deliberate (it reads that way after 70f4240 / [Engine] Tool-call parser silently returns a malformed <function=...> block as plain assistant content #3441, [Engine] a tool call the request never offered still becomes tool_calls on the content path (frontend.py:1285, 0.2.1) #3075), then a doc note naming the trigger would help: a complete block for a foreign name is normal for models trained or previously served under another harness's tool names, and this is the shape users hit.

   For others running llama-swap: v266's request captures (Activity view, or GET /api/captures/{id}; both bodies are base64) retain the response body the client actually received, which is enough to tell this case apart from a truncated-block case afterwards. The budget is captureBuffer in MB (default 10; that held about 60 requests with 20k-40k token prompts). And STRATA_DEBUG=1 in the model profile's env: list makes serve dump the raw decoded model text and the token ids per request - that raw line is the evidence neither I nor the OP had - at the cost of one very long line per request.
7. Malspherus commented on Oct 26, 2026

   

   Malspherus

   on Oct 26, 2026

   More actions

   I also ran into this issue, both with OpenClaw and with OpenCode (both via LiteLLM). In the OpenCode session it was happening for the first time, and I reverted the message and retried multiple times with the same result. So it's not only poisoned sessions, there is something problematic happening in Strata already.

   Some more information from my agent:
   Another data point — OpenClaw harness, and the "always on some starts / never on others"
   pattern quantified.

   Environment. Strata 0.1.48 (CUDA, Windows), upgraded from 0.1.40.3 the same week the
   failures started. Models: Qwen3.8-Flash-Next IQ3_S quant + Qwen3.8-27B GSQ, served behind
   a LiteLLM proxy (OpenAI-compatible Chat Completions). Harness: OpenClaw.

   What we saw (Oct 24-25).

   1. Exactly this issue's warning surfaced: "2 <tool_call> block(s) ... the names the model
      wrote: bash, bash". Our harness declares exec, not bash — same undeclared-name path
      as [Engine] Agent recovery when undeclared tool calls are returned as text #3796's bash vs shell case.
   2. Separately: 7 assistant replies in one session contained complete call-shaped XML for
      tools that WERE declared in the request (subagents, sessions_spawn), as plain text.
      If declared-only validation is the only cause, that second shape shouldn't happen for
      declared names — worth checking whether those blocks were malformed (truncated or
      mis-nested) rather than complete.
   3. Bimodality, measured: one session produced 7 malformed replies in ~15 minutes; other
      sessions on the same engine, same model, same period produced zero. Once one
      call-shaped block lands in the transcript, the model imitates it — that session
      degrades fast and stays degraded. This is what "almost always on some starts, almost
      never on others" looks like from the transcript side: it's not start luck, it's
      self-reinforcing context.
   4. Intermittent, cannot trigger at will — matches the OP.

   What worked client-side. Instruct the model to route every tool not in the visible
   list through a declared wrapper tool (tool_search -> tool_call with id/args).
   Undeclared direct blocks stopped. Poisoned sessions still needed reset/compaction — the
   instruction alone did not outweigh the bad examples already in context.

   Endorses #3796's ask. A machine-readable signal (usage counter or response field,
   per @ackalker) is what lets a harness distinguish "model chose not to call" from "parser
   rejected a complete call", attempt bounded recovery, and quarantine the poisoned turn
   before the feedback loop starts.

   Question. Did tool-call parsing / declared-only validation change between 0.1.40.3
   and 0.1.48? Our timeline points squarely at that update: same models, same harness,
   worked before, fails after.
8. OutRunner commented on Oct 26, 2026

   

   OutRunner

   on Oct 26, 2026

   More actions

   I've been seeing this all day today too. I keep having to prompt it to try again.
9. icalder commented on Oct 28, 2026

   

   icalder

   on Oct 28, 2026

   Last edited by icalder

   More actions

   I'm seeing this too. NVFP4. Only started happening with 0.2.7.
10. zhangjg8116 commented on Oct 29, 2026

    

    zhangjg8116

    on Oct 29, 2026

    More actions

    [strata] 1 <tool_call> block(s) in the answer did not read as a call of a declared tool and went back as text (a name the request did not declare, or a form the template does not use); the names the model wrote: shell=command; the response carries them as strata_rejected_tool_calls, /metrics counts them
11. pluciorx commented on Oct 29, 2026

    

    pluciorx

    on Oct 29, 2026

    Last edited by pluciorx

    More actions

    <tool_call> <function=grep_search=isRegexp <parameter=query> list-options|core.|bmm.|output_folder|planning_artifacts|implementation_artifacts|Available|--set <parameter=filePath> content.txt <parameter=isRegexp> true <parameter=maxResults> 60 </tool_call>
    same issue on VS Code with 0.2.7 Strata
12. Niko1221 commented on Oct 29, 2026

    

    Niko1221

    on Oct 29, 2026

    Owner

    More actions

    Thank you for the log lines. Several reports here show it. Please send the raw model output of one failing turn and the tool names the client declares (OpenCode's tools list), and the Strata and engine version. The name in one report (shell=command) may be a malformed opener rather than an undeclared tool; the raw text shows which.
13. james-martinez commented on Oct 29, 2026

    

    james-martinez

    on Oct 29, 2026

    Last edited by james-martinez

    More actions

    Fenced variant of this issue, reproduced locally end-to-end, with a fix in #4613.

    Environment. Strata 0.2.7 (both the rocm and sycl builds), Qwen3.8-Flash-Next IQ3_S, served behind an OpenAI-compatible gateway, consumed through Open WebUI chat.

    The witnessed case. Chat asks the model to "make an SVG of a red circle and save it". The request declares no tools (a plain Open WebUI chat - like the toolless-request shape other reporters hit after a wrapper dropped the list). The reply: finish_reason: stop, tool_calls absent, and content ends with a complete call block for write_file - opener, function name, both parameters, all closers in order - inside a ```xml code fence. Open WebUI renders it as the original report describes: the tool XML sits inside a code block in the answer.

    What the code says. Two rules keep this text and both are right to: the declared-only rule (#3075) and the fence rule (#1058 - inside a fence an opener is a quoted example). The problem is the diagnosis channel from #3796: the fence branch of OutputParser returns the block as content without touching unread. At main (46c152a), feeding the exact shape through the parser gives unread=0, unread_names=[] - so the warning line in this issue's title never prints, strata_rejected_tool_calls is absent, /metrics does not move. A fenced block is invisible to every recovery signal a client has. That also sharpens the OP's "cannot trigger at will": whether the model fences the block (invisible) or writes it bare (the warning prints) looks like start luck from the client side.

    What #4071 is, as I read it, after tracing the parser: one parser contract with three entry paths - undeclared name outside code (counted, named), malformed opener (counted, unnamed), and complete block inside a fence (until #4613: silent). The bimodality @Malspherus measured is a fourth, trans-script effect: once one block lands as assistant text, the model imitates it every turn.

    Fix (#4613). Text and finish_reason stay exactly as they are - the rule the maintainer stated is kept. The fence/inline-code path now counts and names the block like the others, with a reason on each strata_rejected_tool_calls entry: undeclared, in_code (a declared name quoted in a fence), or malformed. Partial names are held until decidable so streamed and whole replies count identically (checked at chunk sizes 1/3/17). Regression test for the fenced shape included; it fails on stock main and passes with the commit, and the parser-suite failure lists are identical before/after.

    One ask beyond the PR, for the poisoned-transcript loop that keeps sessions in this state: a client can only quarantine the bad turn if it can see it - the in_code reason is the minimum signal an OWUI function or harness needs to say "that was a dropped call, not the answer".

    👍React with 👍1Reacted by OutRunner