Fable refusals if interrupted / anthropic_thinking_signature not provided

[rxgrant]

The model declined this request (reasoning_extraction): This request was blocked as it seems to violate Anthropic's Terms of Service restrictions on reverse engineering or duplicating model outputs.

Model is anthropic/claude-fable-5.
Reasoning style is "full".
Interrupt strategy is "stop".
User action: Interrupted mid-reasoning.
Consequence: all further prompts fail.

Workaround: never append a turn after interruption, instead undo last turn using jp query --replay --edit and pull a "Minority Report" by appending preemptive correction onto the last turn, so that you can let it complete and get a proper signature.

[JeanMertz]

Oh wow, interesting.

What JP does if you interrupt mid-reasoning: it takes the reasoning tokens, and puts them back as a regular <think>...<</think>, precisely because the reasoning was interrupted mid-stream, and so we don't have any thought signature yet, so the original reasoning tokens can't be used as actual reasoning data.

I made this change a while ago. Before this change, when interrupting mid-reasoning, we'd throw away the reasoning tokens entirely and start the response fresh from the start of the last non-reasoning response. But this was bad, because if your interruption reacted to a part of the reasoning text (very likely, since you interrupted mid-reasoning), then the assistant got your interruption message, but NOT its original reasoning tokens, so it often had no idea what you were talking about, and it would - often - be confused with a worse outcome.

appending the new thoughts onto the last turn, so that you can let it complete and get a proper signature.

I don't think I quite follow your proposal here. That is what we are doing right now, and while I haven't tested this myself, it seems you/the error is saying our current technique of appending the reasoning tokens as fake <think>...</think> reasoning is no longer valid for Fable? Or are you saying we should append the reasoning tokens inside the user's chat request turn, marking them as "these were your thoughts before I cut you off" or something?

[JeanMertz]

I do see now that your action was interrupt-and-stop, whereas the usecase I mentioned above is interrupt-and-reply, which has a bigger downside to throwing away the reasoning tokens up to that point.

I am going to experiment a bit and try out different examples and outcomes to see what the right solution is here.

[rxgrant]

Or are you saying we should append the reasoning tokens inside the user's chat request turn [...]

No, I'm saying that in order to not trigger the refusal, I rewind time and pretend that when submitting that turn I already knew what I needed to say (now that in fact I do) in order for the bot to not steer off in a direction I didn't want. So I edit the last request as if the interrupted response never happened, so that I don't need to interrupt it. Instead of responding to the thought tokens, I preempt them.

I don't think I quite follow your proposal here.

It's definitely not a proposal for the best we can do, just a currently-available workaround using existing commands; and it requires the user to adapt the request using their "precog" knowledge, so it can't be generalized.

[JeanMertz]

I see, yes now your post makes sense. I have an RFD that I'll push up tomorrow, it'll leverage the existing stream-patching infrastructure:

jp/crates/jp_llm/src/event.rs

Lines 52 to 66 in b9f599a

	/// Instruct the caller to patch the conversation stream based on the
	/// patching rules.
	///
	/// This allows LLM providers to inform the caller about an invalid event
	/// stream that the provider can fix, but which should ideally also be
	/// applied to the conversation stream itself so that the error doesn't show
	/// up again in the future.
	///
	/// Currently, the caller applies these patches by mutating the stream
	/// in-place (see `apply_history_patches` in `signals.rs`).
	/// This is a known deviation from the append-only stream principle (RFD
	/// 064).
	/// When RFD 064's overlay/projection infrastructure lands, patches should
	/// be stored as stream events and applied at projection time instead.
	Patch(Vec<EventPatch>),

Here's the summary of the proposal:

This RFD records that a reasoning event is partial — the stream ended before
the block completed — as a first-class fact, and adds refusal-driven
recovery to the Anthropic provider: when a request is refused with the
reasoning_extraction category, the provider patches the conversation's
unsigned reasoning events to a quote-on-replay representation and retries
once.
It fixes [#829], where interrupting Claude Fable 5 mid-reasoning permanently
bricks the conversation with reasoning_extraction refusals.

[rxgrant]

Another instance of this:

> Run local fs_modify_file tool? [y,Y,n,N,p,r,e,?] r
Reason for skipping (optional) please go read patterns on tests.

...results in a bunch of work, and then:

The model declined this request (reasoning_extraction): This request was blocked as it seems to violate Anthropic's Terms of Service restrictions on reverse engineering or duplicating model outputs. [...]

[rxgrant]

If this proves difficult, one option might be to map (first) interrupt to "stop after next thinking signature arrives". That way, users could at least be spared future misguided tool calls, without dropping context that may have been used for prior tool calls in the turn. I'll try a few interruptions and conversation edits to see if there are any gotchas at that boundary.

[JeanMertz]

The solution I have (mostly) working right now stops as expected, and then when a new query starts it fixed the issue retroactively when the error occurs by moving the relevant thinking block as a pre-reponse block in front of the new user message. Once it compiles I can test how well this works in terms of the model understanding it and the reasoning extraction classifier not tripping up on it (which I don't expect).

Should have a test today, yesterday as an unexpected short day for me.