fix(chat): give AskUserQuestion a real answer channel

The harness answered every question itself, instantly, with "The user did not
answer the questions." — before the card had painted, let alone been read. The
agent then reasoned about a refusal that never happened: on 2026-08-19 it asked
the same question three times in a row while the answers sat on the screen.

The web UI already worked around it by rendering the card ungated and sending
the click back as an ordinary chat message. That got an answer through, but the
agent read it as a new instruction arriving from nowhere rather than as the
reply to what it asked, and nothing could distinguish a live question from one
answered last week — every replayed transcript rendered its old questions as
clickable.

A PreToolUse hook is the fix, and it has to be a hook rather than canUseTool:
under bypassPermissions — which is every session here — the SDK shadows the
callback and never calls it. Measured against 0.2.59: a bypassing session's
AskUserQuestion reached the hook and never reached canUseTool. Hooks run
regardless of permission mode.

The hook parks on a promise keyed by the harness's own tool_use_id, and returns
`deny` with the human's answer as the reason — which the harness feeds back as
the tool result verbatim. "Deny" reads wrong and is right: what it denies is the
tool's ability to answer for the user, and what replaces its output is what the
user actually said. Held open for 150s in testing with the turn completing
normally, so a person has time to think.

Keying on tool_use_id rather than an invented id means no new durable event: the
questions are already in the session log as an ordinary tool:start, and the
answer arrives as its tool:result. One source of truth, replay included — which
is what re-enables gating the card on `output`, and with it the staleness fix.
The two tests that pinned the old ungated behaviour asserted the workaround as
the contract; they now pin the reversal, including a pre-fix transcript reading
as settled, which it is.

Unanswered questions settle like plans do: 20 minutes, every session-ending path
draining them first, and wording that tells the agent the question is still on
the user's screen and must not be asked again.

Mobile (Butler) has the same workaround and is not fixed here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-19 16:46:38 +01:00
co-authored by Claude Opus 5
parent 520fad2288
commit 44a3d5f655
12 changed files with 260 additions and 25 deletions
@@ -81,6 +81,7 @@ export function useEmbeddableChat(params: UseEmbeddableChatParams, onMessageComp
setThinkingLevel,
setPlanMode,
decidePlan,
answerQuestion,
sendPrompt,
stopGeneration: stopTurn,
loadOlder,
@@ -589,6 +590,7 @@ export function useEmbeddableChat(params: UseEmbeddableChatParams, onMessageComp
setThinkingLevel,
setPlanMode,
decidePlan,
answerQuestion,
sendPrompt,
stopGeneration,
input,
@@ -122,7 +122,12 @@ const ReadAloudButton = ({ id, text }: { id: string; text: string }) => {
type MessageBubbleProps = {
message: ChatMessage;
onAnswer?: (text: string) => void;
/**
* Answer a question the agent is parked on. Takes the tool call id as well as the text because that id
* is how the sidecar finds the promise to resolve, and this component is the last place that still knows
* it — the caller sees a `ChatMessage` union whose user variant has no tool call at all.
*/
onAnswer?: (text: string, toolCallId: string) => void;
onRetry?: (prompt: string) => void;
/** Answer a plan the agent is parked on. Absent in read-only views, where the card renders without buttons. */
onDecidePlan?: (requestId: string, approve: boolean, feedback?: string) => void;
@@ -231,7 +236,7 @@ export const MessageBubble = ({ message, onAnswer, onRetry, onDecidePlan, defaul
// was arriving correctly and the UI simply did not recognise it. Nothing errored, which is why it
// read as "the model never asks" rather than as a bug.
if (QUESTION_TOOLS.has(message.toolName) && onAnswer) {
return <QuestionActivity message={message} onAnswer={onAnswer} />;
return <QuestionActivity message={message} onAnswer={(text) => onAnswer(text, message.toolCallId)} />;
}
return <ToolActivity message={message} defaultOpen={defaultOpen} />;
@@ -25,6 +25,7 @@ export const MessageList = ({ manager }: MessageListProps) => {
jumpToBottom,
sendPrompt,
decidePlan,
answerQuestion,
scrollViewportRef,
bottomRef,
isLoadingOlder,
@@ -130,7 +131,7 @@ export const MessageList = ({ manager }: MessageListProps) => {
{row.kind === 'message' && (
<MessageBubble
message={row.message}
onAnswer={(text) => sendPrompt(text)}
onAnswer={(text, toolCallId) => answerQuestion(toolCallId, text)}
onRetry={(prompt) => sendPrompt(prompt)}
onDecidePlan={decidePlan}
// Deliberately not `isGenerating`: a turn's work stays open after it finishes, and
@@ -106,29 +106,45 @@ describe('the escape hatch', () => {
});
});
// The regression that made this component dead on arrival, pinned from both sides.
// What an `output` means, which reversed on 2026-08-19 and is the reason this block reads oddly.
//
// AskUserQuestion has no answer channel: it resolves ITSELF, immediately, with the string below — so a
// real question always carries an output by the time it renders. The component gated on that output, and
// the test here asserted the resulting greyed-out state was correct. Both were wrong in the same
// direction, so the suite passed and every question in the product was unclickable.
describe('a question the SDK has already self-resolved', () => {
test('is still answerable — the output ends the tool call, not the conversation', () => {
// BEFORE: AskUserQuestion had no answer channel and resolved ITSELF, immediately, with "The user did not
// answer the questions." — so every real question carried an output before it was even readable. The
// component gated on that output, this test asserted the resulting greyed-out card was correct, and the
// suite stayed green over a feature nobody could click.
//
// AFTER: the sidecar parks the tool in a `PreToolUse` hook and feeds the human's answer back as its result.
// An output now means the question is settled — answered, timed out, or ended with its session — so the
// card is correctly read-only, and the pre-fix transcripts carrying the old string read as settled too,
// which they are: nobody can answer them now.
describe('a question that already carries an output', () => {
test('is read-only — the output is the answer, or the reason there will not be one', () => {
const answers: string[] = [];
const resolved = {
const answered = {
...ask([{ question: 'Which?', header: 'Store', options: OPTIONS }]),
output: 'Postgres',
} as never;
render(<QuestionActivity message={answered} onAnswer={(t) => answers.push(t)} />);
fireEvent.click(screen.getByText('Postgres'));
expect(answers).toEqual([]);
});
test('including a transcript from before the fix, whose output is the harness answering for the user', () => {
const answers: string[] = [];
const stale = {
...ask([{ question: 'Which?', header: 'Store', options: OPTIONS }]),
output: 'The user did not answer the questions.',
} as never;
render(<QuestionActivity message={resolved} onAnswer={(t) => answers.push(t)} />);
render(<QuestionActivity message={stale} onAnswer={(t) => answers.push(t)} />);
fireEvent.click(screen.getByText('Postgres'));
expect(answers).toEqual(['Postgres']);
expect(answers).toEqual([]);
});
test('but answering twice is refused — the latch is what we sent, not what the tool returned', () => {
test('while a LIVE question — no output yet — is answerable exactly once', () => {
const answers: string[] = [];
render(
<QuestionActivity
message={{ ...ask([{ question: 'Which?', header: 'Store', options: OPTIONS }]), output: 'x' } as never}
message={ask([{ question: 'Which?', header: 'Store', options: OPTIONS }])}
onAnswer={(t) => answers.push(t)}
/>,
);
@@ -67,16 +67,24 @@ export const QuestionActivity = ({ message, onAnswer }: QuestionActivityProps) =
const questions = input.questions;
if (!questions || questions.length === 0) return null;
// Deliberately NOT gated on `message.output`.
//
// AskUserQuestion resolves itself the instant it is called — the SDK has no answer channel, so it
// returns "The user did not answer the questions." before these buttons have painted. Gating on the
// output therefore disabled every option on every question, always: the tool was rendered, greyed out,
// and unanswerable. A test asserted that as correct, so the suite stayed green over a dead feature.
//
// The output says the TOOL CALL is over, which is true and irrelevant. The answer travels back as an
// ordinary chat message, and a message can always be sent. `sent` is the only real latch.
const isDisabled = sent;
/*
Gated on `message.output` again, because as of 2026-08-19 the output finally means what it says.
The history matters, since the obvious reading of this line is that it is the old bug coming back.
AskUserQuestion used to resolve ITSELF the instant it was called — no answer channel existed, so the
harness answered on your behalf with "The user did not answer the questions." before these buttons had
painted. Gating on that disabled every question, always, and a test asserted the dead feature as
correct. So the gate was removed and answers were sent as ordinary chat messages.
The sidecar now parks the tool in a `PreToolUse` hook until a person answers, and feeds the answer back
as the tool result. So an output is now proof of exactly one thing — this question has been dealt with:
answered, timed out, or ended with its session. All three should be read-only.
It also fixes staleness for free. A replayed transcript renders every past question with its answer
already in `output`, so old cards can no longer be clicked as if they were live — which is what a
refresh used to show, and what mobile hit too.
*/
const isDisabled = sent || message.output !== undefined;
/**
* Record one answer, and send once EVERY question has one.
@@ -777,6 +777,19 @@ export function useChat(initialSessionId?: string, initialModel?: string | null,
send({ type: 'plan:decide', requestId, approve, ...(feedback ? { feedback } : {}) });
}
/**
* Answer an `AskUserQuestion` the agent is blocked on.
*
* Note what this is NOT: sending the answer as a chat message. That was the old shape, forced by there
* being no answer channel at all — the harness had already resolved the tool with "The user did not
* answer the questions." before the card painted. The answer now goes back as the tool's own result, so
* the agent reads it as the reply to what it asked instead of as a new instruction arriving out of
* nowhere.
*/
function answerQuestion(toolCallId: string, answer: string) {
send({ type: 'question:answer', toolCallId, answer });
}
function stopGeneration() {
send({ type: 'stop' });
}
@@ -804,6 +817,7 @@ export function useChat(initialSessionId?: string, initialModel?: string | null,
setThinkingLevel,
setPlanMode,
decidePlan,
answerQuestion,
sendPrompt,
stopGeneration,
disconnectSession,