carry attached images through to the model
The composer already uploaded an image, split its data URL and put the bytes on the wire as `images`. Nothing on the server read them. The `chat` ClientMessage had no such field, and the prompt reached the sidecar as a bare string, so all the model ever saw was the client-generated `[Attached image: …]` placeholder — a label describing a picture it was never shown. The transport was never the obstacle: `query()` consumes an async iterable of user messages whose `content` is an Anthropic `MessageParam`, and only `pushTurn` hardcoding a string kept it to text. So `images` is threaded through the four hops that dropped it and turned into native image content blocks at the end, renaming `mediaType` to the API's `media_type` at that last step. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -45,6 +45,15 @@ export type GroupMeta = {
|
||||
|
||||
export type ThinkingLevel = 'off' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh';
|
||||
|
||||
/**
|
||||
* An image pasted or attached in the composer, already split out of its data URL by the client.
|
||||
*
|
||||
* `/upload` also writes the file to disk, but the on-disk copy is for the attachment list and history —
|
||||
* what the model actually reads is these bytes, carried through to a native image content block. Naming
|
||||
* is camelCase on the wire and renamed to the Anthropic `media_type` at the last step.
|
||||
*/
|
||||
export type PromptImage = { mediaType: string; data: string };
|
||||
|
||||
export type ClientMessage =
|
||||
| {
|
||||
type: 'chat';
|
||||
@@ -56,6 +65,7 @@ export type ClientMessage =
|
||||
cwdRoot?: string;
|
||||
groupSlug?: string;
|
||||
attachmentIds?: string[];
|
||||
images?: PromptImage[];
|
||||
thinking?: ThinkingLevel;
|
||||
context?: string;
|
||||
contextId?: string;
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { ServerWebSocket } from 'bun';
|
||||
import { randomUUID } from 'crypto';
|
||||
import type { ClientMessage, ServerMessage, Message, TurnMessage, UserSession } from './types';
|
||||
import type { ClientMessage, ServerMessage, Message, PromptImage, TurnMessage, UserSession } from './types';
|
||||
import { sessionManager } from './session-manager';
|
||||
import { sendClaudeCodeStreaming } from '@@/channels/send-claude-code';
|
||||
import { sendOpenCodeStreaming } from '@@/channels/send-opencode';
|
||||
@@ -248,6 +248,7 @@ async function handleChat(
|
||||
cwdRoot?: string;
|
||||
groupSlug?: string;
|
||||
attachmentIds?: string[];
|
||||
images?: PromptImage[];
|
||||
thinking?: string;
|
||||
context?: string;
|
||||
contextId?: string;
|
||||
@@ -286,6 +287,7 @@ async function handleClaudeCodeChat(
|
||||
contextId?: string;
|
||||
cwd?: string;
|
||||
cwdRoot?: string;
|
||||
images?: PromptImage[];
|
||||
resumeSessionId?: string;
|
||||
},
|
||||
effectivePrompt: string,
|
||||
@@ -338,6 +340,7 @@ async function handleClaudeCodeChat(
|
||||
email,
|
||||
username,
|
||||
prompt: effectivePrompt,
|
||||
images: msg.images,
|
||||
sessionKey: sessionId,
|
||||
cwd,
|
||||
model,
|
||||
@@ -353,6 +356,7 @@ async function handleClaudeCodeChat(
|
||||
email,
|
||||
username,
|
||||
prompt: effectivePrompt,
|
||||
images: msg.images,
|
||||
sessionKey: sessionId,
|
||||
cwd,
|
||||
model,
|
||||
|
||||
Reference in New Issue
Block a user