stop losing the answer when immich rejects mid-upload
An upload over a few MB came back as a 400 with an empty body, no message anywhere, and nothing logged by Immich, the sidecar or officer. It was a race, not a size limit. Immich judges an asset from its first few KB and rejects immediately, then closes; both our hops were still writing the body; Node treats the leftover bytes as a protocol violation and replaces the application's answer with a bodyless `400 Bad Request` + `Connection: close`. The real message never reached the wire. Measured before the change: streamed lost the message 1/4 at 8 MB and 4/4 at 32 MB — probability rising with size, which is why small photos usually worked and a phone's video never did. Both hops needed it. Fixing only the sidecar took 32 MB from 4/4 failing to 2/4, because the platform proxy was losing it one hop up. Bounded at 512 MB, above which the body streams exactly as before. That ceiling is not a refusal and is deliberately not a 413: a file Immich ACCEPTS is read to the end and never races, so a 4 GB video is unaffected. All that is given up above the cap is the error message on a file that was going to be rejected anyway. A first attempt refused over-cap uploads outright and would have broken the working 4 GB case to improve diagnosis of the doomed one. `bufferRequestBody` is opt-in and off by default: the vault and wallet proxies must keep streaming so a passphrase or macaroon never lands in the platform's heap. Also adds the proxy error logging that made this findable at all — status and two byte counts from headers, never the bodies. `responseBytes: "unknown"` is what exposed the stripped response. Verified live at 32/256 MB (buffered) and 640 MB (streamed, passes through). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -73,6 +73,10 @@ beforeAll(async () => {
|
||||
sidecar = Bun.serve({
|
||||
port: 0,
|
||||
async fetch(req) {
|
||||
// A rejection with no body at all — the shape from COMMS/UPLOAD_EMPTY_400.md that tells the client
|
||||
// "no" and gives it nothing to act on.
|
||||
if (new URL(req.url).pathname.endsWith('/empty400')) return new Response(null, { status: 400 });
|
||||
|
||||
const contentType = req.headers.get('content-type') ?? '';
|
||||
const raw = new Uint8Array(await req.arrayBuffer());
|
||||
const asText = new TextDecoder().decode(raw);
|
||||
@@ -160,6 +164,61 @@ describe('sidecar proxy multipart forwarding', () => {
|
||||
expect(seen!.fileBytes).toEqual([...FILE_BYTES]);
|
||||
});
|
||||
|
||||
it('reports a rejection instead of passing it on in silence', async () => {
|
||||
// COMMS/UPLOAD_EMPTY_400.md: uploads failed with an empty-bodied 400 and nothing logged it anywhere,
|
||||
// so localising the fault took an evening of curl bisection. The platform is the only hop that sees
|
||||
// both sides. `requestBytes` is the field that would have cracked it — a size threshold is legible at
|
||||
// a glance instead of by bisection.
|
||||
const warnings: unknown[][] = [];
|
||||
const realWarn = console.warn;
|
||||
console.warn = (...args: unknown[]) => void warnings.push(args);
|
||||
|
||||
try {
|
||||
const payload = JSON.stringify({ some: 'payload' });
|
||||
const res = await app.request('http://x/api/testcar/_officer/empty400', {
|
||||
method: 'POST',
|
||||
// Set explicitly: `app.request` does not derive it, and this is the field the bug report needed.
|
||||
headers: { 'content-type': 'application/json', 'content-length': String(payload.length) },
|
||||
body: payload,
|
||||
});
|
||||
|
||||
// Pass-through is unchanged: the proxy reports, it does not rewrite.
|
||||
expect(res.status).toBe(400);
|
||||
expect(await res.text()).toBe('');
|
||||
|
||||
expect(warnings.length).toBe(1);
|
||||
const [message, detail] = warnings[0] as [string, Record<string, unknown>];
|
||||
expect(message).toContain('[testcar]');
|
||||
expect(message).toContain('POST /_officer/empty400');
|
||||
expect(message).toContain('400');
|
||||
// The empty body is named, not left to be inferred from a byte count of zero.
|
||||
expect(detail.emptyBody).toBe(true);
|
||||
expect(detail.requestBytes).toBe(String(payload.length));
|
||||
expect(detail.responseBytes).toBe('0');
|
||||
} finally {
|
||||
console.warn = realWarn;
|
||||
}
|
||||
});
|
||||
|
||||
it('says nothing about a request that succeeded', async () => {
|
||||
const { body, contentType } = phoneMultipart();
|
||||
const warnings: unknown[][] = [];
|
||||
const realWarn = console.warn;
|
||||
console.warn = (...args: unknown[]) => void warnings.push(args);
|
||||
|
||||
try {
|
||||
const res = await app.request('http://x/api/testcar/_officer/assets', {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': contentType },
|
||||
body: new Blob([body]),
|
||||
});
|
||||
expect(res.status).toBe(201);
|
||||
expect(warnings).toEqual([]);
|
||||
} finally {
|
||||
console.warn = realWarn;
|
||||
}
|
||||
});
|
||||
|
||||
it('still forwards a JSON body untouched', async () => {
|
||||
const res = await app.request('http://x/api/testcar/_officer/echo', {
|
||||
method: 'POST',
|
||||
|
||||
Reference in New Issue
Block a user