diff --git a/TODO.md b/TODO.md index 229e1b3d..6909ac36 100644 --- a/TODO.md +++ b/TODO.md @@ -22,45 +22,45 @@ the owner's OS user and can never be granted. Indirection there really is accide ## Themes -**107 open, 33 closed**, across 35 themes. +**108 open, 33 closed**, across 35 themes. -| Theme | Open | Closed | -| ------------------------------------------------------------------------------------------ | ---- | ------ | -| [Affordances](todo/affordances/) | 2 | 0 | -| [Agents coordinating agents](todo/agent-coordination/) | 5 | 0 | -| [Speaking into a session from the backend](todo/agent-delivery/) | 1 | 1 | -| [Backburner](todo/backburner/) | 3 | 0 | -| [Background jobs](todo/background-jobs/) | 2 | 1 | -| [Calendar and contacts](todo/calendar/) | 1 | 0 | -| [Chat interface](todo/chat/) | 1 | 0 | -| [Code editor](todo/code-editor/) | 1 | 0 | -| [Components — the shared UI layer](todo/components/) | 1 | 0 | -| [Documentation conventions — AGENTS.md vs CLAUDE.md](todo/docs-conventions/) | 2 | 0 | -| [File watchers](todo/file-watchers/) | 1 | 0 | -| [Fleet — one command across every server](todo/fleet/) | 6 | 2 | -| [Headscale](todo/headscale/) | 0 | 5 | -| [Infra (alpha)](todo/infra/) | 7 | 0 | -| [Integrations — external tools worth wrapping as a plugin](todo/integrations/) | 1 | 0 | -| [Known bugs](todo/known-bugs/) | 6 | 4 | -| [Memory — one store instead of per-harness files](todo/memory/) | 1 | 0 | -| [Mobile apps](todo/mobile-apps/) | 5 | 0 | -| [Multi-user](todo/multi-user/) | 17 | 5 | -| [Notifications](todo/notifications/) | 1 | 0 | -| [Plugin ecosystem — before it opens to third parties (2026-08-18)](todo/plugin-ecosystem/) | 4 | 1 | -| [Plugin infrastructure](todo/plugin-infrastructure/) | 2 | 1 | -| [Plugin repos — what writing their READMEs turned up](todo/plugin-repos/) | 10 | 9 | -| [Plugin system — gaps the email extraction found (2026-08-18)](todo/plugin-system/) | 3 | 0 | -| [Prompt library](todo/prompt-library/) | 1 | 0 | -| [SendOff](todo/sendoff/) | 1 | 0 | -| [Settings — portable, and readable by an agent](todo/settings/) | 2 | 0 | -| [Shell — spotlight and the dock](todo/shell/) | 4 | 1 | -| [Sub-agents — agents spawning agents](todo/sub-agents/) | 0 | 2 | -| [Sudo terminal modal](todo/sudo-terminal-modal/) | 1 | 0 | -| [Tooling — the list about the list](todo/tooling/) | 0 | 1 | -| [Transmission](todo/transmission/) | 6 | 0 | -| [URL state — routing, query strings and selection](todo/url-state/) | 4 | 0 | -| [Waiting on the owner to run it](todo/waiting-on-owner/) | 3 | 0 | -| [Workspaces & Panels](todo/workspaces-panels/) | 2 | 0 | +| Theme | Open | Closed | +| --- | --- | --- | +| [Affordances](todo/affordances/) | 2 | 0 | +| [Agents coordinating agents](todo/agent-coordination/) | 5 | 0 | +| [Speaking into a session from the backend](todo/agent-delivery/) | 1 | 1 | +| [Backburner](todo/backburner/) | 3 | 0 | +| [Background jobs](todo/background-jobs/) | 2 | 1 | +| [Calendar and contacts](todo/calendar/) | 1 | 0 | +| [Chat interface](todo/chat/) | 1 | 0 | +| [Code editor](todo/code-editor/) | 1 | 0 | +| [Components — the shared UI layer](todo/components/) | 1 | 0 | +| [Documentation conventions — AGENTS.md vs CLAUDE.md](todo/docs-conventions/) | 2 | 0 | +| [File watchers](todo/file-watchers/) | 1 | 0 | +| [Fleet — one command across every server](todo/fleet/) | 6 | 2 | +| [Headscale](todo/headscale/) | 0 | 5 | +| [Infra (alpha)](todo/infra/) | 7 | 0 | +| [Integrations — external tools worth wrapping as a plugin](todo/integrations/) | 1 | 0 | +| [Known bugs](todo/known-bugs/) | 7 | 4 | +| [Memory — one store instead of per-harness files](todo/memory/) | 1 | 0 | +| [Mobile apps](todo/mobile-apps/) | 5 | 0 | +| [Multi-user](todo/multi-user/) | 17 | 5 | +| [Notifications](todo/notifications/) | 1 | 0 | +| [Plugin ecosystem — before it opens to third parties (2026-08-18)](todo/plugin-ecosystem/) | 4 | 1 | +| [Plugin infrastructure](todo/plugin-infrastructure/) | 2 | 1 | +| [Plugin repos — what writing their READMEs turned up](todo/plugin-repos/) | 10 | 9 | +| [Plugin system — gaps the email extraction found (2026-08-18)](todo/plugin-system/) | 3 | 0 | +| [Prompt library](todo/prompt-library/) | 1 | 0 | +| [SendOff](todo/sendoff/) | 1 | 0 | +| [Settings — portable, and readable by an agent](todo/settings/) | 2 | 0 | +| [Shell — spotlight and the dock](todo/shell/) | 4 | 1 | +| [Sub-agents — agents spawning agents](todo/sub-agents/) | 0 | 2 | +| [Sudo terminal modal](todo/sudo-terminal-modal/) | 1 | 0 | +| [Tooling — the list about the list](todo/tooling/) | 0 | 1 | +| [Transmission](todo/transmission/) | 6 | 0 | +| [URL state — routing, query strings and selection](todo/url-state/) | 4 | 0 | +| [Waiting on the owner to run it](todo/waiting-on-owner/) | 3 | 0 | +| [Workspaces & Panels](todo/workspaces-panels/) | 2 | 0 | ## Records — no items, kept for the reasoning @@ -71,6 +71,7 @@ the owner's OS user and can never be granted. Indirection there really is accide ## Newest ten +- `2026-08-29` **Officer blocks its own event loop for ~0.9s re-parsing a live transcript, and everything it serves stalls with it** — [known-bugs/officer-blocks-its-event-loop-parsing-transcripts](todo/known-bugs/officer-blocks-its-event-loop-parsing-transcripts.md) - `2026-08-27` **Turn an ordinary chat into a commander or a bot, carrying its context along** — [agent-coordination/turn-a-chat-into-a-commander-or-a-bot](todo/agent-coordination/turn-a-chat-into-a-commander-or-a-bot.md) - `2026-08-27` **The calendar API cannot express repeating events, reminders, invitees or a cancelled occurrence** — [calendar/the-json-api-cannot-express-recurrence-alarms](todo/calendar/the-json-api-cannot-express-recurrence-alarms.md) - `2026-08-27` **AGENTS.md sweep — what is left after the static docs landed** — [docs-conventions/agents-md-remaining](todo/docs-conventions/agents-md-remaining.md) @@ -80,4 +81,3 @@ the owner's OS user and can never be granted. Indirection there really is accide - `2026-08-27` **Marketplace: browse by tag — filter list, card grid, detail with a back button** — [plugin-ecosystem/marketplace-browse-by-tag](todo/plugin-ecosystem/marketplace-browse-by-tag.md) - `2026-08-25` **An affordance: a scoped agent warns when a request lands outside its scope** — [affordances/an-affordance-a-scoped-agent-warns-on-the](todo/affordances/an-affordance-a-scoped-agent-warns-on-the.md) - `2026-08-25` **Commit signing is inconsistent across harnesses — Claude names itself, opencode names nothing** — [agent-coordination/commit-signing-is-inconsistent-across-harnesses](todo/agent-coordination/commit-signing-is-inconsistent-across-harnesses.md) -- `2026-08-25` **A commander's watcher does not survive a reboot, and nothing says so** — [agent-coordination/a-commander-s-watcher-does-not-survive-a-reboot](todo/agent-coordination/a-commander-s-watcher-does-not-survive-a-reboot.md) diff --git a/todo/known-bugs/officer-blocks-its-event-loop-parsing-transcripts.md b/todo/known-bugs/officer-blocks-its-event-loop-parsing-transcripts.md new file mode 100644 index 00000000..14d5d09d --- /dev/null +++ b/todo/known-bugs/officer-blocks-its-event-loop-parsing-transcripts.md @@ -0,0 +1,47 @@ +--- +id: known-bugs/officer-blocks-its-event-loop-parsing-transcripts +title: 'Officer blocks its own event loop for ~0.9s re-parsing a live transcript, and everything it serves stalls with it' +theme: known-bugs +status: open +created: 2026-08-29 +date_source: stated +order: 7 +--- + +**Found 2026-08-29 while chasing "Minesweeper is laggy today".** It is not a Minesweeper bug. The game's +display is relayed through officer, and officer periodically stops answering anything for up to a second. + +**The measurements, in order.** Press-to-paint at x11vnc's own socket: **median 29ms, max 35ms**, stable +even with a 1.5s idle gap before each click. Through the sidecar's own `/ws` relay: **median 29ms, max +43ms** — so the sidecar adds nothing. In the browser, where it is actually felt: **35ms to 200ms+, varying +per click**. All of the variance is introduced between the sidecar and the browser, and the only thing +there is officer. + +Officer answering a trivial unauthenticated 401 on `/api/health`, 30 samples: **median 1ms, p90 2ms, max +490ms.** The loop is fine until it is blocked outright. + +**What blocks it.** `summarizeTranscript` (`api/chat/claude-sessions.ts`) full-reads and `JSON.parse`s a +whole transcript to build a session-list row. Timed against the largest on this machine — 196 MB, 70,137 +entries: `readFileSync` **323ms**, split + parse every line **541ms**, **864ms blocking, per call**. + +It is cached, and the cache cannot help. The key is mtime, and the comment above it says why that was +believed to be exact: _"Claude only ever appends"_. But a **live** session appends every few seconds, so +its mtime moves, the entry misses, and the whole 196 MB is re-read and re-parsed — for a row in a list. +The busier the agent, the more often. `468 MB across 174 transcripts` here, and the biggest one belongs to +a commander that has been running all day. + +**Why it reads as a Minesweeper regression.** Nothing in the plugin changed (last commit two days ago), and +the same cost was invisible when that transcript was 10 MB and this took ~45ms. It scales with transcript +size, so it arrives gradually and then all at once. + +**Scope is much wider than the game.** Every HTTP request, every chat socket frame and every relayed plugin +websocket shares that loop. The game is just the one surface where a 500ms stall is impossible to miss. + +**The fix the file argues for itself.** _"Claude only ever appends"_ is exactly what makes incremental +summarisation correct: keep the running summary keyed on `(path, size)` and consume only the bytes appended +since last time, instead of re-reading from zero. Turns ~864ms into the cost of the new tail. `listClaudePwds` +already does a bounded head read for `cwd` (`claude-sessions.ts`, "a bounded head read"), so the precedent +for not reading the whole file is in the same module. + +Not attempted here: it is a hot path shared by every surface, and the right shape — incremental parse versus +moving summaries out of the request path entirely — is an architecture call rather than a patch.