officerdev/platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

officer blocks its event loop re-parsing a live 196 MB transcript

Browse Source 
Chasing "Minesweeper is laggy" ended somewhere else entirely. Press-to-paint is
29ms median at x11vnc's socket and 29ms through the sidecar relay, but 35-200ms+
in the browser — so every bit of the variance is added by the one hop between,
which is officer.

Officer answering a trivial 401 on /api/health: median 1ms, p90 2ms, max 490ms.
summarizeTranscript full-reads and JSON.parses an entire transcript to build a
session-list row; timed against the largest here (196 MB, 70,137 entries) that
is 323ms to read plus 541ms to parse — 864ms of blocked loop per call. It is
cached on mtime, and a LIVE session appends every few seconds, so the cache
misses exactly when the agent is busy.

Filed rather than fixed: it is a hot path shared by every surface, and the right
shape (incremental parse keyed on size, versus moving summaries off the request
path) is an architecture call. The file already contains the argument for the
incremental version — "Claude only ever appends".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Officer-Session: f7a62308-0977-4cf4-aa29-c020853fd420
This commit is contained in:
pastilhas
2026-08-29 13:10:20 +01:00
co-authored by Claude Opus 5
parent 882cb67f7a
commit 73a9b0bf51
2 changed files with 86 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TODO.md
+39 -39
View File
@@ -22,45 +22,45 @@ the owner's OS user and can never be granted. Indirection there really is accide
## Themes
**107 open, 33 closed**, across 35 themes.
**108 open, 33 closed**, across 35 themes.
| Theme | Open | Closed |
| ------------------------------------------------------------------------------------------ | ---- | ------ |
| [Affordances](todo/affordances/) | 2 | 0 |
| [Agents coordinating agents](todo/agent-coordination/) | 5 | 0 |
| [Speaking into a session from the backend](todo/agent-delivery/) | 1 | 1 |
| [Backburner](todo/backburner/) | 3 | 0 |
| [Background jobs](todo/background-jobs/) | 2 | 1 |
| [Calendar and contacts](todo/calendar/) | 1 | 0 |
| [Chat interface](todo/chat/) | 1 | 0 |
| [Code editor](todo/code-editor/) | 1 | 0 |
| [Components — the shared UI layer](todo/components/) | 1 | 0 |
| [Documentation conventions — AGENTS.md vs CLAUDE.md](todo/docs-conventions/) | 2 | 0 |
| [File watchers](todo/file-watchers/) | 1 | 0 |
| [Fleet — one command across every server](todo/fleet/) | 6 | 2 |
| [Headscale](todo/headscale/) | 0 | 5 |
| [Infra (alpha)](todo/infra/) | 7 | 0 |
| [Integrations — external tools worth wrapping as a plugin](todo/integrations/) | 1 | 0 |
| [Known bugs](todo/known-bugs/) | 6 | 4 |
| [Memory — one store instead of per-harness files](todo/memory/) | 1 | 0 |
| [Mobile apps](todo/mobile-apps/) | 5 | 0 |
| [Multi-user](todo/multi-user/) | 17 | 5 |
| [Notifications](todo/notifications/) | 1 | 0 |
| [Plugin ecosystem — before it opens to third parties (2026-08-18)](todo/plugin-ecosystem/) | 4 | 1 |
| [Plugin infrastructure](todo/plugin-infrastructure/) | 2 | 1 |
| [Plugin repos — what writing their READMEs turned up](todo/plugin-repos/) | 10 | 9 |
| [Plugin system — gaps the email extraction found (2026-08-18)](todo/plugin-system/) | 3 | 0 |
| [Prompt library](todo/prompt-library/) | 1 | 0 |
| [SendOff](todo/sendoff/) | 1 | 0 |
| [Settings — portable, and readable by an agent](todo/settings/) | 2 | 0 |
| [Shell — spotlight and the dock](todo/shell/) | 4 | 1 |
| [Sub-agents — agents spawning agents](todo/sub-agents/) | 0 | 2 |
| [Sudo terminal modal](todo/sudo-terminal-modal/) | 1 | 0 |
| [Tooling — the list about the list](todo/tooling/) | 0 | 1 |
| [Transmission](todo/transmission/) | 6 | 0 |
| [URL state — routing, query strings and selection](todo/url-state/) | 4 | 0 |
| [Waiting on the owner to run it](todo/waiting-on-owner/) | 3 | 0 |
| [Workspaces & Panels](todo/workspaces-panels/) | 2 | 0 |
| Theme | Open | Closed |
| --- | --- | --- |
| [Affordances](todo/affordances/) | 2 | 0 |
| [Agents coordinating agents](todo/agent-coordination/) | 5 | 0 |
| [Speaking into a session from the backend](todo/agent-delivery/) | 1 | 1 |
| [Backburner](todo/backburner/) | 3 | 0 |
| [Background jobs](todo/background-jobs/) | 2 | 1 |
| [Calendar and contacts](todo/calendar/) | 1 | 0 |
| [Chat interface](todo/chat/) | 1 | 0 |
| [Code editor](todo/code-editor/) | 1 | 0 |
| [Components — the shared UI layer](todo/components/) | 1 | 0 |
| [Documentation conventions — AGENTS.md vs CLAUDE.md](todo/docs-conventions/) | 2 | 0 |
| [File watchers](todo/file-watchers/) | 1 | 0 |
| [Fleet — one command across every server](todo/fleet/) | 6 | 2 |
| [Headscale](todo/headscale/) | 0 | 5 |
| [Infra (alpha)](todo/infra/) | 7 | 0 |
| [Integrations — external tools worth wrapping as a plugin](todo/integrations/) | 1 | 0 |
| [Known bugs](todo/known-bugs/) | 7 | 4 |
| [Memory — one store instead of per-harness files](todo/memory/) | 1 | 0 |
| [Mobile apps](todo/mobile-apps/) | 5 | 0 |
| [Multi-user](todo/multi-user/) | 17 | 5 |
| [Notifications](todo/notifications/) | 1 | 0 |
| [Plugin ecosystem — before it opens to third parties (2026-08-18)](todo/plugin-ecosystem/) | 4 | 1 |
| [Plugin infrastructure](todo/plugin-infrastructure/) | 2 | 1 |
| [Plugin repos — what writing their READMEs turned up](todo/plugin-repos/) | 10 | 9 |
| [Plugin system — gaps the email extraction found (2026-08-18)](todo/plugin-system/) | 3 | 0 |
| [Prompt library](todo/prompt-library/) | 1 | 0 |
| [SendOff](todo/sendoff/) | 1 | 0 |
| [Settings — portable, and readable by an agent](todo/settings/) | 2 | 0 |
| [Shell — spotlight and the dock](todo/shell/) | 4 | 1 |
| [Sub-agents — agents spawning agents](todo/sub-agents/) | 0 | 2 |
| [Sudo terminal modal](todo/sudo-terminal-modal/) | 1 | 0 |
| [Tooling — the list about the list](todo/tooling/) | 0 | 1 |
| [Transmission](todo/transmission/) | 6 | 0 |
| [URL state — routing, query strings and selection](todo/url-state/) | 4 | 0 |
| [Waiting on the owner to run it](todo/waiting-on-owner/) | 3 | 0 |
| [Workspaces & Panels](todo/workspaces-panels/) | 2 | 0 |
## Records — no items, kept for the reasoning
@@ -71,6 +71,7 @@ the owner's OS user and can never be granted. Indirection there really is accide
## Newest ten
- `2026-08-29` **Officer blocks its own event loop for ~0.9s re-parsing a live transcript, and everything it serves stalls with it** — [known-bugs/officer-blocks-its-event-loop-parsing-transcripts](todo/known-bugs/officer-blocks-its-event-loop-parsing-transcripts.md)
- `2026-08-27` **Turn an ordinary chat into a commander or a bot, carrying its context along** — [agent-coordination/turn-a-chat-into-a-commander-or-a-bot](todo/agent-coordination/turn-a-chat-into-a-commander-or-a-bot.md)
- `2026-08-27` **The calendar API cannot express repeating events, reminders, invitees or a cancelled occurrence** — [calendar/the-json-api-cannot-express-recurrence-alarms](todo/calendar/the-json-api-cannot-express-recurrence-alarms.md)
- `2026-08-27` **AGENTS.md sweep — what is left after the static docs landed** — [docs-conventions/agents-md-remaining](todo/docs-conventions/agents-md-remaining.md)
@@ -80,4 +81,3 @@ the owner's OS user and can never be granted. Indirection there really is accide
- `2026-08-27` **Marketplace: browse by tag — filter list, card grid, detail with a back button** — [plugin-ecosystem/marketplace-browse-by-tag](todo/plugin-ecosystem/marketplace-browse-by-tag.md)
- `2026-08-25` **An affordance: a scoped agent warns when a request lands outside its scope** — [affordances/an-affordance-a-scoped-agent-warns-on-the](todo/affordances/an-affordance-a-scoped-agent-warns-on-the.md)
- `2026-08-25` **Commit signing is inconsistent across harnesses — Claude names itself, opencode names nothing** — [agent-coordination/commit-signing-is-inconsistent-across-harnesses](todo/agent-coordination/commit-signing-is-inconsistent-across-harnesses.md)
- `2026-08-25` **A commander's watcher does not survive a reboot, and nothing says so** — [agent-coordination/a-commander-s-watcher-does-not-survive-a-reboot](todo/agent-coordination/a-commander-s-watcher-does-not-survive-a-reboot.md)
todo/known-bugs/officer-blocks-its-event-loop-parsing-transcripts.md
+47
View File
@@ -0,0 +1,47 @@
---
id: known-bugs/officer-blocks-its-event-loop-parsing-transcripts
title: 'Officer blocks its own event loop for ~0.9s re-parsing a live transcript, and everything it serves stalls with it'
theme: known-bugs
status: open
created: 2026-08-29
date_source: stated
order: 7
---
**Found 2026-08-29 while chasing "Minesweeper is laggy today".** It is not a Minesweeper bug. The game's
display is relayed through officer, and officer periodically stops answering anything for up to a second.
**The measurements, in order.** Press-to-paint at x11vnc's own socket: **median 29ms, max 35ms**, stable
even with a 1.5s idle gap before each click. Through the sidecar's own `/ws` relay: **median 29ms, max
43ms** — so the sidecar adds nothing. In the browser, where it is actually felt: **35ms to 200ms+, varying
per click**. All of the variance is introduced between the sidecar and the browser, and the only thing
there is officer.
Officer answering a trivial unauthenticated 401 on `/api/health`, 30 samples: **median 1ms, p90 2ms, max
490ms.** The loop is fine until it is blocked outright.
**What blocks it.** `summarizeTranscript` (`api/chat/claude-sessions.ts`) full-reads and `JSON.parse`s a
whole transcript to build a session-list row. Timed against the largest on this machine — 196 MB, 70,137
entries: `readFileSync` **323ms**, split + parse every line **541ms**, **864ms blocking, per call**.
It is cached, and the cache cannot help. The key is mtime, and the comment above it says why that was
believed to be exact: _"Claude only ever appends"_. But a **live** session appends every few seconds, so
its mtime moves, the entry misses, and the whole 196 MB is re-read and re-parsed — for a row in a list.
The busier the agent, the more often. `468 MB across 174 transcripts` here, and the biggest one belongs to
a commander that has been running all day.
**Why it reads as a Minesweeper regression.** Nothing in the plugin changed (last commit two days ago), and
the same cost was invisible when that transcript was 10 MB and this took ~45ms. It scales with transcript
size, so it arrives gradually and then all at once.
**Scope is much wider than the game.** Every HTTP request, every chat socket frame and every relayed plugin
websocket shares that loop. The game is just the one surface where a 500ms stall is impossible to miss.
**The fix the file argues for itself.** _"Claude only ever appends"_ is exactly what makes incremental
summarisation correct: keep the running summary keyed on `(path, size)` and consume only the bytes appended
since last time, instead of re-reading from zero. Turns ~864ms into the cost of the new tail. `listClaudePwds`
already does a bounded head read for `cwd` (`claude-sessions.ts`, "a bounded head read"), so the precedent
for not reading the whole file is in the same module.
Not attempted here: it is a hot path shared by every surface, and the right shape — incremental parse versus
moving summaries out of the request path entirely — is an architecture call rather than a patch.