pastilhasandClaude Opus 5
0f93262a8c
the owner confirmed it live the same day. what that leaves untested is the half the cache exists for — a second login, the tabs, a stored row surviving a switch — so the docs now say which half is proven instead of claiming neither is. also records the bug the first deploy found, since the doc had asserted the degradation that turned out to be broken. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
255 lines
17 KiB
Markdown
255 lines
17 KiB
Markdown
# The AI usage panel
|
|
|
|
**Built and running 2026-08-18. Verified for ONE account, never for two.** The owner confirmed the panel
|
|
live the same day: limits, countdowns, tokens by day and by model all render against the signed-in account.
|
|
|
|
What that leaves untested is the half the cache exists for — a second Anthropic login, the account tabs, and
|
|
a stored row surviving a switch. Group C of the test plan below is that, and it is the only part still worth
|
|
running. It will be run when it is needed rather than pre-emptively, since the failure it would catch costs
|
|
nothing but a wrong tab.
|
|
|
|
Click the harness pill in the chat composer — the label reading **Claude Code**, left of the background-job
|
|
button — and it opens what Claude Code's own usage display shows: limits with reset countdowns, tokens by
|
|
day, tokens by model, with one tab per Anthropic account this machine has seen.
|
|
|
|
## Before it works
|
|
|
|
```bash
|
|
bun db:push # creates ai_provider_usage
|
|
pm2 restart officer officer-claude-code
|
|
```
|
|
|
|
`db:push` first. Without the table the panel still opens and still shows the signed-in account — the stored
|
|
read catches and returns an empty list — but there are no tabs and nothing survives an account switch.
|
|
|
|
That degradation was not free: it was broken on the first deploy and fixed the same day. `snapshotClaudeUsage`
|
|
had the read and the write in ONE `try`, so a missing table threw on the upsert and discarded a perfectly good
|
|
reading — the panel said "Not signed in" over an account that was signed in fine. The write has its own `try`
|
|
now, and `usage-store-degradation.test.ts` holds the line.
|
|
|
|
Both processes are needed for the _other_ change that shipped alongside this (`model:resolved`, see
|
|
"Related" below). The usage panel itself only needs `officer`.
|
|
|
|
## Where the numbers come from, and why not from Officer
|
|
|
|
**Officer does not have these numbers and cannot derive them.** Per-session cost is accumulated in memory in
|
|
`websocket.ts` and dies with `pm2 restart officer`; nothing aggregates it across sessions, days or models.
|
|
More importantly the rate-limit windows are a property of the **Anthropic account**, spent by every `claude`
|
|
on the machine — this chat, a terminal, a task runner, a pipeline step. A figure derived from what the chat
|
|
socket happened to observe would be lower than the truth by however much was spent elsewhere, and silently so.
|
|
|
|
So everything is read from the CLI's own record:
|
|
|
|
| what | source |
|
|
| ------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------- |
|
|
| Session / Weekly / model-scoped windows | `GET https://api.anthropic.com/api/oauth/usage`, `Authorization: Bearer <accessToken>`, `anthropic-beta: oauth-2025-04-20` |
|
|
| The access token, and the plan label (`Max 20x`) | `~/.claude/.credentials.json` → `claudeAiOauth` |
|
|
| Which account is signed in | `~/.claude.json` → `oauthAccount.emailAddress` |
|
|
| Tokens by day, tokens by model | scanning `~/.claude/projects/**/*.jsonl` |
|
|
|
|
`CLAUDE_CONFIG_DIR` is honoured for the first, third and fourth.
|
|
|
|
The shape and the non-obvious correctness details are lifted from omarchy's `omarchy-agent-usage-claude`
|
|
collector (`basecamp/omarchy`, branch `quattro`) rather than rediscovered — specifically the dedup by message
|
|
id, the four token counters, the percent-versus-fraction scale detection, and reading model-scoped windows
|
|
out of `limits[]`. The account-email part comes from the owner's fork, `gitea.officer.dev/OMARCHI/pastilhas.agents`,
|
|
which added it for the same reason it matters here: percentages you cannot attribute to an account are
|
|
percentages you cannot act on.
|
|
|
|
### Two things the live payload settled
|
|
|
|
Probed 2026-08-18 against the real endpoint; the response is the fixture in `claude-usage.test.ts`.
|
|
|
|
1. **Every model-scoped bucket is null.** `seven_day_opus`, `seven_day_sonnet` and friends are all `null`,
|
|
while `limits[]` carries a real `weekly_scoped` entry for Fable at 6%. A reader that trusts the flat
|
|
buckets silently drops a window the account is actually spending against.
|
|
2. **The window is named by `kind`, never by the model name.** A display name like `Opus 5 (1M context)`
|
|
contains "1M", and parsing a window out of the title turns that into a one-minute window.
|
|
|
|
### Cost
|
|
|
|
Measured on this machine, 30 transcript files totalling ~150MB:
|
|
|
|
| | cold | note |
|
|
| --------------- | ---------------- | ----------------------------- |
|
|
| Transcript scan | ~440ms, one core | 43k lines, 18.6k `JSON.parse` |
|
|
| Limits probe | ~600ms | one HTTPS round trip |
|
|
| Account read | ~2ms | 50KB JSON |
|
|
|
|
They run concurrently, so a cold read is **~600ms**; warm is ~1ms. Both halves are cached server-side for
|
|
60s, independently, so a failed probe never throws away a good scan.
|
|
|
|
**The scan grows monotonically** — transcripts are never deleted. Today it is fine. It is the number that
|
|
eventually is not; see "Deferred".
|
|
|
|
## The cache, and why it exists
|
|
|
|
Not for speed. **Half a reading cannot be taken at all for an account you are not signed into**: the limits
|
|
need the credential the CLI holds for the _current_ login, and the token counts come from that login's
|
|
transcripts. Switch accounts and the previous one is not slow to read — it is unreadable.
|
|
|
|
The owner holds a personal account and a company one and switches between them. The question worth answering
|
|
at a glance is _which of my accounts has room right now_, and only a stored reading per account can answer it.
|
|
|
|
### The table
|
|
|
|
`ai_provider_usage` (`src/databases/officer_db/src/ai-provider-usage/`)
|
|
|
|
- Unique on `(user_id, ai_provider, email_account)` — so switching logins **accumulates** rather than
|
|
overwrites. Keyed on the Officer user as well, because one Officer account holds several Anthropic logins
|
|
and eventually several Officer accounts will each hold their own.
|
|
- `ai_provider` carries a CHECK constraint (`claude-code`, `opencode`), per the house rule — a provider the
|
|
code stops supporting must not sit there unnoticed.
|
|
- The reading itself is one `jsonb` document. The three sections are arrays whose shape is provider-specific
|
|
— a prepaid provider reports a credit balance and no windows at all — so columns would be mostly null for
|
|
everyone and a migration per provider. A snapshot is never edited, only replaced whole, which is what makes
|
|
a document safe here.
|
|
- `updated_at` is when the reading was **measured**, not when the row was touched. It is set explicitly on
|
|
the update half of the upsert, because a column default only applies to inserts.
|
|
|
|
### The write
|
|
|
|
`snapshotClaudeUsage` fires on the `result` message in `createMessageHandler` — the end of a turn, while
|
|
you are reading the answer. `void`, so nothing about the turn depends on it, and it swallows its own
|
|
failures: a missed snapshot costs one stale row until the next turn.
|
|
|
|
Capped by the reader's own 60s cache, so a burst of short turns is not a burst of 150MB scans.
|
|
|
|
**Owner-gated, and not as a permissions nicety.** `getOwnerHomeDir` ignores the email it is passed, so a
|
|
member's turn would write _the owner's_ usage under _the member's_ user id. That is not a leak, it is a
|
|
wrong row, and it would be invisible.
|
|
|
|
### The read
|
|
|
|
`GET /chat/usage` answers from Postgres alone — no scan, no network — so the panel paints instantly with
|
|
every account, **including the ones that cannot be measured**. `?refresh=1` re-measures the signed-in account,
|
|
upserts it, and returns the merged list. The client fires both and swaps the fresh tab in when it lands.
|
|
|
|
One exception: on a first-ever open, where the signed-in account has no stored row, the plain call measures
|
|
inline rather than returning an empty panel.
|
|
|
|
Owner-only in the handler rather than in the registry — `chat` is a `confined` permission, so a member
|
|
holding it reaches every route in this router. A 403, not an empty reading: members run `claude` as their own
|
|
Linux account with their own credential, so "what is my usage" for them is a different query against a
|
|
different login, and answering it with the owner's data would be the wrong answer confidently given.
|
|
|
|
## Staleness: what a cached row means later
|
|
|
|
This is the part most likely to be wrong in a way that matters, because it is the part that asserts things
|
|
about data it did not measure.
|
|
|
|
**Countdowns survive staleness exactly.** `resetsAt` is an absolute instant, not a duration, so a row measured
|
|
six hours ago has an as-of percentage and a _perfectly correct_ "resets in 14h 20m", ticked client-side with
|
|
no network and no signed-in session. That asymmetry is why a cached row is worth showing rather than
|
|
apologising for.
|
|
|
|
**Past its reset, the percentage is zero** (`currentLimit` in `useClaudeUsage.ts`). The stored figure is not
|
|
merely stale there — it is wrong in the dangerous direction: a company account snapshotted at 78% whose weekly
|
|
window rolled over overnight would still read 78% when in fact it is empty and available.
|
|
|
|
- **Weekly rolls forward.** The window length is in the `kind`, so the next boundary is arithmetic rather
|
|
than a guess: add seven days until it lands in the future. This is the case the feature is for.
|
|
- **Session does not.** The five-hour window starts on first use rather than on a grid, so once it lapses
|
|
there is no clock running. 0%, and no countdown line — not because it cannot be computed, but because
|
|
there is nothing to count down to.
|
|
|
|
A non-current tab carries a **"Not signed in · as of 20m ago"** line. The live tab does not: saying "just
|
|
now" under a number being measured as you look at it is noise.
|
|
|
|
`fetchedAt` is the **older** of the limits and scan halves, not a stamp applied on the way out. A reading
|
|
served from cache was measured when the cache was filled; stamping on exit would make every stored row claim
|
|
to be current, which defeats the age label entirely.
|
|
|
|
## Files
|
|
|
|
| file | what |
|
|
| --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
|
|
| `src/servers/api/chat/claude-usage.ts` | reads credentials, account, limits, transcripts. Owner-only _structurally_ — takes no identity at all |
|
|
| `src/servers/api/chat/usage-store.ts` | snapshot / list / merge across accounts |
|
|
| `src/servers/api/chat/chat.ts` | `GET /chat/usage[?refresh=1]` |
|
|
| `src/servers/api/chat/websocket.ts` | the turn-end snapshot |
|
|
| `src/databases/officer_db/src/ai-provider-usage/` | schema + queries |
|
|
| `src/workspaces/officerdev/src/apps/Chat/useClaudeUsage.ts` | the two-query hook and every formatter |
|
|
| `src/workspaces/officerdev/src/apps/Chat/components/UsagePopover.tsx` | the pill and the panel |
|
|
|
|
## What has unit tests, and what has not
|
|
|
|
**Tested** — `limitsFromPayload` against the live payload (7 cases), `friendlyModelName`, `currentLimit`'s
|
|
staleness rules, `mergeUsage`'s invariants, and the formatters.
|
|
|
|
**Not tested by anything** — every I/O path and the whole UI: the route, the owner refusal, the turn-end
|
|
write, the transcript scan, the tabs, the cache-first swap.
|
|
|
|
## Test plan
|
|
|
|
Do these in order; each depends on the last. `pastilhas` is the owner.
|
|
|
|
### A. It works at all
|
|
|
|
1. `bun db:push`, then `pm2 restart officer`. Confirm the server boots — `assertPermissionTotality` runs
|
|
before `serve()`, and a router without a registry entry refuses to boot. No registry line was added
|
|
(the route lives inside the existing `chat` router), so a boot failure here means something else.
|
|
2. `psql officer -c '\d ai_provider_usage'` — four columns plus the unique index and the CHECK.
|
|
3. Open a chat, click the **Claude Code** pill. Expect: plan and email in the header, three limit bars with
|
|
countdowns, seven days, models. Compare against the omarchy bar panel on the same machine — the
|
|
percentages should agree, the token totals may not (omarchy merges opencode and pi sessions; this does not).
|
|
|
|
### B. The snapshot
|
|
|
|
4. Send one prompt and let the turn finish. `psql officer -c 'select user_id, ai_provider, email_account, plan, updated_at from ai_provider_usage'` — one row, `updated_at` within the last minute.
|
|
5. Send a second prompt. `updated_at` moves. It should NOT move by more than the turn took plus 60s of cache.
|
|
6. `pm2 logs officer` during a turn — no error from `snapshotClaudeUsage`, and no added latency to the turn.
|
|
|
|
### C. Two accounts — the actual feature
|
|
|
|
7. `claude` → sign out → sign in as the _other_ account. Confirm `~/.claude.json` now names it.
|
|
8. Reopen the pill. Expect: **two tabs**, the new account selected, the old one showing
|
|
_"Not signed in · as of Nm ago"_ with its last-known percentages.
|
|
9. Click the old tab. Percentages are as-of; the **weekly countdown still ticks** and is correct.
|
|
10. Take a turn on the new account, reopen. The new account's row updates; **the old account's row is
|
|
untouched** — this is the one that would silently break, since a refresh must not overwrite an account
|
|
it could not measure.
|
|
|
|
### D. Staleness
|
|
|
|
11. Hardest to test honestly: it needs a row whose window has lapsed. Either wait, or
|
|
`update ai_provider_usage set payload = jsonb_set(payload, '{limits,0,resetsAt}', '"2026-08-01T01:00:00Z"')`
|
|
on a non-current row. Expect **0%** and a countdown rolled forward to a future date for a weekly window;
|
|
**0% and no countdown line** for a session window.
|
|
12. Leave the panel open for a minute — countdowns tick down (30s interval), and only while it is open.
|
|
|
|
### E. Refusals and degradation
|
|
|
|
13. As a **member** with the `chat` grant: the pill is a plain label, not a button. `curl` the route with
|
|
their token → 403.
|
|
14. Rename `~/.claude/.credentials.json` and reopen: limits section reports "not signed in", **token history
|
|
still renders**. Put it back.
|
|
15. Drop the table (`drop table ai_provider_usage`) and reopen: the panel still shows the signed-in account,
|
|
with no tabs. Then `bun db:push` again.
|
|
16. Open the pill on an **OpenCode** conversation: plain label, no button.
|
|
|
|
## Deferred
|
|
|
|
- **The scan is O(all history) and history only grows.** ~440ms today at 150MB. The files are append-only
|
|
JSONL, so an incremental scan keyed on `(path, size, mtime)` is the obvious fix — but dedup is by message
|
|
id and is _global_ across files, which is what makes it non-trivial. A measurement on 2026-08-18 found
|
|
**74.7% of message ids appear in more than one file** (19,696 usage entries → 5,869 unique), so per-file
|
|
aggregates cannot simply be summed. Not urgent; revisit when the cold read passes a second or two.
|
|
- **Members have no usage of their own.** They run their own `claude` with their own credential, and the
|
|
reader is structurally owner-only. The table is already keyed by `user_id`, so this is a reader change,
|
|
not a schema one.
|
|
- **`opencode` is in the provider CHECK but nothing writes it.** It has no usage endpoint and no readable
|
|
transcripts. The constraint names it so the day it does, this is a writer and not a migration.
|
|
- **No way to forget an account.** `forgetProviderUsage` exists and nothing calls it; a tab for a login you
|
|
no longer use is clutter with no delete affordance.
|
|
|
|
## Related
|
|
|
|
Shipped the same day, same area, also untested live: **`model:resolved`** — the model dropdown used to render
|
|
the literal string `claude-code` for any chat that named no tier, because `DEFAULT_MODEL` is the bare harness
|
|
and names no catalogue entry. The harness reports its actual model in `system.init`; the parser was throwing
|
|
it away. Now forwarded, but **only when the turn named no tier** — the alias map is lossy (`opus[1m]` and
|
|
`opusplan` both answer as `claude-opus-5`), so forwarding unconditionally would relabel the user's own choice.
|
|
Display only: not durable, not recorded, never written into `selectedModel`. That one needs
|
|
`officer-claude-code` restarted as well as `officer`.
|