diff --git a/CLAUDE.md b/CLAUDE.md index 8c6db9d9..cfbbdf11 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -421,6 +421,9 @@ explaining why it was safe. - `docs/navigation-audit.md` — **authoritative** on routing/navigation: the opaque-click anti-pattern, a severity-ranked findings table, the channel-selection map and the four-phase plan +- `docs/email-recall.md` — **design, not built**: recall over your own correspondence. The split that + matters is counting (needs every message judged) versus retrieval (needs the best few, fast) — and why + a number produced by searching is confidently wrong with no signal that it is - `docs/find-my-device.md` — **design, not built**: locating your own devices from Officer. What is assembly (most of it), what is impossible (offline finding — there is no crowd to relay it), and the question to settle first, which is retention rather than anything technical diff --git a/docs/email-recall.md b/docs/email-recall.md new file mode 100644 index 00000000..cba7210d --- /dev/null +++ b/docs/email-recall.md @@ -0,0 +1,159 @@ +# Recall over your own correspondence + +**Status: DESIGN, NOT BUILT.** Sketched 2026-08-20, for the email plugin. + +## The goal, which is not the technology + +Someone is on a phone call with their ex-wife. She says something about the weekend arrangement. He has +eight years of email with her, and he needs the fact **now** — not after a search, not after reading +twenty messages. + +That is the whole requirement. Everything below is in service of it, and any part of this that does not +make that moment faster or more accurate is decoration. + +The user will never write a query. They ask the agent a question in their own words, and the agent picks +its tools. So the design question is not "what can the user search" — it is **"what does the agent have +to work with, and which of its answers can be trusted"**. + +## Two mechanisms, because there are two kinds of question + +The examples that prompted this split cleanly, and they need different machinery: + +| the question | what it needs | +| ---------------------------------------------------------------- | ------------------------------------- | +| _"how many times did she ask me to break the weekend agreement"_ | **exhaustion** — every message judged | +| _"what did she say about the school run"_ | **retrieval** — the best few, fast | + +``` +COMPILE exhaustive, once, expensive → counting, timelines, "show me all of them" +RETRIEVE cheap, approximate, always → exploring, mid-call, "what did she say about…" +``` + +They are not competing designs. The agent should have both and choose. + +### Counting cannot be done by searching + +This is the load-bearing point, and it is the one most likely to be skipped. + +An agent asked "how many times" will search, find some, and answer with a number. That number is bounded +by the phrasings it happened to think of. **"I found four" is indistinguishable from "there were nine"**, +and nothing in the output signals which it was. + +Vector search does not fix this. Top-k also misses — it just misses differently. Ask for the top 20 and +the 21st is silently gone. + +The only honest way to count is to judge every message once and store the verdict. Then counting is +`SELECT count(*)`: exact, instant, and re-askable a hundred ways without re-reading anything. + +**A stored verdict is a judgement a model made about someone's divorce.** It records what it was asked, +which message it read, and when — so the answer can be inspected, disbelieved and redone. A number with +no provenance is worse than no number, because it will be quoted. + +### Retrieval is where embeddings earn their place + +Keyword search finds `school run`. It will never find it from _"the pickup arrangement"_ or _"when I +collect her on Fridays"_. Over eight years, the same thing is described twenty ways and never once in the +words you would search for. + +An agent closes some of that gap by itself — it knows those are synonyms and can fire several searches. +But that is **guess-bounded**: it finds the phrasings it thought of, and cannot know the ones it did not. + +Two things decide it in favour of embeddings anyway: + +- **Latency.** Five keyword searches and twenty candidate reads is thirty seconds. On a live call that is + a lost moment. A vector query is milliseconds plus one read. +- **What comes later.** WhatsApp and audio transcripts share no vocabulary, schema or structure with + email. Embeddings collapse all of it into one space. ASR output is noisy in a way keyword search is + brittle against and embeddings are not. + +**But we do not have WhatsApp or transcripts.** They shape exactly one decision — see the schema — and +nothing else. Build against email. + +## It has to run locally + +Every message passes through an embedding model. Doing that through an API means uploading eight years of +private correspondence to a third party. + +For this content that is not a trade-off, it is a disqualification. **Local embedding is a requirement, +not a preference.** + +Today there is no ollama and no embedding code on any of these machines. **That is the real new +dependency — not the database.** + +## Not a vector database + +`sqlite-vec` is an extension to the SQLite already in use. Same file as `emails.db`, no service, no +sidecar, no Docker, no port, nothing to supervise or back up separately. + +For a plugin that already owns that file, this is **a table, not an architecture**. A separate vector +database would add an operational surface for a corpus of 19,000 rows, which is small — vector stores +begin earning their keep in the millions. What is being bought here is recall on paraphrase and speed on +a phone call, not scale. + +## The genuinely hard part is chunking + +Not the vectors. **Quoted reply chains.** + +The same paragraph exists in forty messages of one thread. Embed naively and every query returns that +thread forty times, and it will feel worse than what exists today. + +So: strip quoted history, thread properly, embed the new text of each message. `emails` already has +`thread_id`. This is the work. + +Two other email-specific traps measured on the real 19,097-message store: + +- **Bodies are ~0.5 GB of HTML.** Embed the text, not the markup. +- **`labels` is a comma-separated string**, and `read` is populated on 93 rows. Neither is a filter to + build on. See the `email-db` skill. + +## Schema sketch + +Deliberately **not** email-shaped. That is the one concession to a future that does not exist yet, and it +costs nothing now: + +``` +recall_chunks + id integer pk + source text -- 'email' today. 'whatsapp', 'transcript' later. + source_id text -- the message id in ITS store — no foreign key across sources + account text -- which mailbox/number this came from + thread_id text -- for dedupe and for showing context + author text -- who said it + said_at timestamp + text text -- quoted history stripped + embedding float[] -- sqlite-vec + +recall_verdicts -- the COMPILE half + id integer pk + question text -- what the model was asked, verbatim + source text + source_id text + verdict boolean + reasoning text + model text -- which model, so a re-run is comparable + judged_at timestamp +``` + +Everything else assumes email until something else exists. + +## Build order + +1. **Chunking and dedupe**, with no embeddings at all. Strip quotes, thread, count what is left. If this + is wrong nothing after it can be right, and it is testable on its own. +2. **A local embedding model**, and a decision about which. The new dependency. +3. **`sqlite-vec` + backfill** of the existing corpus. One-off, slow, resumable. +4. **A retrieval tool** the agent can call, returning chunks with their message ids so it can go read the + original. +5. **The compile half** — a verdict table and a way to ask "judge every message against this question". +6. Multi-source, if and when WhatsApp or transcripts exist. + +Steps 1 and 5 are useful without any of the others. Step 5 in particular answers the questions that +prompted this, and needs no vectors at all. + +## Open questions + +- **Which local model**, and how big. Quality against a laptop-or-server-sized footprint. +- **Re-embedding.** Changing model means re-embedding everything. Store which model produced each vector. +- **Retention and deletion.** A deleted email should lose its chunks and its verdicts. Nothing here + currently cascades. +- **Who can query it.** This is the most sensitive data in the platform, and it is per-account.