The previous commit migrated only when SQLite was completely empty, on the assumption that a non-empty
file is an authoritative one. A real account disproved that within minutes of it landing.
The older Gmail backfill had already written SOME keys into SQLite — last_sync_at and the uidvalidity
set — and never the imap_lastuid ones. So the file was non-empty and half-migrated at the same time,
all-or-nothing skipped the migration, and nine imap_lastuid keys stayed only in Postgres. A missing
lastuid makes the next sync refetch that folder from UID 1: on the mailbox this was found on, 18,755
messages and 6.9 GB.
Now merged per key with the file always winning a conflict. That keeps the property all-or-nothing was
protecting — a restored older emails.db still overrides a newer Postgres row for every key it has, so it
cannot be advanced past mail it does not contain — and adds the keys the file never had, which are
exactly the ones whose absence is expensive.
Verified against the live account: all 22 Postgres keys present afterwards, imap_lastuid:INBOX restored
to 208407, and last_sync_at left at the file's older value, so it re-checks a fortnight rather than
skipping it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The messages were in emails.db and the position — last_sync_at, and per-folder uidvalidity/lastuid —
was a jsonb column on email_accounts in Postgres. Two stores for one fact, with an edge that only shows
up when you try to move a mailbox to another machine.
The expensive part of an email account is the first sync: hours of IMAP for a large mailbox, which is
exactly why "copy emails.db to the new server" is the obvious way to bring one across. With the position
in Postgres that silently does not work — the new server's column is empty, !last_sync_at says first
sync, and the whole mailbox downloads again on top of the one just restored.
The other direction is quieter and worse. Restore an OLDER emails.db while Postgres holds a NEWER
position and the sidecar skips every message between the two, permanently, because nothing looks below
lastuid again. Re-syncing is slow; skipping mail is data loss nobody notices.
Not a new idea — the Gmail path already read SQLite and fell back to Postgres, backfilling so the
fallback was taken once. Only the IMAP path had not followed. This extracts that pattern so both use one
copy, and unifies the isFirstSync fork in accounts.ts, which is how the two drifted apart to begin with.
The file wins over Postgres, always, and only migrates when it holds nothing at all. Topping up a
partial position from Postgres would reintroduce precisely the divergence this removes.
email_accounts.sync_meta is kept and marked legacy rather than dropped: it is the one-time backfill
source for every account created before this, and dropping it would strand any that has not synced
since. Nothing writes to it now.
11 tests on the migration, aimed at both expensive failures — migrating when we should not, and failing
to migrate an account that predates the change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
pastilhasandClaude Opus 5