reported from the field on the first real push: it failed with BadDeviceToken and the server deleted the registration. the build was production-signed and the row said sandbox — a configuration mismatch, with the device and the app both perfectly fine. pruning there destroys the evidence. the phone re-registers on next launch, the row comes back looking healthy, and the next send fails identically; you can lose a day to that loop without ever seeing what is actually wrong. so the one set becomes three. Unregistered/UNREGISTERED/NOT_FOUND prune — apple and google are saying the app is uninstalled. BadDeviceToken, DeviceTokenNotForTopic, TopicDisallowed, INVALID_ARGUMENT and SENDER_ID_MISMATCH are OURS: the row is kept and the log says what to check. everything else is transient and still counts toward three strikes. INVALID_ARGUMENT was the sharpest one. it is how fcm reports a malformed MESSAGE — including a non-string in `data`, which is the mistake this codebase is most likely to make and already has a test for. deleting on it meant one bug in the payload builder would wipe every android device from the registry on a single send, presenting as "android push stopped working" with nothing left to explain why. misconfigured is also not counted as a strike. not deleting is not enough on its own — a persistent mismatch would reach three failures and delete the row anyway, which is the same bug with a slower fuse. the apns log now names the mismatch rather than the reason code, since the code alone sends people to look at the device: it prints the row's environment and topic and says to check the SHIPPED artefact rather than the intermediate build, which is exactly where this one hid.
343 lines
20 KiB
Markdown
343 lines
20 KiB
Markdown
# Push notifications — design
|
|
|
|
**Status (2026-08-19):** server side **built and core**; app side **not started**. This is the second
|
|
design. It supersedes the 2026-07-31 one, which is summarised under "What changed and why" so the
|
|
reversals are legible rather than silently overwritten.
|
|
|
|
The app-side contract and build order is **`docs/push-notifications-app.md`** — hand that to whoever builds
|
|
the client. This file is the design and the reasoning.
|
|
|
|
## The shape, in one paragraph
|
|
|
|
One app on the device — the main Officer app — receives every push and is the only one that can. It is a
|
|
**hub**: the payload it gets is a **code** plus a few numbers, and the app turns that into the notification
|
|
you actually see, deep-linked to whichever app and event it concerns. The code's meaning lives in a
|
|
**mapping owned by the server** and synced to the device ahead of time, so adding a new kind of
|
|
notification is a server change and never an app build. Plugins declare their own codes the same way they
|
|
declare affordances.
|
|
|
|
## What changed and why
|
|
|
|
The first design was a **doorbell**: a category and an id, never content, with the visible text composed
|
|
server-side from the category alone. It called that "the load-bearing decision, and it is not optional".
|
|
|
|
Three things moved.
|
|
|
|
**1. Privacy relaxed, but not abandoned.** The original weighting — nothing readable may reach Apple or
|
|
Google — was reconsidered on 2026-08-19 as stricter than the owner actually wanted. What replaced it is
|
|
not "send plaintext": it is **codes**, which keeps content off Apple's and Google's servers while removing
|
|
the reason the doorbell was painful (a rigid category enum). The privacy property survives almost intact;
|
|
what was given up is the claim that it was inviolable.
|
|
|
|
**2. One receiver, not many.** The first design had each app register its own token and `appSlug` select
|
|
which app a notification went to. Now a single hub app receives everything and `appSlug` names the
|
|
**deep-link destination** instead. Same field, opposite direction — see "Migration notes".
|
|
|
|
**3. Extensibility became the priority.** The binding constraint is now "a new notification kind must not
|
|
require an app store build". That single requirement is what forces the server-owned mapping, and it is
|
|
worth more than the category purity the first design optimised for.
|
|
|
|
**What did not change:** content still does not travel. Apple and Google see a code and some integers.
|
|
|
|
## Why a hub app
|
|
|
|
- **One entitlement, one topic, one sender.** A single bundle id means one `aps-environment`, one APNs
|
|
topic and one FCM sender, instead of credentials multiplying per app.
|
|
- **It resolves an existing contradiction.** `apps/chat` ships `withoutPushEntitlement.js`, a config plugin
|
|
that deliberately STRIPS `aps-environment` because chat uses local notifications only. Under the first
|
|
design that was a blocker. Under this one it is correct and stays — chat is not a push receiver, the hub
|
|
is.
|
|
- **It concentrates the Android problem** rather than multiplying it (see "Android and GrapheneOS").
|
|
|
|
## The payload
|
|
|
|
An ordinary **alert** push — not a silent one — carrying a code and numbers:
|
|
|
|
```jsonc
|
|
{
|
|
"aps": {
|
|
"alert": { "title": "Officer" }, // fallback only; see "What the OS shows"
|
|
"mutable-content": 1,
|
|
"sound": "default",
|
|
"thread-id": "officerdev/music",
|
|
"badge": 3,
|
|
},
|
|
"officer": {
|
|
"c": "officerdev/music:12.3", // the code — namespaced, see below
|
|
"n": 3, // variable numbers, never content
|
|
"id": 412, // what to fetch/open, never a human-readable string
|
|
"mv": 47, // mapping version the server rendered against
|
|
},
|
|
}
|
|
```
|
|
|
|
**Codes are namespaced, for the same reason affordances are.** `<publisher>/<appName>:<code>`. Plugins from
|
|
different publishers will otherwise collide on a bare integer, and a code in a log should say whose it is
|
|
without a lookup. `core` occupies the appName slot for the platform's own.
|
|
|
|
**Variable data rides beside the code, never inside it.** The mapping holds a template — `"{n} new emails"` —
|
|
and the push supplies `n`. Numbers and ids are not content, so the privacy property survives templating.
|
|
|
|
**What does NOT go in:** subject lines, senders, filenames, error strings, agent output, paths. If a value
|
|
would embarrass you in a Google log, it belongs behind `id` and gets fetched over the tailnet on tap.
|
|
|
|
## What the OS shows, and why `mutable-content` is required
|
|
|
|
A normal alert push is rendered **by the operating system, not by the app**. Send `alert.title = "12.3"`
|
|
and the user sees `12.3` on the lock screen — the app never gets a chance to touch it.
|
|
|
|
So the hub needs a **Notification Service Extension**, enabled per-push by `mutable-content: 1`. The
|
|
extension runs before display, resolves the code, and rewrites the banner.
|
|
|
|
**Crucially, the extension does a LOCAL lookup, not a network fetch.** The mapping is synced in advance by
|
|
the app into a shared App Group container; at notification time it is a dictionary read. This is what makes
|
|
the design work where a fetch-on-arrival design would not:
|
|
|
|
- no dependence on the tailnet being connected when the push lands
|
|
- no 30-second budget to overrun
|
|
- no partial failure mode on cellular with the VPN off
|
|
|
|
**This costs exactly one app build, ever — never one per notification kind.** An extension that hardcoded a
|
|
`switch` on type would need rebuilding for every new kind; this one is a generic resolver and does not.
|
|
|
|
**The alert text you send is the degraded case.** If the extension does not run — low memory, a crash, an
|
|
older OS — the user sees exactly what was in `alert`. So it is a graceful generic ("Officer"), deliberately
|
|
**not** plausible-looking decoy text: a convincing fake sentence would mislead the one person who most needs
|
|
to trust it.
|
|
|
|
**Silent pushes were considered and rejected.** `content-available: 1` with no alert would let the app
|
|
compose everything and leak least, but Apple throttles silent pushes and explicitly does not guarantee
|
|
delivery. For notifications that must arrive, that is the wrong tool.
|
|
|
|
**Android needs no extension.** An FCM **data-only message at high priority** wakes the app, which composes
|
|
the notification itself. Same codes, same mapping.
|
|
|
|
## The mapping
|
|
|
|
The server owns it; the device caches it.
|
|
|
|
```
|
|
GET /api/notify/mapping → { version: 47, entries: { "officerdev/music:12.3": {...} } }
|
|
```
|
|
|
|
An entry says how to render and where to go:
|
|
|
|
| field | note |
|
|
| -------- | -------------------------------------------------------------- |
|
|
| `title` | template, `{n}`-style placeholders |
|
|
| `body` | template, optional |
|
|
| `target` | which app the deep link opens |
|
|
| `route` | the deep link, with `{id}` interpolated |
|
|
| `thread` | grouping key, so like-notifications collapse rather than stack |
|
|
|
|
**Plugins declare their own**, on the affordances pattern (`src/servers/notifications/publish.ts`, which
|
|
deliberately mirrors `affordances/publish.ts`): a `notifications/` directory of YAML in the plugin
|
|
repository, copied to `capabilities/notifications/<publisher>/<appName>/` on install and removed on
|
|
uninstall.
|
|
|
|
The namespace is **derived from where the file lands, never from what it declares** — the same rule as
|
|
`mountPrefix`. A plugin therefore cannot claim `officer/core:` or another publisher's codes by writing
|
|
them into its own catalogue; it would have to install itself somewhere it cannot. This is pinned by test. That mechanism already exists and is understood; a second one for
|
|
notifications would drift from it. The platform aggregates every installed plugin's catalogue plus its own
|
|
into the response above, and `version` increments whenever that aggregate changes.
|
|
|
|
### Staying in sync, and the fallback
|
|
|
|
Every push carries `mv`, the mapping version it was rendered against. The app compares it to its cached
|
|
version and **resyncs in the background when they differ** — so the mapping normally catches up before an
|
|
unknown code can arrive.
|
|
|
|
`version` is a **content hash, not a counter**, so the test is `mv !== version` rather than `mv > version`.
|
|
A counter would need somewhere to live and would then be wrong after a restore, a rollback, or two servers
|
|
built from the same install; a hash needs no storage and is right in all three.
|
|
|
|
When a push still outruns the sync, the fallback renders:
|
|
|
|
> **Unknown notification**
|
|
> code `officerdev/music:12.3` — tap to sync
|
|
|
|
deep-linking to `officer://resync-notification-mapping`. The code goes **in the body on purpose**: when this
|
|
fires on the owner's own phone, the first useful question is which code the server sent that the app did not
|
|
have, and without it that is debugged blind.
|
|
|
|
**The resync screen must be able to give up.** If a plugin is uninstalled its codes leave the catalogue, so
|
|
a queued notification for one will resync, still not resolve, and otherwise loop forever. That path ends
|
|
with "this came from something no longer installed", not another retry.
|
|
|
|
## What this still leaks
|
|
|
|
Apple and Google see **which code, how often, at what time, to which device**. Over months that is a
|
|
fingerprint of the owner's category vocabulary even though every code is opaque. That is much less than
|
|
plaintext and is judged acceptable — but it is not nothing, and this document does not claim otherwise.
|
|
|
|
## Where it lives
|
|
|
|
`officer-notify`, a **core** PM2 process since 2026-08-19 (built 2026-07-31; spent 2026-08-13 to 08-19
|
|
switched off after being slated for plugin extraction and never extracted). Core rather than a plugin
|
|
because the producers are the platform's own — the job queue, an agent turn ending, mail arriving — and a
|
|
notifier the owner can uninstall is one those producers cannot rely on.
|
|
|
|
```
|
|
producer (queue / email / agent / platform / plugin)
|
|
│ POST /_officer/notify { code, userId, n?, id? }
|
|
▼
|
|
officer-notify ─┬─ apns → api.push.apple.com (HTTP/2, ES256 JWT)
|
|
├─ fcm → fcm.googleapis.com/v1 (OAuth2 bearer)
|
|
└─ discord→ webhook
|
|
```
|
|
|
|
It boots with **no credentials at all** — every channel reports `isConfigured() === false` and `dispatch`
|
|
resolves empty — so an install that never sets an APNs key runs it as a no-op rather than crash-looping.
|
|
That is what makes it safe as core.
|
|
|
|
## Credentials
|
|
|
|
Env only, on the sidecar, never in the platform process and never in the database. All read
|
|
`OFFICERDEV_<NAME> ?? <NAME>`.
|
|
|
|
| var | what |
|
|
| --------------------- | ------------------------------------------------ |
|
|
| `APNS_KEY_P8` | contents of the `.p8` auth key (PKCS#8 EC P-256) |
|
|
| `APNS_KEY_ID` | the key's 10-char Key ID |
|
|
| `APNS_TEAM_ID` | Apple Developer Team ID |
|
|
| `FCM_SERVICE_ACCOUNT` | the service-account JSON |
|
|
| `DISCORD_WEBHOOK_URL` | existing |
|
|
|
|
The `.p8` does not expire and can push to your apps forever. Treat it as spending-grade.
|
|
|
|
**The secret store is where these live now** — env is only the fallback.
|
|
|
|
```bash
|
|
cat AuthKey_ABCD123456.p8 | bun run secret:set apns-p8
|
|
echo -n ABCD123456 | bun run secret:set apns-key-id
|
|
echo -n RGHC4NJC77 | bun run secret:set apns-team-id
|
|
pm2 restart officer-notify # _health should then report channels: ["apns"]
|
|
```
|
|
|
|
Purposes: `apns-p8`, `apns-key-id`, `apns-team-id`, `fcm-service-account`, `discord-webhook`. Read with
|
|
`getSecret` and NOT `getKey` — `getKey` mints a random value for an absent purpose, which for a credential
|
|
supplied from outside would mean silently authenticating to Apple with a fake.
|
|
|
|
Env is still read when the store has nothing, so an install that predates this keeps working. But it is a
|
|
fallback: bun auto-loads `.env` into every PM2 process, so a key there is readable from `/proc/<pid>/environ`
|
|
of twenty of them, and `officer-music` has no business being able to read a credential that can push to your
|
|
apps forever.
|
|
|
|
`APNS_ENV` appeared in the first design's table and is deliberately **not** implemented: the host is chosen
|
|
per device from the `environment` column, not per process, so one server pushes to a TestFlight build and a
|
|
release build at the same time.
|
|
|
|
## Both protocols, verified in Bun before designing around them
|
|
|
|
- **APNs**: HTTP/2 only. `node:http2` works in Bun. Auth is an ES256 JWT signed with the `.p8`, valid ≤1h,
|
|
refreshed no more often than every 20 min or Apple rejects it. **The signature must be raw `r||s`
|
|
(64 bytes)** — `createSign(...).sign({ key, dsaEncoding: 'ieee-p1363' })`. Node's default DER encoding is
|
|
silently rejected.
|
|
- **FCM v1**: ordinary HTTPS. Sign an RS256 JWT with the service-account key, exchange it at
|
|
`oauth2.googleapis.com/token` for a 1-hour access token, cache it, then POST to
|
|
`fcm.googleapis.com/v1/projects/<id>/messages:send`.
|
|
|
|
No push library is needed for either.
|
|
|
|
## Device registry
|
|
|
|
`push_devices` — live in the schema as of 2026-08-19.
|
|
|
|
| column | note |
|
|
| --------------------------------------------- | ----------------------------------------------------------------------------------------------- |
|
|
| `id` | serial |
|
|
| `user_id` | fk users |
|
|
| `token` | APNs device token or FCM registration token |
|
|
| `platform` | `ios` \| `android` |
|
|
| `environment` | `production` \| `sandbox` — a dev-build token fails against prod with a silent `BadDeviceToken` |
|
|
| `bundle_id` | `apns-topic` |
|
|
| `app_slug` | see migration note |
|
|
| `last_seen_at`, `failure_count`, `created_at` | pruning |
|
|
|
|
Unique on `(token, bundle_id)`. Registration is idempotent: the app re-registers on every launch, since
|
|
tokens rotate.
|
|
|
|
### Migration notes from the first design
|
|
|
|
- **`appSlug` on a NOTIFICATION inverts.** It meant "deliver only to this app". It now means "the deep-link
|
|
destination". Nothing consumes it yet, so this is a rename-in-place, not a data migration.
|
|
- **`bundle_id` / `app_slug` on a DEVICE go near-constant.** The columns exist to let several apps register
|
|
separately; with a hub there is one bundle id per device. Keep them — a second receiver later (a watch, a
|
|
desktop build) costs nothing this way — but "which apps has this user got" stops being a delivery question.
|
|
- ~~**`NotifyType` is replaced by codes.**~~ Done. The producer contract is now
|
|
`POST /_officer/notify { code, userId, n?, id? }`, validated on the SHAPE of the code
|
|
(`<publisher>/<appName>:<key>`) and deliberately **not** against the catalogue — a producer firing a
|
|
code the catalogue has not caught up with is sending a real event, and the device already has a defined
|
|
fallback. `ok` is gone: a failed job is its own code (`officer/core:job-failed`), which is what made the
|
|
enum rigid in the first place.
|
|
- ~~**`text.ts` moves to the device.**~~ Done — replaced by `render.ts`, which renders server-side only for
|
|
Discord (no app to resolve a code) and for the `alert` fallback string.
|
|
|
|
## Failure handling
|
|
|
|
**Only "the app is gone" prunes a row.** The first version pruned on every 4xx Apple or Google offered,
|
|
and that was wrong in a way that hides itself.
|
|
|
|
| reason | what it means | what happens |
|
|
| ------------------------------------------------------------- | ------------------------------- | ---------------------------------------------------- |
|
|
| `Unregistered` (410), `UNREGISTERED`, `NOT_FOUND` | the app was uninstalled | **delete the row** |
|
|
| `BadDeviceToken`, `DeviceTokenNotForTopic`, `TopicDisallowed` | **our** configuration is wrong | keep it, log the diagnosis |
|
|
| `INVALID_ARGUMENT`, `SENDER_ID_MISMATCH` | **we** sent something malformed | keep it, log the diagnosis |
|
|
| anything else | transient | `failure_count += 1`, three strikes and the row goes |
|
|
|
|
`BadDeviceToken` is almost always an environment mismatch — a token minted under
|
|
`aps-environment: development` sent to the production host, or the reverse. The device is fine, the app is
|
|
installed, the registration is good; the server dialled the wrong Apple. **Deleting the row there destroys
|
|
the evidence**: the phone re-registers on next launch, the row comes back looking healthy, and the next
|
|
send fails identically. Found in the field on the first real push, 2026-08-19.
|
|
|
|
`INVALID_ARGUMENT` is worse still, because it is how FCM reports a malformed MESSAGE — including a
|
|
non-string in `data`, the mistake this codebase is most likely to make and has a test for. Pruning on it
|
|
means one bug in the payload builder wipes every Android device in the registry on a single send.
|
|
|
|
**Misconfigured is not counted as a strike either.** Not deleting is not enough on its own: a persistent
|
|
mismatch would otherwise reach three failures and delete the row anyway. Same bug, slower fuse.
|
|
|
|
## Android and GrapheneOS — UNRESOLVED, and it decides Android
|
|
|
|
`apps/mobile/PUSH_NOTIFICATIONS.md` (2026-07-24) rejects FCM outright: _"requires Google Play Services; the
|
|
Pixel runs GrapheneOS without it."_ The 2026-07-31 platform design chose FCM for Android a week later and
|
|
does not mention GrapheneOS. **Nobody reconciled the two, and this design does not either.**
|
|
|
|
It matters because FCM cannot reach a device with no Play Services, so on such a phone the entire Android
|
|
path is inert regardless of anything above.
|
|
|
|
The recorded fallback is a **foreground service holding a tailnet socket**, with notifications raised
|
|
locally. It was rejected in the first design for a permanent "Officer is running" entry in the shade and
|
|
unreliability against OEM battery managers. **The hub changes that calculus**: one socket and one shade
|
|
entry for the whole platform, not one per app. The battery-manager objection is also weaker on GrapheneOS
|
|
than on Samsung or Xiaomi.
|
|
|
|
This needs an owner decision before Android work starts. iOS is unaffected and can proceed independently.
|
|
|
|
## Build order
|
|
|
|
1. ~~`push_devices` table.~~ done
|
|
2. ~~`officer-notify` sidecar, PM2 entry, `/api/notify` proxy, Discord channel.~~ done
|
|
3. ~~APNs channel + device registration endpoint.~~ done, 27 tests
|
|
4. ~~FCM channel.~~ done, 27 tests
|
|
5. **APNs credentials.** Owner action; nothing can be tested end-to-end without them.
|
|
6. **Hub app registers its token** against `POST /api/notify/_officer/devices`. Small.
|
|
7. **First light**: a `test` push rendered by the OS with plain text, no codes, no extension. Proves
|
|
credential → APNs → device → banner as one thing, before any of the mapping exists.
|
|
8. **Codes and the mapping**: `NotifyType` → codes, `GET /api/notify/mapping`, plugin catalogues on the
|
|
affordances pattern.
|
|
9. **The Notification Service Extension**, local lookup, App Group container.
|
|
10. **First real producer** — job finished is the obvious one.
|
|
|
|
Steps 7 and 8 are deliberately separable: the transport can be proven working before the rendering exists,
|
|
and if it is not, no amount of mapping work will help.
|
|
|
|
## Still open
|
|
|
|
- **Which events notify**, beyond the first producer. Quiet hours and batching — new mail on an 18k-mail
|
|
account is unusable at one push per message.
|
|
- **Whether members can register devices.** The `notify` permission is `kind: 'app'`, so it is grantable per
|
|
role and denied by default; a member with no grant cannot subscribe their own phone.
|
|
- **Android**, per above.
|