officerdev/platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

push notifications: the hub design

Browse Source 
second design, superseding 2026-07-31. the reversals are recorded rather than
overwritten, because the first one called its payload rule "not optional" and
this changes it.

one app on the device receives everything and renders it. the payload is a
namespaced code plus integers; the code's meaning lives in a server-owned
mapping synced to the device ahead of time, so a new notification kind is a
server change and never an app build. plugins declare their own codes on the
affordances pattern, which already does exactly this shape.

what moved: privacy relaxed but not abandoned — codes keep content off apple
and google while dropping the rigid category enum that made the doorbell
painful. appSlug inverts from "deliver to this app" to "deep-link to this
app". extensibility becomes the binding constraint.

the piece that makes it work is that the ios extension does a LOCAL lookup
against a synced mapping, not a fetch on arrival — so it does not depend on
the tailnet being up when the push lands, and it costs one build ever rather
than one per kind.

records two things it does not solve: the code vocabulary is still a
fingerprint to apple and google, and the grapheneos/fcm contradiction between
apps/mobile/PUSH_NOTIFICATIONS.md and the platform design is still unresolved.
This commit is contained in:
pastilhas
2026-08-19 18:41:33 +00:00
parent fd7c9becb6
commit 347e679712
1 changed files with 230 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/push-notifications.md
+230 -94
View File
@@ -1,82 +1,187 @@
# Push notifications — design
**Status (2026-08-19):** the server side is **built and core**. The app side is **not started**, and the
handoff document this file promised was never written.
**Status (2026-08-19):** server side **built and core**; app side **not started**. This is the second
design. It supersedes the 2026-07-31 one, which is summarised under "What changed and why" so the
reversals are legible rather than silently overwritten.
- **Server: done and live.** `officer-notify` runs as a core PM2 process, `/api/notify` is mounted and
permission-gated, and `push_devices` is in the schema. Steps 1–4 of the build order below are complete;
step 5 (a first producer) is not.
- **Switched off and back on.** On 2026-08-13 this was slated to become a plugin alongside memos and
photos. Those two were extracted; this never was, so it spent six days mounted nowhere with its schema
commented out. Restored to core on 2026-08-19 — core rather than a plugin because the producers are
the platform's own (the queue, an agent turn ending, mail arriving), and a notifier the owner can
uninstall is one those producers cannot rely on.
- **App side: nothing.** `officer-suite` does not exist; the apps repo is `officerdev/monorepo-mobile`.
It has no `COMMS/` directory, no `PUSH_NOTIFICATIONS_APP.md`, no APNs/FCM token handling and no call to
the device-registration endpoint. `apps/chat` ships a config plugin that deliberately STRIPS the iOS
push entitlement (it uses local notifications only), so today's iOS build cannot receive a remote push
at all. The only app-side note, `apps/mobile/PUSH_NOTIFICATIONS.md` (2026-07-24), predates this design
and CONTRADICTS it — it rejects FCM outright because the Pixel runs GrapheneOS without Play Services.
**That contradiction is unresolved and is the first thing to settle**, because it decides Android.
## The shape, in one paragraph
## What was decided, and what was rejected
One app on the device — the main Officer app — receives every push and is the only one that can. It is a
**hub**: the payload it gets is a **code** plus a few numbers, and the app turns that into the notification
you actually see, deep-linked to whichever app and event it concerns. The code's meaning lives in a
**mapping owned by the server** and synced to the device ahead of time, so adding a new kind of
notification is a server change and never an app build. Plugins declare their own codes the same way they
declare affordances.
**Apple and Google are unavoidable and we accept them.** There is no direct server→phone push on either
platform: iOS suspends apps, so only APNs can wake one, and Android only accepts pushes from FCM. An
earlier plan had these the other way round — believing iOS could be pushed to directly and Android needed
a bridge. It is the reverse.
## What changed and why
**Expo is rejected.** Its push service is a relay in front of APNs and FCM, and it does not remove either
credential — a standalone Android build still needs an FCM service account, which you would upload _to
Expo_. All it adds is fan-out convenience, in exchange for our payloads and our APNs key living on a
fourth party's servers. We talk to Apple and Google ourselves.
The first design was a **doorbell**: a category and an id, never content, with the visible text composed
server-side from the category alone. It called that "the load-bearing decision, and it is not optional".
**The Android foreground-service alternative is rejected for now.** A persistent socket plus a local
notification genuinely avoids Google, and the tailnet makes the connection easy. It costs a permanent
"Officer is running" entry in the notification shade and is unreliable against OEM battery managers
(Samsung, Xiaomi). Recorded here because it remains the fallback if FCM ever becomes unacceptable.
Three things moved.
## Payload policy — the notification is a doorbell, not a message
**1. Privacy relaxed, but not abandoned.** The original weighting — nothing readable may reach Apple or
Google — was reconsidered on 2026-08-19 as stricter than the owner actually wanted. What replaced it is
not "send plaintext": it is **codes**, which keeps content off Apple's and Google's servers while removing
the reason the doorbell was painful (a rigid category enum). The privacy property survives almost intact;
what was given up is the claim that it was inviolable.
**This is the load-bearing decision, and it is not optional.**
**2. One receiver, not many.** The first design had each app register its own token and `appSlug` select
which app a notification went to. Now a single hub app receives everything and `appSlug` names the
**deep-link destination** instead. Same field, opposite direction — see "Migration notes".
Both services see metadata regardless: which device, how often, at what times. What they must not see is
content. So a push carries a category and an id, never the substance:
**3. Extensibility became the priority.** The binding constraint is now "a new notification kind must not
require an app store build". That single requirement is what forces the server-owned mapping, and it is
worth more than the category purity the first design optimised for.
```
✅ { type: 'mail', count: 3 } ❌ { title: 'Re: invoice', from: 'x@y.com' }
✅ { type: 'job', id: 412, ok: false } ❌ { error: 'ENOENT /home/pastilhas/…' }
**What did not change:** content still does not travel. Apple and Google see a code and some integers.
## Why a hub app
- **One entitlement, one topic, one sender.** A single bundle id means one `aps-environment`, one APNs
topic and one FCM sender, instead of credentials multiplying per app.
- **It resolves an existing contradiction.** `apps/chat` ships `withoutPushEntitlement.js`, a config plugin
that deliberately STRIPS `aps-environment` because chat uses local notifications only. Under the first
design that was a blocker. Under this one it is correct and stays — chat is not a push receiver, the hub
is.
- **It concentrates the Android problem** rather than multiplying it (see "Android and GrapheneOS").
## The payload
An ordinary **alert** push — not a silent one — carrying a code and numbers:
```jsonc
{
"aps": {
"alert": { "title": "Officer" }, // fallback only; see "What the OS shows"
"mutable-content": 1,
"sound": "default",
"thread-id": "officerdev/music",
"badge": 3,
},
"officer": {
"c": "officerdev/music:12.3", // the code — namespaced, see below
"n": 3, // variable numbers, never content
"id": 412, // what to fetch/open, never a human-readable string
"mv": 47, // mapping version the server rendered against
},
}
```
The app already has a tailnet connection. On tap it fetches the real content itself. The visible text is
generic ("3 new emails"), assembled on the device from the category, not sent through Apple or Google.
**Codes are namespaced, for the same reason affordances are.** `<publisher>/<appName>:<code>`. Plugins from
different publishers will otherwise collide on a bare integer, and a code in a log should say whose it is
without a lookup. `core` occupies the appName slot for the platform's own.
Cost of both services: **free**. No per-message charge on either.
**Variable data rides beside the code, never inside it.** The mapping holds a template — `"{n} new emails"` —
and the push supplies `n`. Numbers and ids are not content, so the privacy property survives templating.
**What does NOT go in:** subject lines, senders, filenames, error strings, agent output, paths. If a value
would embarrass you in a Google log, it belongs behind `id` and gets fetched over the tailnet on tap.
## What the OS shows, and why `mutable-content` is required
A normal alert push is rendered **by the operating system, not by the app**. Send `alert.title = "12.3"`
and the user sees `12.3` on the lock screen — the app never gets a chance to touch it.
So the hub needs a **Notification Service Extension**, enabled per-push by `mutable-content: 1`. The
extension runs before display, resolves the code, and rewrites the banner.
**Crucially, the extension does a LOCAL lookup, not a network fetch.** The mapping is synced in advance by
the app into a shared App Group container; at notification time it is a dictionary read. This is what makes
the design work where a fetch-on-arrival design would not:
- no dependence on the tailnet being connected when the push lands
- no 30-second budget to overrun
- no partial failure mode on cellular with the VPN off
**This costs exactly one app build, ever — never one per notification kind.** An extension that hardcoded a
`switch` on type would need rebuilding for every new kind; this one is a generic resolver and does not.
**The alert text you send is the degraded case.** If the extension does not run — low memory, a crash, an
older OS — the user sees exactly what was in `alert`. So it is a graceful generic ("Officer"), deliberately
**not** plausible-looking decoy text: a convincing fake sentence would mislead the one person who most needs
to trust it.
**Silent pushes were considered and rejected.** `content-available: 1` with no alert would let the app
compose everything and leak least, but Apple throttles silent pushes and explicitly does not guarantee
delivery. For notifications that must arrive, that is the wrong tool.
**Android needs no extension.** An FCM **data-only message at high priority** wakes the app, which composes
the notification itself. Same codes, same mapping.
## The mapping
The server owns it; the device caches it.
```
GET /api/notify/mapping → { version: 47, entries: { "officerdev/music:12.3": {...} } }
```
An entry says how to render and where to go:
| field | note |
| -------- | -------------------------------------------------------------- |
| `title` | template, `{n}`-style placeholders |
| `body` | template, optional |
| `target` | which app the deep link opens |
| `route` | the deep link, with `{id}` interpolated |
| `thread` | grouping key, so like-notifications collapse rather than stack |
**Plugins declare their own**, on the affordances pattern (`src/servers/affordances/publish.ts`): a
directory in the plugin repository, copied to a platform-managed store on install, removed on uninstall,
namespaced `<publisher>/<appName>/`. That mechanism already exists and is understood; a second one for
notifications would drift from it. The platform aggregates every installed plugin's catalogue plus its own
into the response above, and `version` increments whenever that aggregate changes.
### Staying in sync, and the fallback
Every push carries `mv`, the mapping version it was rendered against. The app compares it to its cached
version and **resyncs in the background when it is behind** — so the mapping normally catches up before an
unknown code can arrive.
When a push still outruns the sync, the fallback renders:
> **Unknown notification**
> code `officerdev/music:12.3` — tap to sync
deep-linking to `officer://resync-notification-mapping`. The code goes **in the body on purpose**: when this
fires on the owner's own phone, the first useful question is which code the server sent that the app did not
have, and without it that is debugged blind.
**The resync screen must be able to give up.** If a plugin is uninstalled its codes leave the catalogue, so
a queued notification for one will resync, still not resolve, and otherwise loop forever. That path ends
with "this came from something no longer installed", not another retry.
## What this still leaks
Apple and Google see **which code, how often, at what time, to which device**. Over months that is a
fingerprint of the owner's category vocabulary even though every code is opaque. That is much less than
plaintext and is judged acceptable — but it is not nothing, and this document does not claim otherwise.
## Where it lives
A new **`officer-notify` sidecar**, PM2 peer, its own loopback HTTP listener, announced as
`notify:server` and proxied at `/api/notify` by `createSidecarProxy`.
Not the platform, because the producers are spread out — the queue, the email sidecar, the agent sidecar —
and a platform-owned notifier would force every sidecar to call _back_ into the platform. That is the
inversion just removed from email. As a sidecar, anything POSTs to it over loopback.
It also absorbs `src/servers/notify/discord.ts`, so there is one outbound-notification surface with
channels behind it rather than a Discord path and a push path that do not know about each other.
`officer-notify`, a **core** PM2 process since 2026-08-19 (built 2026-07-31; spent 2026-08-13 to 08-19
switched off after being slated for plugin extraction and never extracted). Core rather than a plugin
because the producers are the platform's own — the job queue, an agent turn ending, mail arriving — and a
notifier the owner can uninstall is one those producers cannot rely on.
```
producer (queue / email / agent / platform)
│ POST /_officer/notify { type, ... }
producer (queue / email / agent / platform / plugin)
│ POST /_officer/notify { c, userId, n?, id? }
▼
officer-notify ─┬─ apns → api.push.apple.com (HTTP/2, ES256 JWT)
├─ fcm → fcm.googleapis.com/v1 (OAuth2 bearer)
└─ discord→ webhook (existing)
└─ discord→ webhook
```
It boots with **no credentials at all** — every channel reports `isConfigured() === false` and `dispatch`
resolves empty — so an install that never sets an APNs key runs it as a no-op rather than crash-looping.
That is what makes it safe as core.
## Credentials
Env only, on the sidecar, never in the platform process and never in the database.
Env only, on the sidecar, never in the platform process and never in the database. All read
`OFFICERDEV_<NAME> ?? <NAME>`.
| var | what |
| --------------------- | ------------------------------------------------ |
@@ -84,72 +189,103 @@ Env only, on the sidecar, never in the platform process and never in the databas
| `APNS_KEY_ID` | the key's 10-char Key ID |
| `APNS_TEAM_ID` | Apple Developer Team ID |
| `FCM_SERVICE_ACCOUNT` | the service-account JSON |
| `DISCORD_WEBHOOK_URL` | existing, moves here |
| `DISCORD_WEBHOOK_URL` | existing |
The `.p8` does not expire and can push to your apps forever. Treat it as spending-grade.
`APNS_ENV` was in this table and is **not** implemented, deliberately: the host is chosen per device from
the `environment` column, not per process, so one server can push to a TestFlight build and a release
build at the same time. All three read `OFFICERDEV_<NAME> ?? <NAME>`, the house env convention.
`APNS_ENV` appeared in the first design's table and is deliberately **not** implemented: the host is chosen
per device from the `environment` column, not per process, so one server pushes to a TestFlight build and a
release build at the same time.
## Both protocols, verified in Bun before designing around them
- **APNs**: HTTP/2 only. `node:http2` works in Bun 1.3.10 (checked against a live server). Auth is an
ES256 JWT signed with the `.p8`, valid ≤1h, refreshed no more often than every 20 min or Apple rejects
it. **The signature must be raw `r||s` (64 bytes)** — `createSign(...).sign({ key, dsaEncoding:
'ieee-p1363' })`. Node's default DER encoding is silently rejected.
- **APNs**: HTTP/2 only. `node:http2` works in Bun. Auth is an ES256 JWT signed with the `.p8`, valid ≤1h,
refreshed no more often than every 20 min or Apple rejects it. **The signature must be raw `r||s`
(64 bytes)** — `createSign(...).sign({ key, dsaEncoding: 'ieee-p1363' })`. Node's default DER encoding is
silently rejected.
- **FCM v1**: ordinary HTTPS. Sign an RS256 JWT with the service-account key, exchange it at
`oauth2.googleapis.com/token` for a 1-hour access token, cache that, then POST to
`oauth2.googleapis.com/token` for a 1-hour access token, cache it, then POST to
`fcm.googleapis.com/v1/projects/<id>/messages:send`.
No push library is needed for either — `node:crypto` and `node:http2` cover it. All three endpoints are
reachable from this host.
No push library is needed for either.
## Device registry
New table `push_devices`:
`push_devices` — live in the schema as of 2026-08-19.
| column | note |
| --------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| `id` | serial |
| `user_id` | fk users |
| `token` | the native token — APNs device token or FCM registration token |
| `platform` | `ios` \| `android` |
| `environment` | `production` \| `sandbox` — an iOS dev-build token fails against prod with a silent `BadDeviceToken` |
| `bundle_id` | `apns-topic`; also distinguishes the three apps |
| `app_slug` | `mobile` \| `music` \| `read-aloud` |
| `last_seen_at`, `failure_count`, `created_at` | pruning |
| column | note |
| --------------------------------------------- | ----------------------------------------------------------------------------------------------- |
| `id` | serial |
| `user_id` | fk users |
| `token` | APNs device token or FCM registration token |
| `platform` | `ios` \| `android` |
| `environment` | `production` \| `sandbox` — a dev-build token fails against prod with a silent `BadDeviceToken` |
| `bundle_id` | `apns-topic` |
| `app_slug` | see migration note |
| `last_seen_at`, `failure_count`, `created_at` | pruning |
Unique on `(token, bundle_id)`. Registration is idempotent: the app re-registers on every launch, since
tokens rotate.
## Failure handling — the part that is usually skipped
### Migration notes from the first design
- **`appSlug` on a NOTIFICATION inverts.** It meant "deliver only to this app". It now means "the deep-link
destination". Nothing consumes it yet, so this is a rename-in-place, not a data migration.
- **`bundle_id` / `app_slug` on a DEVICE go near-constant.** The columns exist to let several apps register
separately; with a hub there is one bundle id per device. Keep them — a second receiver later (a watch, a
desktop build) costs nothing this way — but "which apps has this user got" stops being a delivery question.
- **`NotifyType` is replaced by codes.** The `'job' | 'mail' | 'agent' | 'download' | 'test'` enum was the
rigidity this design exists to remove. `test` should survive as a reserved code for first light.
- **`text.ts` moves to the device.** Server-side `renderTitle`/`renderBody` becomes the mapping. Keep a
minimal server-side generic for the `alert` fallback.
## Failure handling
**APNs** answers inline: `410 Unregistered` or `400 BadDeviceToken` means delete the row immediately.
**FCM** likewise: `UNREGISTERED` / `INVALID_ARGUMENT` / `NOT_FOUND` means delete.
**FCM** answers inline too: `UNREGISTERED` / `INVALID_ARGUMENT` means delete.
Everything else increments `failure_count`; three strikes and the row goes. Without this the registry fills
with dead tokens and delivery quietly degrades.
Everything else increments `failure_count`; three strikes and the row goes. Without this the registry
fills with dead tokens and delivery quietly degrades.
## Android and GrapheneOS — UNRESOLVED, and it decides Android
(If Expo were used, this would instead require a deferred receipt poll ~15 min after send. Going direct
removes that entire mechanism — a genuine simplification, worth noting against the fan-out we gave up.)
`apps/mobile/PUSH_NOTIFICATIONS.md` (2026-07-24) rejects FCM outright: _"requires Google Play Services; the
Pixel runs GrapheneOS without it."_ The 2026-07-31 platform design chose FCM for Android a week later and
does not mention GrapheneOS. **Nobody reconciled the two, and this design does not either.**
## Open, deliberately deferred
It matters because FCM cannot reach a device with no Play Services, so on such a phone the entire Android
path is inert regardless of anything above.
- **Which events notify.** Out of scope by instruction: build the pipe first. Candidates when we get
there: job finished, new mail, long agent turn done, download complete.
- **Quiet hours and batching.** New mail on an 18k-mail account is unusable at one push per message.
- **`apps/mobile` EAS/bundle id** — only `apps/music` is confirmed to have one (`dev.officer.music`).
The recorded fallback is a **foreground service holding a tailnet socket**, with notifications raised
locally. It was rejected in the first design for a permanent "Officer is running" entry in the shade and
unreliability against OEM battery managers. **The hub changes that calculus**: one socket and one shade
entry for the whole platform, not one per app. The battery-manager objection is also weaker on GrapheneOS
than on Samsung or Xiaomi.
This needs an owner decision before Android work starts. iOS is unaffected and can proceed independently.
## Build order
1. ~~`push_devices` table + `bun db:push`.~~ done
2. ~~`officer-notify` sidecar shell: listener, PM2 entry, `/api/notify` proxy, Discord channel moved in.~~ done
3. ~~APNs channel + `POST /_officer/devices` registration.~~ done (27 tests)
4. ~~FCM channel.~~ done (27 tests)
5. First producer wired. **← next, and nothing has been notified by anything yet.**
1. ~~`push_devices` table.~~ done
2. ~~`officer-notify` sidecar, PM2 entry, `/api/notify` proxy, Discord channel.~~ done
3. ~~APNs channel + device registration endpoint.~~ done, 27 tests
4. ~~FCM channel.~~ done, 27 tests
5. **APNs credentials.** Owner action; nothing can be tested end-to-end without them.
6. **Hub app registers its token** against `POST /api/notify/_officer/devices`. Small.
7. **First light**: a `test` push rendered by the OS with plain text, no codes, no extension. Proves
credential → APNs → device → banner as one thing, before any of the mapping exists.
8. **Codes and the mapping**: `NotifyType` → codes, `GET /api/notify/mapping`, plugin catalogues on the
affordances pattern.
9. **The Notification Service Extension**, local lookup, App Group container.
10. **First real producer** — job finished is the obvious one.
Ahead of 5, two things are unresolved and neither is code: the Android/GrapheneOS contradiction above,
and whether a member can register a device (the `notify` permission is `kind: 'app'`, so it is grantable
per role and denied by default — a member with no grant cannot subscribe their own phone).
Steps 7 and 8 are deliberately separable: the transport can be proven working before the rendering exists,
and if it is not, no amount of mapping work will help.
## Still open
- **Which events notify**, beyond the first producer. Quiet hours and batching — new mail on an 18k-mail
account is unusable at one push per message.
- **Whether members can register devices.** The `notify` permission is `kind: 'app'`, so it is grantable per
role and denied by default; a member with no grant cannot subscribe their own phone.
- **Android**, per above.