push notifications: design, and the device registry

Design in docs/push-notifications.md. The short version: Apple and Google are unavoidable —
iOS suspends apps so only APNs can wake one, and Android only accepts pushes from FCM — but
Expo is not. Its push service is a relay in front of both and does not remove either
credential, so we talk to Apple and Google ourselves.

Both protocols were verified in Bun before designing around them: node:http2 works as a
client (APNs is HTTP/2-only), and ES256 signing produces the raw 64-byte r||s form Apple
requires rather than Node's default DER, which is silently rejected. No push library is
needed for either channel.

The load-bearing decision is the payload: a push is a doorbell, not a message. Apple and
Google see metadata regardless, so they must not also see content — a notification carries a
category and an id, never a subject, sender or error, and the app composes the visible text
locally and fetches the real thing over the tailnet on tap.

This commit is the registry: push_devices, holding native APNs/FCM tokens. environment is a
column because APNs sandbox and production are different hosts AND different token
namespaces — a debug-build token fails against production with a silent BadDeviceToken, so
guessing is not an option. Registration upserts on (token, bundle_id) because tokens rotate
and the app re-registers every launch. Failure counting prunes dead tokens; a hard rejection
deletes at once.

Nothing sensitive lands here: a token is useless without the APNs key or FCM service
account, both of which stay in the sidecar's env.

NOTE: `bun db:push` will fail until the telegram/whatsapp/discord rows are deleted from
server_integrations — the CHECK constraint added earlier refuses while they exist. That is
the enforcement working, not a problem to route around.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-31 21:27:07 +00:00
co-authored by Claude Opus 5
parent cffb99b9d0
commit 15f262539f
6 changed files with 284 additions and 8 deletions
+133
View File
@@ -0,0 +1,133 @@
# Push notifications — design
**Status:** agreed design, 2026-07-31. Implementation starting. Server side; the app side is a separate
document handed to the apps repo (`officer-suite/COMMS/PUSH_NOTIFICATIONS_APP.md`).
## What was decided, and what was rejected
**Apple and Google are unavoidable and we accept them.** There is no direct server→phone push on either
platform: iOS suspends apps, so only APNs can wake one, and Android only accepts pushes from FCM. An
earlier plan had these the other way round — believing iOS could be pushed to directly and Android needed
a bridge. It is the reverse.
**Expo is rejected.** Its push service is a relay in front of APNs and FCM, and it does not remove either
credential — a standalone Android build still needs an FCM service account, which you would upload *to
Expo*. All it adds is fan-out convenience, in exchange for our payloads and our APNs key living on a
fourth party's servers. We talk to Apple and Google ourselves.
**The Android foreground-service alternative is rejected for now.** A persistent socket plus a local
notification genuinely avoids Google, and the tailnet makes the connection easy. It costs a permanent
"Officer is running" entry in the notification shade and is unreliable against OEM battery managers
(Samsung, Xiaomi). Recorded here because it remains the fallback if FCM ever becomes unacceptable.
## Payload policy — the notification is a doorbell, not a message
**This is the load-bearing decision, and it is not optional.**
Both services see metadata regardless: which device, how often, at what times. What they must not see is
content. So a push carries a category and an id, never the substance:
```
✅ { type: 'mail', count: 3 } ❌ { title: 'Re: invoice', from: 'x@y.com' }
✅ { type: 'job', id: 412, ok: false } ❌ { error: 'ENOENT /home/pastilhas/…' }
```
The app already has a tailnet connection. On tap it fetches the real content itself. The visible text is
generic ("3 new emails"), assembled on the device from the category, not sent through Apple or Google.
Cost of both services: **free**. No per-message charge on either.
## Where it lives
A new **`officer-notify` sidecar**, PM2 peer, its own loopback HTTP listener, announced as
`notify:server` and proxied at `/api/notify` by `createSidecarProxy`.
Not the platform, because the producers are spread out — the queue, the email sidecar, the agent sidecar —
and a platform-owned notifier would force every sidecar to call *back* into the platform. That is the
inversion just removed from email. As a sidecar, anything POSTs to it over loopback.
It also absorbs `src/servers/notify/discord.ts`, so there is one outbound-notification surface with
channels behind it rather than a Discord path and a push path that do not know about each other.
```
producer (queue / email / agent / platform)
│ POST /_officer/notify { type, ... }
officer-notify ─┬─ apns → api.push.apple.com (HTTP/2, ES256 JWT)
├─ fcm → fcm.googleapis.com/v1 (OAuth2 bearer)
└─ discord→ webhook (existing)
```
## Credentials
Env only, on the sidecar, never in the platform process and never in the database.
| var | what |
|---|---|
| `APNS_KEY_P8` | contents of the `.p8` auth key (PKCS#8 EC P-256) |
| `APNS_KEY_ID` | the key's 10-char Key ID |
| `APNS_TEAM_ID` | Apple Developer Team ID |
| `APNS_ENV` | `production` or `sandbox` — different hosts AND different tokens |
| `FCM_SERVICE_ACCOUNT` | the service-account JSON |
| `DISCORD_WEBHOOK_URL` | existing, moves here |
The `.p8` does not expire and can push to your apps forever. Treat it as spending-grade.
## Both protocols, verified in Bun before designing around them
- **APNs**: HTTP/2 only. `node:http2` works in Bun 1.3.10 (checked against a live server). Auth is an
ES256 JWT signed with the `.p8`, valid ≤1h, refreshed no more often than every 20 min or Apple rejects
it. **The signature must be raw `r||s` (64 bytes)**`createSign(...).sign({ key, dsaEncoding:
'ieee-p1363' })`. Node's default DER encoding is silently rejected.
- **FCM v1**: ordinary HTTPS. Sign an RS256 JWT with the service-account key, exchange it at
`oauth2.googleapis.com/token` for a 1-hour access token, cache that, then POST to
`fcm.googleapis.com/v1/projects/<id>/messages:send`.
No push library is needed for either — `node:crypto` and `node:http2` cover it. All three endpoints are
reachable from this host.
## Device registry
New table `push_devices`:
| column | note |
|---|---|
| `id` | serial |
| `user_id` | fk users |
| `token` | the native token — APNs device token or FCM registration token |
| `platform` | `ios` \| `android` |
| `environment` | `production` \| `sandbox` — an iOS dev-build token fails against prod with a silent `BadDeviceToken` |
| `bundle_id` | `apns-topic`; also distinguishes the three apps |
| `app_slug` | `mobile` \| `music` \| `read-aloud` |
| `last_seen_at`, `failure_count`, `created_at` | pruning |
Unique on `(token, bundle_id)`. Registration is idempotent: the app re-registers on every launch, since
tokens rotate.
## Failure handling — the part that is usually skipped
**APNs** answers inline: `410 Unregistered` or `400 BadDeviceToken` means delete the row immediately.
**FCM** answers inline too: `UNREGISTERED` / `INVALID_ARGUMENT` means delete.
Everything else increments `failure_count`; three strikes and the row goes. Without this the registry
fills with dead tokens and delivery quietly degrades.
(If Expo were used, this would instead require a deferred receipt poll ~15 min after send. Going direct
removes that entire mechanism — a genuine simplification, worth noting against the fan-out we gave up.)
## Open, deliberately deferred
- **Which events notify.** Out of scope by instruction: build the pipe first. Candidates when we get
there: job finished, new mail, long agent turn done, download complete.
- **Quiet hours and batching.** New mail on an 18k-mail account is unusable at one push per message.
- **`apps/mobile` EAS/bundle id** — only `apps/music` is confirmed to have one (`dev.officer.music`).
## Build order
1. `push_devices` table + `bun db:push`.
2. `officer-notify` sidecar shell: listener, `notify:server`, PM2 entry, `/api/notify` proxy. Discord
channel moved in — one real channel end-to-end before any push credential exists.
3. APNs channel + `POST /_officer/devices` registration.
4. FCM channel.
5. First producer wired.
+9
View File
@@ -181,3 +181,12 @@ export type { WalletChainSnapshot } from './schema/wallet';
export { db } from './db'; export { db } from './db';
export * as schema from './schema'; export * as schema from './schema';
export {
upsertPushDevice,
getPushDevices,
deletePushDevice,
recordPushFailure,
markPushDeviceSeen,
} from './queries/notify';
export type { PushDeviceSelect, PushDeviceInsert } from './types';
@@ -0,0 +1,64 @@
import { eq, and, sql } from 'drizzle-orm';
import { db } from '../db';
import { pushDevices } from '../schema';
import type { PushDeviceSelect, PushDeviceInsert } from '../types';
// The push device registry. Only the officer-notify sidecar uses these.
/**
* Register a device, or refresh one already known. The app calls this on EVERY launch — push tokens
* rotate silently, and a stale row is a device that stops receiving with no error anywhere — so this is
* an upsert on (token, bundleId) rather than an insert, and it clears any accumulated failures.
*/
export async function upsertPushDevice(data: PushDeviceInsert): Promise<PushDeviceSelect> {
const [row] = await db
.insert(pushDevices)
.values(data)
.onConflictDoUpdate({
target: [pushDevices.token, pushDevices.bundleId],
set: {
userId: data.userId,
platform: data.platform,
environment: data.environment ?? 'production',
appSlug: data.appSlug,
failureCount: 0,
lastSeenAt: new Date(),
},
})
.returning();
return row!;
}
/** Every device for a user, optionally narrowed to one app. */
export async function getPushDevices(userId: number, appSlug?: string): Promise<PushDeviceSelect[]> {
const where = appSlug
? and(eq(pushDevices.userId, userId), eq(pushDevices.appSlug, appSlug))
: eq(pushDevices.userId, userId);
return db.select().from(pushDevices).where(where);
}
/** Sign-out, or a hard rejection from Apple/Google (410 Unregistered, UNREGISTERED). */
export async function deletePushDevice(token: string): Promise<void> {
await db.delete(pushDevices).where(eq(pushDevices.token, token));
}
/**
* A soft failure — a timeout, a 5xx. Three consecutive strikes and the row goes: without pruning, the
* registry fills with dead tokens and delivery quietly degrades. Any successful send resets the count
* via `upsertPushDevice`, so only *consecutive* failures accumulate.
*/
export async function recordPushFailure(token: string): Promise<void> {
await db
.update(pushDevices)
.set({ failureCount: sql`${pushDevices.failureCount} + 1` })
.where(eq(pushDevices.token, token));
await db.delete(pushDevices).where(and(eq(pushDevices.token, token), sql`${pushDevices.failureCount} >= 3`));
}
/** A send worked: clear the failure count and mark the device alive. */
export async function markPushDeviceSeen(token: string): Promise<void> {
await db
.update(pushDevices)
.set({ failureCount: 0, lastSeenAt: new Date() })
.where(eq(pushDevices.token, token));
}
+9 -8
View File
@@ -1,13 +1,14 @@
export * from './auth'; export * from './auth';
export * from './user-data';
export * from './dashboards';
export * from './operations';
export * from './server';
export * from './email';
export * from './pipeline-jobs';
export * from './chat-events'; export * from './chat-events';
export * from './music'; export * from './dashboards';
export * from './soulseek'; export * from './email';
export * from './headscale'; export * from './headscale';
export * from './music';
export * from './notify';
export * from './operations';
export * from './pipeline-jobs';
export * from './server';
export * from './soulseek';
export * from './user-data';
export * from './vault'; export * from './vault';
export * from './wallet'; export * from './wallet';
@@ -0,0 +1,65 @@
import { pgTable, serial, integer, text, timestamp, unique, index, check } from 'drizzle-orm/pg-core';
import { sql } from 'drizzle-orm';
import { users } from './auth';
// Devices that can receive a push, for the officer-notify sidecar.
//
// A token here is a NATIVE one — an APNs device token or an FCM registration token — because we send to
// Apple and Google directly rather than through Expo. See docs/push-notifications.md.
//
// Nothing sensitive is stored: a push token identifies a device's ability to receive, not the device or
// its owner, and it is useless without our APNs key or FCM service account. Those live in the sidecar's
// env and never touch this database. Notification CONTENT is never stored here either — payloads are a
// category and an id, assembled at send time and never retained.
//
// Only the officer-notify sidecar reads or writes this table.
const PLATFORMS = ['ios', 'android'] as const;
// APNs sandbox and production are different hosts AND different token namespaces: a token minted by a
// debug build fails against production with a silent BadDeviceToken. Android has no such split, but the
// column is not nullable — an android row is simply always 'production'.
const ENVIRONMENTS = ['production', 'sandbox'] as const;
export const pushDevices = pgTable(
'push_devices',
{
id: serial('id').primaryKey(),
userId: integer('user_id')
.notNull()
.references(() => users.id, { onDelete: 'cascade' }),
token: text('token').notNull(),
platform: text('platform').notNull(),
environment: text('environment').notNull().default('production'),
/** The `apns-topic`, and what distinguishes the three apps from each other. */
bundleId: text('bundle_id').notNull(),
/** `mobile` | `music` | `read-aloud` — for targeting a notification at one app. */
appSlug: text('app_slug').notNull(),
/** Consecutive send failures. Three strikes and the row is deleted; a hard rejection deletes at once. */
failureCount: integer('failure_count').notNull().default(0),
lastSeenAt: timestamp('last_seen_at', { withTimezone: true }).notNull().defaultNow(),
createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
},
(table) => [
// One row per token per app. The app re-registers on every launch because tokens rotate silently, so
// registration is an upsert on this pair rather than an insert.
unique('uq_push_devices_token_bundle').on(table.token, table.bundleId),
index('idx_push_devices_user').on(table.userId),
// The schema is the source of truth for what may exist, not just its shape — see
// src/databases/CLAUDE.md. sql.raw because an interpolated string binds as a parameter, which
// Postgres refuses inside a CHECK and which breaks push for the whole schema.
check(
'ck_push_devices_platform',
sql`${table.platform} IN (${sql.join(
PLATFORMS.map((p) => sql.raw(`'${p}'`)),
sql`, `,
)})`,
),
check(
'ck_push_devices_environment',
sql`${table.environment} IN (${sql.join(
ENVIRONMENTS.map((e) => sql.raw(`'${e}'`)),
sql`, `,
)})`,
),
],
);
+4
View File
@@ -73,3 +73,7 @@ export type ServerIntegrationInsert = typeof Schema.serverIntegrations.$inferIns
export type PipelineJobSelect = typeof Schema.pipelineJobs.$inferSelect; export type PipelineJobSelect = typeof Schema.pipelineJobs.$inferSelect;
export type PipelineJobInsert = typeof Schema.pipelineJobs.$inferInsert; export type PipelineJobInsert = typeof Schema.pipelineJobs.$inferInsert;
// Notifications
export type PushDeviceSelect = typeof Schema.pushDevices.$inferSelect;
export type PushDeviceInsert = typeof Schema.pushDevices.$inferInsert;