diff --git a/docs/file-sync.md b/docs/file-sync.md new file mode 100644 index 00000000..50aa33ee --- /dev/null +++ b/docs/file-sync.md @@ -0,0 +1,128 @@ +# File sync + +Written 2026-08-04. The "final boss" of the NextCloud replacement — separated from +`nextcloud-replacement.md` because the answer is genuinely different from calendar and contacts, and +conflating them is how this becomes a six-month project. + +The ask: _"the synced files functionality that I can run on each computer and phone"_, with the same +acceptance test as the rest — **it has to just work.** + +--- + +## The distinction that decides everything + +**A mount is not a sync.** + +| | Mount (WebDAV, SMB, sshfs) | Sync (NextCloud client, Syncthing, Dropbox) | +| ---------------- | -------------------------- | ------------------------------------------- | +| Offline | Nothing works | Everything works | +| Open a 2 GB file | Streams over the network | Local disk read | +| Flaky connection | Failed saves, hung apps | Queues, retries | +| Conflicts | Last writer wins, silently | Detected and surfaced | +| Cost to build | Days | The rest of the year | + +What NextCloud's desktop client actually gives you is a **local-first replica**: a full copy on every +device, a local database of file state, change detection via filesystem watching, transfer scheduling, +and explicit conflict handling. That is the thing being asked for, and it is a genuinely hard, +well-studied distributed-systems problem. + +--- + +## Decision: run Syncthing, do not write a sync engine + +Same reasoning as choosing Radicale over writing CalDAV, only more so. + +| Option | Verdict | +| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Syncthing** | **Chosen.** Mature (since 2013), open source, peer-to-peer, real clients on Linux/macOS/Windows/Android, explicit conflict files, block-level transfer, no server account model to build. | +| WebDAV from the caldav sidecar | Radicale is calendars and contacts only, so this would be a second server anyway — and it solves "see my files remotely", not "have my files offline". **Complementary, not a substitute.** Cheap to add later. | +| Reimplement NextCloud's sync protocol | Chunked upload, ETag bookkeeping, a local state DB, conflict resolution, and a client on three platforms. The single largest item on the whole list, to arrive at something worse than what exists. | + +### Why Syncthing fits this platform specifically + +It is peer-to-peer, so there is **no server-side account model to build** — devices authenticate each +other by device ID. That removes the entire class of work that a Dropbox-alike would need. Officer's +job shrinks to: run it, own its config, show its state, and point the file browser at the folder. + +It also composes with what is already here: the machine already runs Headscale, so devices are already +on a private network and Syncthing's discovery/relay layer can be turned down or off entirely. + +--- + +## Shape of the work + +### `officer-syncthing` sidecar + +Same pattern as `officer-caldav` (supervise a real server) crossed with `officer-memos` (hold a +credential, proxy an API). + +- Supervise the `syncthing` binary on a **loopback** port, like Radicale. +- Generate its config at boot rather than hand-maintaining it — same reasoning as + `caldav/radicale.ts`: config is derived state, and a hand-edit that disagrees with what the code + believes is very hard to debug. +- Syncthing's own REST API is authenticated with an API key it writes into its config; the sidecar + reads that key and holds it. The platform never sees it. +- Storage under `DATA_PATH/sync/`, so it is inside the existing backup story. + +### Platform side + +Sixteen lines of `createSidecarProxy` at `/api/sync`, exactly like memos and transmission. No sync +knowledge in the platform, ever. + +### UI — replicate Syncthing's own UI first + +Per the standing rule: reproduce what the service already ships, then extend. Syncthing's own web UI +covers folders, devices, sync status/percentage, and conflicts. Those are the four panels. The +extension Officer can add that Syncthing cannot is the interesting part: **the synced tree is just a +directory**, so the existing file browser, code editor and file viewer all work on it for free. That is +the integration nothing else offers. + +### Build order + +1. Sidecar supervising syncthing + config generation + registration. +2. `/api/sync` proxy. +3. Status panel (folders, devices, completion) — read-only. +4. Add/remove folder and device pairing from the UI. +5. Surface the synced root in the file browser as a first-class location. + +--- + +## The honest problem: iOS + +**There is no good Syncthing on iOS, and there will not be one.** iOS does not allow a +general-purpose background daemon doing continuous filesystem work; the only always-on sync clients +that exist there are ones Apple grants entitlements for. This is also why NextCloud's own iOS app is a +manual, foreground, tap-to-upload experience rather than a real sync client — the platform, not the +software, is the limit. + +So "on each computer and phone" splits: + +- **Linux/macOS/Windows** — solved by Syncthing, properly, today. +- **Android** — solved by Syncthing (there are maintained forks; verify which is current before + recommending one). +- **iOS** — _not_ solvable as continuous sync. The realistic shape is a **WebDAV mount for browsing and + on-demand download, plus explicit upload from the Officer mobile app**. That is a different feature + with a different backend, and it is a decision for the owner rather than something to assume. + +This is worth deciding before step 1, because "iOS matters" changes the answer: if it does, a WebDAV +server becomes a first-class part of the design rather than an optional later addition, and it may be +worth serving both from one place. + +--- + +## What this does NOT do + +- **It is not a replacement for the file browser.** Officer already has one, and it already reaches the + whole filesystem. Syncthing adds replication, not access. +- **It is not backup.** Sync propagates deletions. A synced folder is not a backup of itself, and + anyone who believes otherwise finds out at the worst moment. Versioning (Syncthing has several + strategies) should be enabled and surfaced in the UI precisely so this is not confused. +- **It is not sharing.** Single-user remains a hard platform invariant. + +--- + +## Status + +Design only. Nothing built — this was written before implementation on the night of 2026-08-04 and the +implementation ran out of night. The two things to settle before writing code are **the iOS question +above** and whether the synced root should live under `DATA_PATH` or somewhere the owner picks. diff --git a/docs/mobile-photo-sync-api.md b/docs/mobile-photo-sync-api.md new file mode 100644 index 00000000..70ffe423 --- /dev/null +++ b/docs/mobile-photo-sync-api.md @@ -0,0 +1,167 @@ +# Phone photo auto-backup — API contract + +Written 2026-08-04, for the mobile developer. This is the server-side contract for "photos taken on the +phone appear on the server automatically", i.e. what the Immich app does today. + +**The mobile app is not built here.** This document is the boundary: everything below is either already +live or explicitly listed as missing, so the app can be built against it rather than reverse-engineered +from the web client. + +--- + +## The short version + +**Almost all of this already exists.** Officer proxies a self-hosted Immich through the `officer-photos` +sidecar, and the sidecar's route allow-list already permits the `assets` resource for `GET POST PUT +DELETE` (`src/servers/sidecar/photos/routes.ts:22`). Immich's own upload and dedupe endpoints are +therefore already reachable through Officer, with Officer's auth in front and the Immich API key never +leaving the sidecar. + +So the work is not "build a photo ingest service". It is: use the endpoints below, and close the three +gaps at the bottom. + +--- + +## Topology + +``` + phone app officer officer-photos Immich + ───────── ─────── ────────────── ────── + Authorization: → userMiddleware (JWT) → allow-list by first → x-api-key + Bearer injects X-Officer-User method in the sidecar) +``` + +The phone holds a **platform JWT** — the same credential `monorepo-mobile/packages/core/src/services/ +api.ts:58` already sends as `Authorization: Bearer `. It never holds an Immich key and never +talks to Immich directly. + +Base path for everything below: **`/api/photos/_officer/…`**, which maps to `/api/…`. +So `/api/photos/_officer/assets` is Immich's `POST /api/assets`. + +--- + +## Endpoints + +### 1. Is the library reachable at all + +``` +GET /api/photos/_health +→ 200 { ok: true, configured: true, account: "