platform/docs/sidecar-app-store.md

# Sidecars as installable apps

**Status: DESIGN, agreed in conversation 2026-08-10. Nothing implemented.** This supersedes the framing
of `sidecar-bootstrapping.md`, which stays as the record of how the mechanics work _today_.

The goal: a clean machine runs chat, the terminal and the file browser, and **everything else arrives by
the user asking for it** — from an app store inside Officer. Eventually including sidecars the user did
not write.

---

## Why this is mostly not a rewrite

Three things are already true, which is why "nothing exactly blocks it":

- **Every API route stays mounted regardless of which sidecars run.** The light profile's own comment
  states it: features whose sidecars are absent report themselves unavailable rather than disappearing.
  So the app store never needs to mount or unmount routes.
- **Officer already spawns nothing.** Sidecars are PM2 peers that dial in and register by capability.
  Installing one is starting a process, not teaching officer about it.
- **`service_connections` already solves the multi-user case**, including the part nobody would get
  right independently — see below.

What is genuinely new: provisioning containers, per-sidecar schema, and persisted install state.

---

## Light becomes the baseline

`ecosystem.light.config.cjs` stops being a variant and becomes what a fresh install runs:

```
officer  ·  officer-anthropic-proxy  ·  officer-agent  ·  officer-opencode  ·  officer-pty  ·  officer-gitea
```

Chat, terminal, file browser. The file-browsing APIs live in the main process, so they cost nothing
extra.

The other fourteen become app-store entries.

---

## Three install shapes

The prompt the user sees depends on which of these the sidecar is. This is the taxonomy the installer
branches on:

| Shape                                     | What install means                                                                                                              | Examples                                                                                                  |
| ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| **Point at an instance you already have** | Ask for URL + credential, write `service_connections`, start the sidecar                                                        | gitea, memos, photos (Immich), jellyfin, invoiceshelf, headscale                                          |
| **Provision one**                         | Render our compose template, `docker compose up -d`, wait for health, write the connection _we already know_, start the sidecar | vault (Vaultwarden), slskd, transmission, caldav (Radicale), and any of the above where the user has none |
| **Configuration only**                    | Ask for credentials, start the sidecar. No service to reach                                                                     | email (IMAP), notify, music, wallet, vnc                                                                  |

A sidecar can be more than one: Gitea is "existing instance" for someone who runs one and "provision"
for someone who does not. The prompt is the fork.

---

## Docker: we install, the user owns

**Officer is the installer, never the owner.** Concretely:

- A real compose file per service, written into **`<root>/dockers/<id>/`**, from our template — using
  the convention the owner already applies to 47 services: one directory per service,
  `docker-compose.yaml` inside, and **relative bind mounts** (`./data`, `./database`, `./storage`) so
  configuration and data sit beside the compose file where both we and a human can see them. Named
  volumes are used by 3 of those 47 and are the exception; templates use bind mounts, always.
- Started with `docker compose up -d` **as the owner**, not as officer's own identity.
- Found again by **label** (`officer.sidecar=<id>`), not by holding a handle.

Consequences, which are the point:

- `docker ps`, `docker logs`, `docker compose down` all behave normally.
- If Officer is removed, the containers keep running and stay manageable.
- A user who runs his own estate can edit the compose file — it is his file, in his directory.
- We can always find what we installed without pretending to own it.

The template is what makes this non-technical-user-friendly: sensible defaults, ports, volumes and
health checks already correct, so "install Gitea" does not become a tutorial.

**`USER_UID` / `USER_GID` are set to the owner**, as the existing services already do. That answers the
"do containers run as root" question: no, and this is not a new convention — it is the one in use.

**The app store's containers are isolated from the user's own**, and that is the point of the layout:

```
~/officerdev/
  platform/      the app
  data/          DATA_PATH
  dockers/       services the app store provisioned   <- exclusively ours
  capabilities/  the file-based item store
```

`OFFICER_ROOT` is derived as the parent of `DATA_PATH` rather than configured separately — a second
variable that must agree with the first is a second thing to get wrong.

Deliberately **not** `~/dockers`, which is where a seasoned user already keeps their estate. Two
consequences, both wanted:

1. Containers the app store created are distinguishable from the user's own **structurally**, not by a
   naming convention we would have to enforce and they could break.
2. **We never reason about someone else's compose files.** The store does not scan, adopt or modify
   anything outside its own directory. "I already have one of these" is answered by the user giving a
   URL (`mode: 'existing'`) — never by us finding a directory and guessing whose it is.

`[open]` Podman, for anyone wanting genuinely rootless.

### Docker is assumed, and nothing guarantees it

Verified: **nothing in `scripts/` installs Docker, and nothing checks for it.** The host installer
`scripts/setup/setup.sh` calls `scripts/setup/setup-dockers.sh`, which invokes `docker compose` with no
preflight, so a fresh host without Docker fails partway through setup with a bare "command not found".

(Not to be confused with the per-template `setup.sh` below — `app-store/templates/<name>/setup.sh` — which
is a different file with a different contract. The host one provisions the machine; a template one
provisions a single sidecar.)

That is the seam where this project's origin shows — it began as one person's own machine, provisioned
by his own scripts, where Docker was simply always there.

The intended fix is **a `setup.sh` per sidecar**, ensuring its own dependencies before its compose file
is used. That is also the shape a sidecar needs once it lives in its own repository, so a sidecar package
becomes:

```
metadata (catalogue entry)  ·  compose template  ·  setup.sh  ·  schema
```

Until that exists, the app store **detects and reports** rather than guessing or half-installing:
`preflight.ts` checks `docker compose version` — which exercises the binary, the daemon connection and
the plugin in one call, unlike `docker --version`, which passes with a dead daemon — and distinguishes
"not installed" from "daemon unreachable", because the remedies differ.

The check is **per mode, not per entry**: a host without Docker can still install Photos by pointing at
an Immich elsewhere. Refusing the whole entry would be the over-strict check that makes people work
around the installer instead of using it.

---

## Install state

Two independent flags, because they answer different questions:

- **`installed`** — the thing exists: container provisioned, config written, schema applied.
- **`enabled`** — the process should be running.

| Action        | Sidecar process | Container                      | Data & schema |
| ------------- | --------------- | ------------------------------ | ------------- |
| **Disable**   | stopped         | stopped                        | untouched     |
| **Enable**    | started         | started                        | untouched     |
| **Uninstall** | stopped         | `docker compose down`, removed | untouched     |

Disable stops the container too — there is no reason to leave Immich holding memory while Photos is
switched off. For `mode: 'existing'` there is no container of ours, so disable is only the sidecar.

Uninstall additionally deletes the `sidecar_installs` row. It does **not** drop the sidecar's tables.

**Nothing above deletes data, and there is no option that does.**

### Why the schema survives uninstall too

Dropping a sidecar's tables is deleting data. Not media, but real: music favourites, the Jellyfin server
registry, photos configuration, saved connections. That is the same category as volumes and gets the
same answer.

It also buys something. **Reinstall becomes restore** — uninstall Photos in June, reinstall in August,
and the configuration and favourites are still there. Drop the schema and reinstalling hands back a
blank service that looks subtly broken to someone who remembers setting it up.

Keeping them costs nothing: an unused table is a row in `information_schema`. No queries, no memory, no
maintenance. Dropping them joins volume deletion in the later, deliberate cleanup feature, where the
user sees what they are removing.

**Install must be idempotent and resumable.** Provision → health → config → schema → start is five steps
and any of them can fail. The failure mode to design against is a half-installed service that neither
works nor uninstalls. Each step records what it did; re-running install resumes rather than restarts.

---

## Per-sidecar schema

Today all 42 tables live in one Drizzle schema and arrive together via `bun db:push`. That changes:
**each sidecar owns its own schema and applies it on install.**

This is right _because third-party plugins are a real goal_. For our own fourteen it would be
over-engineering — an unused table costs nothing — but a marketplace plugin cannot ship a table into a
schema it does not own.

**The dependency graph makes this tractable.** Measured across the 19 non-core schema files:

```
core:     auth.ts, server.ts, chat-events.ts        (depend on nothing)
sidecars: every single one -> auth.ts, and nothing else
```

There is **no sidecar-to-sidecar dependency anywhere**. One file (`user-data.ts`) touches two, and it is
core. So the contract for a plugin's schema is nearly the smallest it could be:

> **You may reference `users.id`. You may not reference anything else.**

Which also makes full uninstall well-defined: drop the tables this sidecar declared. Nothing else points
at them, by construction.

`[open]` Where do a plugin's migrations live, and what applies them — the installer, or the sidecar on
first boot? Versioning and upgrade are unsolved here.

---

## `service_connections` is part of the contract

Decided: it stays **core and shared**, one table, with each plugin identified by its own ID — rather than
a connections table per service.

It already does the hard part. The row is keyed `(userId, service)` and **a NULL `url` means "inherit the
instance"**: the owner's row carries the URL and _is_ the instance; every other user's row carries only
their own credential and resolves the base from the owner's row at read time.

So "members never see the instance URL" is a property of the schema rather than a filter someone must
remember on every response — and a member cannot supply a URL, which closes what would otherwise be a
per-user SSRF hop wearing a settings form. Gitea is the first service of this kind; five sidecars use the
table today (memos, wallet, transmission, slskd, gitea).

A third-party plugin inherits all of that for free, which is the argument for sharing the table: it is
the part nobody would get right independently.

Two things it needs before third parties touch it:

1. **Namespaced IDs.** `service` is free text — deliberately, so adding a service is not a schema change.
   With a marketplace, two plugins could both claim `"gitea"` and collide on the unique index. Needs a
   convention (reverse-DNS, or IDs issued by the marketplace).
2. **Somewhere for plugin-specific config.** The columns are shaped around the services that exist:
   `url`, `username`, `secret`, `path`, `version`. A plugin needing anything else has nowhere to put it,
   and adding a column per plugin defeats the shared table. Likely a `config` JSONB for the remainder —
   with `url` staying first-class, because the inheritance rule above depends on it being a real column.

---

## The API contract, when we open this up

What a plugin author is promised, and bound by. To be written properly; the shape is:

- **Register** by name + capabilities over `/api/sidecar/register`; be reachable by capability.
- **Declare** an ID, an install shape, a compose template (if it provisions), a config prompt, and a
  schema.
- **May reference** `users.id`, and use `service_connections` under its own ID.
- **May not** reference another plugin's tables, or write outside its own.
- **Must** tolerate being disabled, re-enabled, and uninstalled.

---

## Provisioning has three shapes, not one

This document originally said provisioning "writes the connection we already know". That is only true
some of the time, and the difference decides whether an install can finish unattended:

1. **We set the credentials.** Passed as container environment, so the connection is known the moment it
   is up. Transmission (`USER`/`PASS`), Vaultwarden (`ADMIN_TOKEN`).
2. **We generate a secret into a file.** The bind mount lets us write it before first boot, so it is
   still known without asking. slskd's API key lives in its `slskd.yml`.
3. **A human must mint a token in the service's own UI after it boots.** Immich, Jellyfin and Memos all
   work this way — no environment variable pre-seeds an API key.

Shape 3 means an install can be **provisioned and running but not yet connected**. That is a real state,
not a failure: the container is up, the compose file is written, and we are waiting for a token. The
step machine stops there, and the UI asks for the key with a link to the page that mints it. Resuming
finishes the job — which is what `completedSteps` was for.

---

## Members get their own accounts

The owner installs, but a server may already have members — and a member added next month needs the same
work done. So the unit is **(service × member)**, reachable from two triggers:

```
install a service  ->  provision every member who already exists
add a member       ->  provision every service already installed
```

Only handling the first is the classic thing that works on day one and rots quietly. There is no new
table: a member is provisioned for a service exactly when they hold a `service_connections` row for it —
their own credential, `url` NULL, inheriting the instance from the owner's. That schema was built for
this before this existed.

Three outcomes, declared per catalogue entry as `members`, so the installer never special-cases a
service:

|            | Meaning                                                                                                      | Services                                                          |
| ---------- | ------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------- |
| `accounts` | Admin API creates the user **and** mints a credential. Fully transparent — the member just finds it working. | Immich, Jellyfin, Memos, InvoiceShelf, CalDAV                     |
| `invite`   | The account can be created; a usable credential cannot. The member sets their own password.                  | Vaultwarden                                                       |
| `none`     | Single-tenant daemon, no user concept. Access is mediated by Officer alone.                                  | Transmission, slskd, headscale, email, music, wallet, notify, vnc |

**`invite` is not a weaker `accounts`** — it is the correct outcome. Vaultwarden derives its encryption
key from the master password, so a credential we could mint would mean a vault we could read. Transparent
right up to the point where being transparent would be a defect.

The per-service work is an **interface implemented beside each sidecar**, never a switch in core: a
central function growing one case per service is exactly what would stop any of this shipping from its
own repository. Implementations must be idempotent — both triggers can fire for the same pair, and
creating a second account upstream is not something we can undo.

Deprovision is deliberately optional and defaults to doing nothing upstream. Deleting a user in Immich
deletes their photos; an app store that destroys data as a side effect of an unrelated action is worse
than one that leaves a stale account behind.

**Assumed working:** the vault's own multi-user adaptation is being done separately. Today `/api/vault`
is owner-only by an explicit `ownerGate`, so a member is refused before Vaultwarden is reached — this
design is written as though that has landed.

---

## What Phase 0 must not foreclose

Three things are coming, and each one constrains a decision that looks free today.

**1. `marketplace.officer.dev`.** Phase 1 keeps the catalogue inside this repo; later the app lists what
is on a remote marketplace instead. So catalogue entries must stay **serialisable data** — no functions,
no imports, nothing that only means something at compile time. They are plain objects today and must
remain so, because the same shape has to arrive as JSON over HTTP. Compose templates travel with them.

**2. Every sidecar becomes its own repository.** Today `catalogue.test.ts` asserts the catalogue equals
"everything in ecosystem.config.cjs that light excludes". That is the right check _now_, and it inverts
later: once sidecars live elsewhere, the catalogue entry becomes the source of truth for how to run one
(command, args, env) and the ecosystem file is generated from what is installed, not the other way
round. **Do not treat that test as a permanent law** — it pins Phase 0's invariant, not the design's.

**3. Third-party plugins.** Already the reason per-sidecar schema is in scope. It is also why the
`service_connections` ID needs namespacing before the marketplace opens, not after.

The through-line: **nothing in Phase 0 may assume the catalogue is compiled in, or that a sidecar's code
is in this repository.**

---

## Open questions

1. `user:` in compose, and Podman support for rootless.
2. Plugin migrations: who applies them, how versioned, how upgraded.
3. `config` JSONB on `service_connections` — or a different escape hatch.
4. ID namespacing authority.
5. What the app store does when Docker is absent — hide "provision", or refuse to install?
6. Does an installed-but-unhealthy sidecar surface in the UI as broken, or as not installed?