platform/docs/sidecar-app-store.md

Sidecars as installable apps

Status: DESIGN, agreed in conversation 2026-08-10. Nothing implemented. This supersedes the framing of sidecar-bootstrapping.md, which stays as the record of how the mechanics work today.

The goal: a clean machine runs chat, the terminal and the file browser, and everything else arrives by the user asking for it — from an app store inside Officer. Eventually including sidecars the user did not write.

---

Why this is mostly not a rewrite

Three things are already true, which is why "nothing exactly blocks it":

• Every API route stays mounted regardless of which sidecars run. The light profile's own comment states it: features whose sidecars are absent report themselves unavailable rather than disappearing. So the app store never needs to mount or unmount routes.
• Officer already spawns nothing. Sidecars are PM2 peers that dial in and register by capability. Installing one is starting a process, not teaching officer about it.
• service_connections already solves the multi-user case, including the part nobody would get right independently — see below.

What is genuinely new: provisioning containers, per-sidecar schema, and persisted install state.

---

Light becomes the baseline

ecosystem.light.config.cjs stops being a variant and becomes what a fresh install runs:

officer  ·  officer-anthropic-proxy  ·  officer-agent  ·  officer-opencode  ·  officer-pty  ·  officer-gitea

Chat, terminal, file browser. The file-browsing APIs live in the main process, so they cost nothing extra.

The other fourteen become app-store entries.

---

Three install shapes

The prompt the user sees depends on which of these the sidecar is. This is the taxonomy the installer branches on:

Shape	What install means	Examples
Point at an instance you already have	Ask for URL + credential, write service_connections, start the sidecar	gitea, memos, photos (Immich), jellyfin, invoiceshelf, headscale
Provision one	Render our compose template, docker compose up -d, wait for health, write the connection we already know, start the sidecar	vault (Vaultwarden), slskd, transmission, caldav (Radicale), and any of the above where the user has none
Configuration only	Ask for credentials, start the sidecar. No service to reach	email (IMAP), notify, music, wallet, vnc

A sidecar can be more than one: Gitea is "existing instance" for someone who runs one and "provision" for someone who does not. The prompt is the fork.

---

Docker: we install, the user owns

Officer is the installer, never the owner. Concretely:

• A real compose file per service, written into <root>/dockers/<id>/, from our template — using the convention the owner already applies to 47 services: one directory per service, docker-compose.yaml inside, and relative bind mounts (./data, ./database, ./storage) so configuration and data sit beside the compose file where both we and a human can see them. Named volumes are used by 3 of those 47 and are the exception; templates use bind mounts, always.
• Started with docker compose up -d as the owner, not as officer's own identity.
• Found again by label (officer.sidecar=<id>), not by holding a handle.

Consequences, which are the point:

• docker ps, docker logs, docker compose down all behave normally.
• If Officer is removed, the containers keep running and stay manageable.
• A user who runs his own estate can edit the compose file — it is his file, in his directory.
• We can always find what we installed without pretending to own it.

The template is what makes this non-technical-user-friendly: sensible defaults, ports, volumes and health checks already correct, so "install Gitea" does not become a tutorial.

USER_UID / USER_GID are set to the owner, as the existing services already do. That answers the "do containers run as root" question: no, and this is not a new convention — it is the one in use.

The app store's containers are isolated from the user's own, and that is the point of the layout:

~/officerdev/
  platform/      the app
  data/          DATA_PATH
  dockers/       services the app store provisioned   <- exclusively ours
  capabilities/  the file-based item store

OFFICER_ROOT is derived as the parent of DATA_PATH rather than configured separately — a second variable that must agree with the first is a second thing to get wrong.

Deliberately not ~/dockers, which is where a seasoned user already keeps their estate. Two consequences, both wanted:

1. Containers the app store created are distinguishable from the user's own structurally, not by a naming convention we would have to enforce and they could break.
2. We never reason about someone else's compose files. The store does not scan, adopt or modify anything outside its own directory. "I already have one of these" is answered by the user giving a URL (mode: 'existing') — never by us finding a directory and guessing whose it is.

[open] Podman, for anyone wanting genuinely rootless.

Docker is assumed, and nothing guarantees it

Verified: nothing in scripts/ installs Docker, and nothing checks for it. The host installer scripts/setup/setup.sh calls scripts/setup/setup-dockers.sh, which invokes docker compose with no preflight, so a fresh host without Docker fails partway through setup with a bare "command not found".

(Not to be confused with the per-template setup.sh below — app-store/templates/<name>/setup.sh — which is a different file with a different contract. The host one provisions the machine; a template one provisions a single sidecar.)

That is the seam where this project's origin shows — it began as one person's own machine, provisioned by his own scripts, where Docker was simply always there.

The intended fix is a setup.sh per sidecar, ensuring its own dependencies before its compose file is used. That is also the shape a sidecar needs once it lives in its own repository, so a sidecar package becomes:

metadata (catalogue entry)  ·  compose template  ·  setup.sh  ·  schema

Until that exists, the app store detects and reports rather than guessing or half-installing: preflight.ts checks docker compose version — which exercises the binary, the daemon connection and the plugin in one call, unlike docker --version, which passes with a dead daemon — and distinguishes "not installed" from "daemon unreachable", because the remedies differ.

The check is per mode, not per entry: a host without Docker can still install Photos by pointing at an Immich elsewhere. Refusing the whole entry would be the over-strict check that makes people work around the installer instead of using it.

---

Install state

Two independent flags, because they answer different questions:

• installed — the thing exists: container provisioned, config written, schema applied.
• enabled — the process should be running.

Action	Sidecar process	Container	Data & schema
Disable	stopped	stopped	untouched
Enable	started	started	untouched
Uninstall	stopped	docker compose down, removed	untouched

Disable stops the container too — there is no reason to leave Immich holding memory while Photos is switched off. For mode: 'existing' there is no container of ours, so disable is only the sidecar.

Uninstall additionally deletes the sidecar_installs row. It does not drop the sidecar's tables.

Nothing above deletes data, and there is no option that does.

Why the schema survives uninstall too

Dropping a sidecar's tables is deleting data. Not media, but real: music favourites, the Jellyfin server registry, photos configuration, saved connections. That is the same category as volumes and gets the same answer.

It also buys something. Reinstall becomes restore — uninstall Photos in June, reinstall in August, and the configuration and favourites are still there. Drop the schema and reinstalling hands back a blank service that looks subtly broken to someone who remembers setting it up.

Keeping them costs nothing: an unused table is a row in information_schema. No queries, no memory, no maintenance. Dropping them joins volume deletion in the later, deliberate cleanup feature, where the user sees what they are removing.

Install must be idempotent and resumable. Provision → health → config → schema → start is five steps and any of them can fail. The failure mode to design against is a half-installed service that neither works nor uninstalls. Each step records what it did; re-running install resumes rather than restarts.

---

Per-sidecar schema

Today all 42 tables live in one Drizzle schema and arrive together via bun db:push. That changes: each sidecar owns its own schema and applies it on install.

This is right because third-party plugins are a real goal. For our own fourteen it would be over-engineering — an unused table costs nothing — but a marketplace plugin cannot ship a table into a schema it does not own.

The dependency graph makes this tractable. Measured across the 19 non-core schema files:

core:     auth.ts, server.ts, chat-events.ts        (depend on nothing)
sidecars: every single one -> auth.ts, and nothing else

There is no sidecar-to-sidecar dependency anywhere. One file (user-data.ts) touches two, and it is core. So the contract for a plugin's schema is nearly the smallest it could be:

You may reference users.id. You may not reference anything else.

Which also makes full uninstall well-defined: drop the tables this sidecar declared. Nothing else points at them, by construction.

[open] Where do a plugin's migrations live, and what applies them — the installer, or the sidecar on first boot? Versioning and upgrade are unsolved here.

---

service_connections is part of the contract

Decided: it stays core and shared, one table, with each plugin identified by its own ID — rather than a connections table per service.

It already does the hard part. The row is keyed (userId, service) and a NULL url means "inherit the instance": the owner's row carries the URL and is the instance; every other user's row carries only their own credential and resolves the base from the owner's row at read time.

So "members never see the instance URL" is a property of the schema rather than a filter someone must remember on every response — and a member cannot supply a URL, which closes what would otherwise be a per-user SSRF hop wearing a settings form. Gitea is the first service of this kind; five sidecars use the table today (memos, wallet, transmission, slskd, gitea).

A third-party plugin inherits all of that for free, which is the argument for sharing the table: it is the part nobody would get right independently.

Two things it needs before third parties touch it:

1. Namespaced IDs. service is free text — deliberately, so adding a service is not a schema change. With a marketplace, two plugins could both claim "gitea" and collide on the unique index. Needs a convention (reverse-DNS, or IDs issued by the marketplace).
2. Somewhere for plugin-specific config. The columns are shaped around the services that exist: url, username, secret, path, version. A plugin needing anything else has nowhere to put it, and adding a column per plugin defeats the shared table. Likely a config JSONB for the remainder — with url staying first-class, because the inheritance rule above depends on it being a real column.

---

The API contract, when we open this up

What a plugin author is promised, and bound by. To be written properly; the shape is:

• Register by name + capabilities over /api/sidecar/register; be reachable by capability.
• Declare an ID, an install shape, a compose template (if it provisions), a config prompt, and a schema.
• May reference users.id, and use service_connections under its own ID.
• May not reference another plugin's tables, or write outside its own.
• Must tolerate being disabled, re-enabled, and uninstalled.

---

Provisioning has three shapes, not one

This document originally said provisioning "writes the connection we already know". That is only true some of the time, and the difference decides whether an install can finish unattended:

1. We set the credentials. Passed as container environment, so the connection is known the moment it is up. Transmission (USER/PASS), Vaultwarden (ADMIN_TOKEN).
2. We generate a secret into a file. The bind mount lets us write it before first boot, so it is still known without asking. slskd's API key lives in its slskd.yml.
3. A human must mint a token in the service's own UI after it boots. Immich, Jellyfin and Memos all work this way — no environment variable pre-seeds an API key.

Shape 3 means an install can be provisioned and running but not yet connected. That is a real state, not a failure: the container is up, the compose file is written, and we are waiting for a token. The step machine stops there, and the UI asks for the key with a link to the page that mints it. Resuming finishes the job — which is what completedSteps was for.

---

Members get their own accounts

The owner installs, but a server may already have members — and a member added next month needs the same work done. So the unit is (service × member), reachable from two triggers:

install a service  ->  provision every member who already exists
add a member       ->  provision every service already installed

Only handling the first is the classic thing that works on day one and rots quietly. There is no new table: a member is provisioned for a service exactly when they hold a service_connections row for it — their own credential, url NULL, inheriting the instance from the owner's. That schema was built for this before this existed.

Three outcomes, declared per catalogue entry as members, so the installer never special-cases a service:

	Meaning	Services
accounts	Admin API creates the user and mints a credential. Fully transparent — the member just finds it working.	Immich, Jellyfin, Memos, InvoiceShelf, CalDAV
invite	The account can be created; a usable credential cannot. The member sets their own password.	Vaultwarden
none	Single-tenant daemon, no user concept. Access is mediated by Officer alone.	Transmission, slskd, headscale, email, music, wallet, notify, vnc

invite is not a weaker accounts — it is the correct outcome. Vaultwarden derives its encryption key from the master password, so a credential we could mint would mean a vault we could read. Transparent right up to the point where being transparent would be a defect.

The per-service work is an interface implemented beside each sidecar, never a switch in core: a central function growing one case per service is exactly what would stop any of this shipping from its own repository. Implementations must be idempotent — both triggers can fire for the same pair, and creating a second account upstream is not something we can undo.

Deprovision is deliberately optional and defaults to doing nothing upstream. Deleting a user in Immich deletes their photos; an app store that destroys data as a side effect of an unrelated action is worse than one that leaves a stale account behind.

Assumed working: the vault's own multi-user adaptation is being done separately. Today /api/vault is owner-only by an explicit ownerGate, so a member is refused before Vaultwarden is reached — this design is written as though that has landed.

---

What Phase 0 must not foreclose

Three things are coming, and each one constrains a decision that looks free today.

1. marketplace.officer.dev. Phase 1 keeps the catalogue inside this repo; later the app lists what is on a remote marketplace instead. So catalogue entries must stay serialisable data — no functions, no imports, nothing that only means something at compile time. They are plain objects today and must remain so, because the same shape has to arrive as JSON over HTTP. Compose templates travel with them.

2. Every sidecar becomes its own repository. Today catalogue.test.ts asserts the catalogue equals "everything in ecosystem.config.cjs that light excludes". That is the right check now, and it inverts later: once sidecars live elsewhere, the catalogue entry becomes the source of truth for how to run one (command, args, env) and the ecosystem file is generated from what is installed, not the other way round. Do not treat that test as a permanent law — it pins Phase 0's invariant, not the design's.

3. Third-party plugins. Already the reason per-sidecar schema is in scope. It is also why the service_connections ID needs namespacing before the marketplace opens, not after.

The through-line: nothing in Phase 0 may assume the catalogue is compiled in, or that a sidecar's code is in this repository.

---

Open questions

1. user: in compose, and Podman support for rootless.
2. Plugin migrations: who applies them, how versioned, how upgraded.
3. config JSONB on service_connections — or a different escape hatch.
4. ID namespacing authority.
5. What the app store does when Docker is absent — hide "provision", or refuse to install?
6. Does an installed-but-unhealthy sidecar surface in the UI as broken, or as not installed?