diff --git a/docs/sidecar-app-store.md b/docs/sidecar-app-store.md index 015d0875..8a6097dc 100644 --- a/docs/sidecar-app-store.md +++ b/docs/sidecar-app-store.md @@ -227,6 +227,25 @@ What a plugin author is promised, and bound by. To be written properly; the shap --- +## Provisioning has three shapes, not one + +This document originally said provisioning "writes the connection we already know". That is only true +some of the time, and the difference decides whether an install can finish unattended: + +1. **We set the credentials.** Passed as container environment, so the connection is known the moment it + is up. Transmission (`USER`/`PASS`), Vaultwarden (`ADMIN_TOKEN`). +2. **We generate a secret into a file.** The bind mount lets us write it before first boot, so it is + still known without asking. slskd's API key lives in its `slskd.yml`. +3. **A human must mint a token in the service's own UI after it boots.** Immich, Jellyfin and Memos all + work this way — no environment variable pre-seeds an API key. + +Shape 3 means an install can be **provisioned and running but not yet connected**. That is a real state, +not a failure: the container is up, the compose file is written, and we are waiting for a token. The +step machine stops there, and the UI asks for the key with a link to the page that mints it. Resuming +finishes the job — which is what `completedSteps` was for. + +--- + ## What Phase 0 must not foreclose Three things are coming, and each one constrains a decision that looks free today. diff --git a/src/servers/app-store/templates/README.md b/src/servers/app-store/templates/README.md new file mode 100644 index 00000000..5fd5320e --- /dev/null +++ b/src/servers/app-store/templates/README.md @@ -0,0 +1,82 @@ +# Compose templates and their setup scripts + +One directory per provisionable service. Each holds a `docker-compose.yaml` and a `setup.sh`, and +together they are everything needed to bring that service up. + +This is deliberately the shape a sidecar will need when it lives in **its own repository**: metadata +(the catalogue entry), a compose template, a setup script, and a schema. Nothing here may assume it is +being read out of this repo. + +--- + +## The setup.sh contract + +**Answers come from the environment. It never prompts when they are already there.** + +The app store collects them in a web form and passes them as environment variables. A person running it +by hand on a VPS gets prompted for anything missing, but only when stdin is a TTY — so the same script +serves both, and neither is a second code path. + +```sh +OFFICER_SERVICE_DIR=~/officerdev/dockers/transmission \ +OFFICER_UID=1000 OFFICER_GID=1000 \ +TRANSMISSION_PORT=9091 \ +bash setup.sh +``` + +Every script must: + +| Rule | Why | +|---|---| +| **Be idempotent.** Running twice must be safe and must not create a second anything. | Install is resumable; a retry after a half-failure re-runs steps that already succeeded. | +| **Never prompt when `OFFICER_NONINTERACTIVE=1`.** Fail with a clear message instead. | A prompt behind a web form is a hang with no output, which is the worst failure to diagnose. | +| **Write only inside `OFFICER_SERVICE_DIR`.** | The app store owns that directory and nothing else. The user's own estate is never touched. | +| **Emit progress on stdout.** | The installer streams it to a terminal panel in the UI, so the user watches it happen rather than staring at a spinner. | +| **Print `OFFICER_RESULT_=value` for anything the platform must store.** | How a generated secret or a resolved port gets back to `service_connections` without the installer parsing free text. | + +Exit non-zero on failure, with the reason on stderr. The installer records it in `last_error` and the +row stays `failed` rather than pretending to be installed. + +--- + +## Volumes are always relative bind mounts + +`./data`, `./config`, `./database` — never named volumes. Configuration and data sit beside the compose +file so both the platform and a human can see exactly what a service keeps and where. A named volume +hides it behind `docker volume inspect`, which is the opposite of the point. + +## Containers run as the owner + +`user: "${OFFICER_UID}:${OFFICER_GID}"`, so files a container writes are owned by the person who +installed it and not by root. This is the convention already in use across the owner's own services. + +## Ports bind to loopback unless the service genuinely needs to be reachable + +`127.0.0.1:9091:9091`, not `9091:9091`. Officer reaches these over loopback; anything published on all +interfaces is a service exposed to the network by an installer the user trusted to be careful. The +exception is a service whose whole function is inbound connections — slskd's P2P listener, for example — +and those say so in a comment. + +## No external networks + +The owner's own composes attach to an external `nginx` network that exists on his machine. Templates +must not require one: a fresh VPS has no such network and `docker compose up` would fail before it +started. Default network only. + +--- + +## Three provisioning shapes, not one + +Worth knowing before writing a template, because the design doc originally assumed only the first: + +1. **We set the credentials.** Passed as environment to the container, so the connection is fully known + the moment it is up. Transmission (`USER`/`PASS`), Vaultwarden (`ADMIN_TOKEN`). +2. **We generate a secret into a config file.** The bind mount lets us write it before first boot, so it + is still known without asking. slskd's API key lives in its `slskd.yml`. +3. **A human must mint a token in the service's own UI after it boots.** Immich, Jellyfin and Memos all + work this way — there is no environment variable that pre-seeds an API key. + +Shape 3 means an install can be **provisioned and running, but not yet connected**. That is a real state, +not an error: the container is up, the compose file is written, and the platform is waiting for a token. +The installer stops there with the step recorded, and the UI asks for the key with a link to the page +that mints it. Resuming finishes the job. diff --git a/src/servers/app-store/templates/transmission/docker-compose.yaml b/src/servers/app-store/templates/transmission/docker-compose.yaml new file mode 100644 index 00000000..043fcb2f --- /dev/null +++ b/src/servers/app-store/templates/transmission/docker-compose.yaml @@ -0,0 +1,40 @@ +# Transmission — the daemon Officer's transmission sidecar drives over RPC. +# +# Rendered by setup.sh; ${...} are substituted before this reaches disk. Everything the container keeps +# lives beside this file, so `ls` answers "what is this service storing" without docker involved. +name: officer-transmission + +services: + transmission: + image: lscr.io/linuxserver/transmission:latest + container_name: officer-transmission + restart: unless-stopped + + # As the installing user, so completed downloads are not root-owned — the single most annoying + # thing about a container that writes to a shared folder. + user: '${OFFICER_UID}:${OFFICER_GID}' + + environment: + - PUID=${OFFICER_UID} + - PGID=${OFFICER_GID} + - TZ=${OFFICER_TZ} + # Blank means no RPC auth, which is Transmission's normal posture and is safe HERE specifically + # because the RPC port is bound to loopback below. setup.sh fills these only if the user asked for + # credentials; an empty USER is not the same as the variable being unset. + - USER=${TRANSMISSION_USER} + - PASS=${TRANSMISSION_PASS} + + ports: + # Loopback only. Officer talks to this over 127.0.0.1; nothing else has any business reaching the + # RPC endpoint, and publishing it on all interfaces would put an unauthenticated control API on the + # network because "no auth" is the default above. + - '127.0.0.1:${TRANSMISSION_PORT}:9091' + # Peer traffic, and the exception to the loopback rule: BitTorrent peers must be able to connect + # inbound or the client is crippled to outbound-only connections. + - '${TRANSMISSION_PEER_PORT}:51413' + - '${TRANSMISSION_PEER_PORT}:51413/udp' + + volumes: + - ./config:/config + - ./downloads:/downloads + - ./watch:/watch diff --git a/src/servers/app-store/templates/transmission/setup.sh b/src/servers/app-store/templates/transmission/setup.sh new file mode 100755 index 00000000..0c6fa4bf --- /dev/null +++ b/src/servers/app-store/templates/transmission/setup.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# Provision Transmission for Officer. +# +# Answers come from the environment; a human running this by hand is prompted for what is missing, but +# only when stdin is a TTY. See ../README.md for the contract every one of these obeys. +# +# OFFICER_SERVICE_DIR where to write. Everything this script creates is inside it. +# OFFICER_UID/GID who the container runs as +# TRANSMISSION_PORT RPC port on loopback (default 9091) +# TRANSMISSION_PEER_PORT BitTorrent listen port (default 51413) +# TRANSMISSION_USER/PASS optional RPC credentials (default: none, loopback-only) +# +# Idempotent: safe to re-run, which is what makes a resumed install work rather than duplicate. + +set -euo pipefail + +SERVICE_DIR="${OFFICER_SERVICE_DIR:?OFFICER_SERVICE_DIR is required}" +TEMPLATE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +OFFICER_UID="${OFFICER_UID:-$(id -u)}" +OFFICER_GID="${OFFICER_GID:-$(id -g)}" +OFFICER_TZ="${OFFICER_TZ:-$(cat /etc/timezone 2>/dev/null || echo UTC)}" + +# ── Asking ──────────────────────────────────────────────────────────────────────────────────────────── +# A prompt behind a web form is a hang with no output. When the caller says it cannot answer, fail with +# the reason instead of blocking forever on a read nobody will ever satisfy. +ask() { + local var="$1" prompt="$2" default="${3:-}" + local current="${!var:-}" + if [ -n "$current" ]; then return 0; fi + if [ "${OFFICER_NONINTERACTIVE:-0}" = "1" ] || [ ! -t 0 ]; then + if [ -n "$default" ]; then printf -v "$var" '%s' "$default"; return 0; fi + echo "error: $var is required and this is a non-interactive run" >&2 + exit 2 + fi + local answer + read -r -p "$prompt${default:+ [$default]}: " answer + printf -v "$var" '%s' "${answer:-$default}" +} + +ask TRANSMISSION_PORT 'Transmission RPC port (loopback only)' '9091' +ask TRANSMISSION_PEER_PORT 'BitTorrent peer port' '51413' +TRANSMISSION_USER="${TRANSMISSION_USER:-}" +TRANSMISSION_PASS="${TRANSMISSION_PASS:-}" + +# ── Render ──────────────────────────────────────────────────────────────────────────────────────────── +echo "==> Preparing $SERVICE_DIR" +mkdir -p "$SERVICE_DIR/config" "$SERVICE_DIR/downloads" "$SERVICE_DIR/watch" + +# envsubst with an explicit variable list, never the bare form: unrestricted envsubst would also expand +# anything in the template that merely looks like a variable, and a compose file is full of $ that +# belongs to other tools. +export OFFICER_UID OFFICER_GID OFFICER_TZ TRANSMISSION_PORT TRANSMISSION_PEER_PORT TRANSMISSION_USER TRANSMISSION_PASS +envsubst '${OFFICER_UID} ${OFFICER_GID} ${OFFICER_TZ} ${TRANSMISSION_PORT} ${TRANSMISSION_PEER_PORT} ${TRANSMISSION_USER} ${TRANSMISSION_PASS}' \ + < "$TEMPLATE_DIR/docker-compose.yaml" > "$SERVICE_DIR/docker-compose.yaml" + +echo "==> Starting the container" +# `up -d` is already idempotent: an unchanged compose file against a running container is a no-op, and a +# changed one recreates. That is the whole reason a re-run is safe. +docker compose --project-directory "$SERVICE_DIR" up -d + +# ── Wait ────────────────────────────────────────────────────────────────────────────────────────────── +# Reporting success the moment `up -d` returns would be a lie: the container exists, the RPC endpoint is +# not listening yet, and the platform's first call would fail against a service we just said was ready. +echo "==> Waiting for the RPC endpoint" +for i in $(seq 1 60); do + # 409 is the correct healthy answer here — Transmission demands a session id and rejects the first + # request by design. Treating only 200 as healthy would wait out the full timeout on a working daemon. + code="$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:${TRANSMISSION_PORT}/transmission/rpc" || true)" + if [ "$code" = "409" ] || [ "$code" = "200" ] || [ "$code" = "401" ]; then + echo " up after ${i}s (HTTP $code)" + break + fi + if [ "$i" = "60" ]; then + echo "error: Transmission did not answer on 127.0.0.1:${TRANSMISSION_PORT} within 60s" >&2 + echo " check: docker compose --project-directory '$SERVICE_DIR' logs" >&2 + exit 1 + fi + sleep 1 +done + +# ── Hand the connection back ────────────────────────────────────────────────────────────────────────── +# Shape 1 from the README: we set the credentials, so nothing has to be asked for after the fact. These +# lines are the installer's only interface to this script's results — parsed by prefix, never by +# scraping the log above. +echo "OFFICER_RESULT_URL=http://127.0.0.1:${TRANSMISSION_PORT}" +echo "OFFICER_RESULT_PATH=/transmission/rpc" +echo "OFFICER_RESULT_USERNAME=${TRANSMISSION_USER}" +echo "OFFICER_RESULT_SECRET=${TRANSMISSION_PASS}" +echo "==> Done" diff --git a/src/servers/app-store/templates/vaultwarden/docker-compose.yaml b/src/servers/app-store/templates/vaultwarden/docker-compose.yaml new file mode 100644 index 00000000..bcb31d3d --- /dev/null +++ b/src/servers/app-store/templates/vaultwarden/docker-compose.yaml @@ -0,0 +1,35 @@ +# Vaultwarden — the Bitwarden-compatible server behind Officer's vault sidecar. +# +# Every request reaches it through us: app → /api/vault → officer-vault → here. Nothing else should be +# able to, which is why the port below is loopback-only. +name: officer-vault + +services: + vaultwarden: + image: vaultwarden/server:latest + container_name: officer-vault + restart: unless-stopped + + user: '${OFFICER_UID}:${OFFICER_GID}' + + environment: + - TZ=${OFFICER_TZ} + # Argon2 hash of a token setup.sh generated. The plaintext is printed once, to the installer, and + # never written to disk here — a compose file is not a secret store, and this one sits in a + # directory the user is encouraged to read. + - ADMIN_TOKEN=${VAULTWARDEN_ADMIN_TOKEN_HASH} + # Closed by default. An open Vaultwarden on a machine the owner just handed a password manager to + # is the wrong default at every scale, and the owner can open it from the admin page if they want. + - SIGNUPS_ALLOWED=false + # WebSocket notifications, so the Bitwarden clients get live sync rather than polling. + - WEBSOCKET_ENABLED=true + + ports: + # Loopback only, always. This is a password vault: the only thing that should reach it is the + # sidecar on this host, and publishing it on all interfaces would put it on the network. + - '127.0.0.1:${VAULTWARDEN_PORT}:80' + + volumes: + # Holds the SQLite database, the attachments and the RSA keys. Bind-mounted rather than a named + # volume so the owner can see — and back up — exactly where their passwords live. + - ./data:/data diff --git a/src/servers/app-store/templates/vaultwarden/setup.sh b/src/servers/app-store/templates/vaultwarden/setup.sh new file mode 100755 index 00000000..9fb8c5fc --- /dev/null +++ b/src/servers/app-store/templates/vaultwarden/setup.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +# Provision Vaultwarden for Officer. +# +# Shape 2 from ../README.md: nothing is asked for, because the one credential that matters is GENERATED +# here. There is no sensible way for a user to invent an admin token, and asking for one produces a +# weaker secret than `openssl rand` does. +# +# OFFICER_SERVICE_DIR where to write +# OFFICER_UID/GID who the container runs as +# VAULTWARDEN_PORT loopback port (default 8222) +# +# Idempotent, including the token: an existing one is REUSED rather than rotated, because rotating on a +# re-run would lock the owner out of the admin page during a routine resumed install. + +set -euo pipefail + +SERVICE_DIR="${OFFICER_SERVICE_DIR:?OFFICER_SERVICE_DIR is required}" +TEMPLATE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +OFFICER_UID="${OFFICER_UID:-$(id -u)}" +OFFICER_GID="${OFFICER_GID:-$(id -g)}" +OFFICER_TZ="${OFFICER_TZ:-$(cat /etc/timezone 2>/dev/null || echo UTC)}" +VAULTWARDEN_PORT="${VAULTWARDEN_PORT:-8222}" + +command -v docker >/dev/null || { echo "error: docker is not installed" >&2; exit 2; } +command -v openssl >/dev/null || { echo "error: openssl is required to generate the admin token" >&2; exit 2; } + +mkdir -p "$SERVICE_DIR/data" + +# ── The admin token ─────────────────────────────────────────────────────────────────────────────────── +# Kept in a 0600 file inside the service directory rather than only in the database, so the owner can +# still reach the admin page if Officer is down — which is exactly when they might need to. +TOKEN_FILE="$SERVICE_DIR/.admin-token" + +if [ -f "$TOKEN_FILE" ]; then + echo "==> Reusing the existing admin token" + VAULTWARDEN_ADMIN_TOKEN="$(cat "$TOKEN_FILE")" +else + echo "==> Generating an admin token" + VAULTWARDEN_ADMIN_TOKEN="$(openssl rand -base64 48 | tr -d '\n')" + ( umask 077; printf '%s' "$VAULTWARDEN_ADMIN_TOKEN" > "$TOKEN_FILE" ) +fi + +# Vaultwarden accepts a plaintext token but warns loudly and recommends an Argon2 hash; `vaultwarden +# hash` does not exist as a standalone binary, so the hash is produced by the image itself. Falling back +# to plaintext rather than failing: a working install with a warning beats no install at all, and the +# token is only reachable over loopback. +echo "==> Hashing it" +if VAULTWARDEN_ADMIN_TOKEN_HASH="$(printf '%s' "$VAULTWARDEN_ADMIN_TOKEN" \ + | docker run --rm -i vaultwarden/server:latest /vaultwarden hash --preset owasp 2>/dev/null \ + | grep -oE '\$argon2[^ ]*' | head -1)" && [ -n "$VAULTWARDEN_ADMIN_TOKEN_HASH" ]; then + # Compose reads `$` as interpolation, so a literal Argon2 hash must have every `$` doubled or the + # container receives a mangled token and rejects every admin login with no useful error. + VAULTWARDEN_ADMIN_TOKEN_HASH="${VAULTWARDEN_ADMIN_TOKEN_HASH//\$/\$\$}" +else + echo " (could not hash — falling back to a plaintext token, which Vaultwarden will warn about)" + VAULTWARDEN_ADMIN_TOKEN_HASH="$VAULTWARDEN_ADMIN_TOKEN" +fi + +# ── Render and start ────────────────────────────────────────────────────────────────────────────────── +echo "==> Preparing $SERVICE_DIR" +export OFFICER_UID OFFICER_GID OFFICER_TZ VAULTWARDEN_PORT VAULTWARDEN_ADMIN_TOKEN_HASH +envsubst '${OFFICER_UID} ${OFFICER_GID} ${OFFICER_TZ} ${VAULTWARDEN_PORT} ${VAULTWARDEN_ADMIN_TOKEN_HASH}' \ + < "$TEMPLATE_DIR/docker-compose.yaml" > "$SERVICE_DIR/docker-compose.yaml" + +echo "==> Starting the container" +docker compose --project-directory "$SERVICE_DIR" up -d + +echo "==> Waiting for Vaultwarden to answer" +for i in $(seq 1 60); do + code="$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:${VAULTWARDEN_PORT}/alive" || true)" + if [ "$code" = "200" ]; then echo " up after ${i}s"; break; fi + if [ "$i" = "60" ]; then + echo "error: Vaultwarden did not answer on 127.0.0.1:${VAULTWARDEN_PORT} within 60s" >&2 + echo " check: docker compose --project-directory '$SERVICE_DIR' logs" >&2 + exit 1 + fi + sleep 1 +done + +# The URL is all the platform stores. The admin token is deliberately NOT returned: the vault sidecar +# proxies the Bitwarden protocol and never needs it, and a secret the platform does not hold is a secret +# it cannot leak. It is in $TOKEN_FILE for the owner. +echo "OFFICER_RESULT_URL=http://127.0.0.1:${VAULTWARDEN_PORT}" +echo "==> Done. Admin token: $TOKEN_FILE (0600, not stored by Officer)"