From 890f57a7a6a3e39a74a5baf2b7f3d455c9bfc6f4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9=20Padez?= Date: Mon, 10 Aug 2026 12:44:47 +0000 Subject: [PATCH] app store: compose templates and their setup scripts, with two proven end to end MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Each provisionable service gets a directory holding a compose template and a setup.sh. Deliberately the shape a sidecar needs once it lives in its own repository: metadata, compose, setup script, schema. The contract (templates/README.md): answers come from the ENVIRONMENT, so the web form fills them in and a person on a VPS is prompted only for what is missing, and only on a TTY — one script for both, not two code paths. Idempotent, writes only inside its own directory, streams progress on stdout (the installer pipes it to a terminal panel), and returns results as OFFICER_RESULT_= lines so nothing has to scrape a log. House conventions throughout: relative bind mounts so data sits beside the compose file rather than hiding behind `docker volume inspect`, containers running as the installing user so downloads are not root-owned, loopback-only ports unless the service's whole job is inbound connections, and no external networks — the owner's own composes attach to an `nginx` network that a fresh VPS does not have. Transmission verified end to end on this machine, on non-conflicting ports, then torn down: renders, starts, waits, reports. Its health check accepts 409 because Transmission rejects the first request by design — only-200 would have waited out the full timeout against a working daemon. Re-run produced exactly one container, and files landed owned by the user rather than root. Vaultwarden covers the case where we GENERATE the credential rather than asking for one. An existing token is reused, never rotated, because rotating during a resumed install would lock the owner out of the admin page. The Argon2 hash has its `$` doubled or compose interpolation mangles it. The token is not returned to the platform at all — the vault sidecar proxies the Bitwarden protocol and never needs it, and a secret we do not hold is one we cannot leak. Corrects the design doc, which assumed provisioning always knows the connection. Three shapes: we set the credential, we generate it, or a human must mint it in the service's UI afterwards (Immich, Jellyfin, Memos). The third makes "provisioned and running but not yet connected" a real state rather than a failure. Co-Authored-By: Claude Opus 5 --- docs/sidecar-app-store.md | 19 ++++ src/servers/app-store/templates/README.md | 82 +++++++++++++++++ .../transmission/docker-compose.yaml | 40 +++++++++ .../app-store/templates/transmission/setup.sh | 90 +++++++++++++++++++ .../templates/vaultwarden/docker-compose.yaml | 35 ++++++++ .../app-store/templates/vaultwarden/setup.sh | 85 ++++++++++++++++++ 6 files changed, 351 insertions(+) create mode 100644 src/servers/app-store/templates/README.md create mode 100644 src/servers/app-store/templates/transmission/docker-compose.yaml create mode 100755 src/servers/app-store/templates/transmission/setup.sh create mode 100644 src/servers/app-store/templates/vaultwarden/docker-compose.yaml create mode 100755 src/servers/app-store/templates/vaultwarden/setup.sh diff --git a/docs/sidecar-app-store.md b/docs/sidecar-app-store.md index 015d0875..8a6097dc 100644 --- a/docs/sidecar-app-store.md +++ b/docs/sidecar-app-store.md @@ -227,6 +227,25 @@ What a plugin author is promised, and bound by. To be written properly; the shap --- +## Provisioning has three shapes, not one + +This document originally said provisioning "writes the connection we already know". That is only true +some of the time, and the difference decides whether an install can finish unattended: + +1. **We set the credentials.** Passed as container environment, so the connection is known the moment it + is up. Transmission (`USER`/`PASS`), Vaultwarden (`ADMIN_TOKEN`). +2. **We generate a secret into a file.** The bind mount lets us write it before first boot, so it is + still known without asking. slskd's API key lives in its `slskd.yml`. +3. **A human must mint a token in the service's own UI after it boots.** Immich, Jellyfin and Memos all + work this way — no environment variable pre-seeds an API key. + +Shape 3 means an install can be **provisioned and running but not yet connected**. That is a real state, +not a failure: the container is up, the compose file is written, and we are waiting for a token. The +step machine stops there, and the UI asks for the key with a link to the page that mints it. Resuming +finishes the job — which is what `completedSteps` was for. + +--- + ## What Phase 0 must not foreclose Three things are coming, and each one constrains a decision that looks free today. diff --git a/src/servers/app-store/templates/README.md b/src/servers/app-store/templates/README.md new file mode 100644 index 00000000..5fd5320e --- /dev/null +++ b/src/servers/app-store/templates/README.md @@ -0,0 +1,82 @@ +# Compose templates and their setup scripts + +One directory per provisionable service. Each holds a `docker-compose.yaml` and a `setup.sh`, and +together they are everything needed to bring that service up. + +This is deliberately the shape a sidecar will need when it lives in **its own repository**: metadata +(the catalogue entry), a compose template, a setup script, and a schema. Nothing here may assume it is +being read out of this repo. + +--- + +## The setup.sh contract + +**Answers come from the environment. It never prompts when they are already there.** + +The app store collects them in a web form and passes them as environment variables. A person running it +by hand on a VPS gets prompted for anything missing, but only when stdin is a TTY — so the same script +serves both, and neither is a second code path. + +```sh +OFFICER_SERVICE_DIR=~/officerdev/dockers/transmission \ +OFFICER_UID=1000 OFFICER_GID=1000 \ +TRANSMISSION_PORT=9091 \ +bash setup.sh +``` + +Every script must: + +| Rule | Why | +|---|---| +| **Be idempotent.** Running twice must be safe and must not create a second anything. | Install is resumable; a retry after a half-failure re-runs steps that already succeeded. | +| **Never prompt when `OFFICER_NONINTERACTIVE=1`.** Fail with a clear message instead. | A prompt behind a web form is a hang with no output, which is the worst failure to diagnose. | +| **Write only inside `OFFICER_SERVICE_DIR`.** | The app store owns that directory and nothing else. The user's own estate is never touched. | +| **Emit progress on stdout.** | The installer streams it to a terminal panel in the UI, so the user watches it happen rather than staring at a spinner. | +| **Print `OFFICER_RESULT_=value` for anything the platform must store.** | How a generated secret or a resolved port gets back to `service_connections` without the installer parsing free text. | + +Exit non-zero on failure, with the reason on stderr. The installer records it in `last_error` and the +row stays `failed` rather than pretending to be installed. + +--- + +## Volumes are always relative bind mounts + +`./data`, `./config`, `./database` — never named volumes. Configuration and data sit beside the compose +file so both the platform and a human can see exactly what a service keeps and where. A named volume +hides it behind `docker volume inspect`, which is the opposite of the point. + +## Containers run as the owner + +`user: "${OFFICER_UID}:${OFFICER_GID}"`, so files a container writes are owned by the person who +installed it and not by root. This is the convention already in use across the owner's own services. + +## Ports bind to loopback unless the service genuinely needs to be reachable + +`127.0.0.1:9091:9091`, not `9091:9091`. Officer reaches these over loopback; anything published on all +interfaces is a service exposed to the network by an installer the user trusted to be careful. The +exception is a service whose whole function is inbound connections — slskd's P2P listener, for example — +and those say so in a comment. + +## No external networks + +The owner's own composes attach to an external `nginx` network that exists on his machine. Templates +must not require one: a fresh VPS has no such network and `docker compose up` would fail before it +started. Default network only. + +--- + +## Three provisioning shapes, not one + +Worth knowing before writing a template, because the design doc originally assumed only the first: + +1. **We set the credentials.** Passed as environment to the container, so the connection is fully known + the moment it is up. Transmission (`USER`/`PASS`), Vaultwarden (`ADMIN_TOKEN`). +2. **We generate a secret into a config file.** The bind mount lets us write it before first boot, so it + is still known without asking. slskd's API key lives in its `slskd.yml`. +3. **A human must mint a token in the service's own UI after it boots.** Immich, Jellyfin and Memos all + work this way — there is no environment variable that pre-seeds an API key. + +Shape 3 means an install can be **provisioned and running, but not yet connected**. That is a real state, +not an error: the container is up, the compose file is written, and the platform is waiting for a token. +The installer stops there with the step recorded, and the UI asks for the key with a link to the page +that mints it. Resuming finishes the job. diff --git a/src/servers/app-store/templates/transmission/docker-compose.yaml b/src/servers/app-store/templates/transmission/docker-compose.yaml new file mode 100644 index 00000000..043fcb2f --- /dev/null +++ b/src/servers/app-store/templates/transmission/docker-compose.yaml @@ -0,0 +1,40 @@ +# Transmission — the daemon Officer's transmission sidecar drives over RPC. +# +# Rendered by setup.sh; ${...} are substituted before this reaches disk. Everything the container keeps +# lives beside this file, so `ls` answers "what is this service storing" without docker involved. +name: officer-transmission + +services: + transmission: + image: lscr.io/linuxserver/transmission:latest + container_name: officer-transmission + restart: unless-stopped + + # As the installing user, so completed downloads are not root-owned — the single most annoying + # thing about a container that writes to a shared folder. + user: '${OFFICER_UID}:${OFFICER_GID}' + + environment: + - PUID=${OFFICER_UID} + - PGID=${OFFICER_GID} + - TZ=${OFFICER_TZ} + # Blank means no RPC auth, which is Transmission's normal posture and is safe HERE specifically + # because the RPC port is bound to loopback below. setup.sh fills these only if the user asked for + # credentials; an empty USER is not the same as the variable being unset. + - USER=${TRANSMISSION_USER} + - PASS=${TRANSMISSION_PASS} + + ports: + # Loopback only. Officer talks to this over 127.0.0.1; nothing else has any business reaching the + # RPC endpoint, and publishing it on all interfaces would put an unauthenticated control API on the + # network because "no auth" is the default above. + - '127.0.0.1:${TRANSMISSION_PORT}:9091' + # Peer traffic, and the exception to the loopback rule: BitTorrent peers must be able to connect + # inbound or the client is crippled to outbound-only connections. + - '${TRANSMISSION_PEER_PORT}:51413' + - '${TRANSMISSION_PEER_PORT}:51413/udp' + + volumes: + - ./config:/config + - ./downloads:/downloads + - ./watch:/watch diff --git a/src/servers/app-store/templates/transmission/setup.sh b/src/servers/app-store/templates/transmission/setup.sh new file mode 100755 index 00000000..0c6fa4bf --- /dev/null +++ b/src/servers/app-store/templates/transmission/setup.sh @@ -0,0 +1,90 @@ +#!/usr/bin/env bash +# Provision Transmission for Officer. +# +# Answers come from the environment; a human running this by hand is prompted for what is missing, but +# only when stdin is a TTY. See ../README.md for the contract every one of these obeys. +# +# OFFICER_SERVICE_DIR where to write. Everything this script creates is inside it. +# OFFICER_UID/GID who the container runs as +# TRANSMISSION_PORT RPC port on loopback (default 9091) +# TRANSMISSION_PEER_PORT BitTorrent listen port (default 51413) +# TRANSMISSION_USER/PASS optional RPC credentials (default: none, loopback-only) +# +# Idempotent: safe to re-run, which is what makes a resumed install work rather than duplicate. + +set -euo pipefail + +SERVICE_DIR="${OFFICER_SERVICE_DIR:?OFFICER_SERVICE_DIR is required}" +TEMPLATE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +OFFICER_UID="${OFFICER_UID:-$(id -u)}" +OFFICER_GID="${OFFICER_GID:-$(id -g)}" +OFFICER_TZ="${OFFICER_TZ:-$(cat /etc/timezone 2>/dev/null || echo UTC)}" + +# ── Asking ──────────────────────────────────────────────────────────────────────────────────────────── +# A prompt behind a web form is a hang with no output. When the caller says it cannot answer, fail with +# the reason instead of blocking forever on a read nobody will ever satisfy. +ask() { + local var="$1" prompt="$2" default="${3:-}" + local current="${!var:-}" + if [ -n "$current" ]; then return 0; fi + if [ "${OFFICER_NONINTERACTIVE:-0}" = "1" ] || [ ! -t 0 ]; then + if [ -n "$default" ]; then printf -v "$var" '%s' "$default"; return 0; fi + echo "error: $var is required and this is a non-interactive run" >&2 + exit 2 + fi + local answer + read -r -p "$prompt${default:+ [$default]}: " answer + printf -v "$var" '%s' "${answer:-$default}" +} + +ask TRANSMISSION_PORT 'Transmission RPC port (loopback only)' '9091' +ask TRANSMISSION_PEER_PORT 'BitTorrent peer port' '51413' +TRANSMISSION_USER="${TRANSMISSION_USER:-}" +TRANSMISSION_PASS="${TRANSMISSION_PASS:-}" + +# ── Render ──────────────────────────────────────────────────────────────────────────────────────────── +echo "==> Preparing $SERVICE_DIR" +mkdir -p "$SERVICE_DIR/config" "$SERVICE_DIR/downloads" "$SERVICE_DIR/watch" + +# envsubst with an explicit variable list, never the bare form: unrestricted envsubst would also expand +# anything in the template that merely looks like a variable, and a compose file is full of $ that +# belongs to other tools. +export OFFICER_UID OFFICER_GID OFFICER_TZ TRANSMISSION_PORT TRANSMISSION_PEER_PORT TRANSMISSION_USER TRANSMISSION_PASS +envsubst '${OFFICER_UID} ${OFFICER_GID} ${OFFICER_TZ} ${TRANSMISSION_PORT} ${TRANSMISSION_PEER_PORT} ${TRANSMISSION_USER} ${TRANSMISSION_PASS}' \ + < "$TEMPLATE_DIR/docker-compose.yaml" > "$SERVICE_DIR/docker-compose.yaml" + +echo "==> Starting the container" +# `up -d` is already idempotent: an unchanged compose file against a running container is a no-op, and a +# changed one recreates. That is the whole reason a re-run is safe. +docker compose --project-directory "$SERVICE_DIR" up -d + +# ── Wait ────────────────────────────────────────────────────────────────────────────────────────────── +# Reporting success the moment `up -d` returns would be a lie: the container exists, the RPC endpoint is +# not listening yet, and the platform's first call would fail against a service we just said was ready. +echo "==> Waiting for the RPC endpoint" +for i in $(seq 1 60); do + # 409 is the correct healthy answer here — Transmission demands a session id and rejects the first + # request by design. Treating only 200 as healthy would wait out the full timeout on a working daemon. + code="$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:${TRANSMISSION_PORT}/transmission/rpc" || true)" + if [ "$code" = "409" ] || [ "$code" = "200" ] || [ "$code" = "401" ]; then + echo " up after ${i}s (HTTP $code)" + break + fi + if [ "$i" = "60" ]; then + echo "error: Transmission did not answer on 127.0.0.1:${TRANSMISSION_PORT} within 60s" >&2 + echo " check: docker compose --project-directory '$SERVICE_DIR' logs" >&2 + exit 1 + fi + sleep 1 +done + +# ── Hand the connection back ────────────────────────────────────────────────────────────────────────── +# Shape 1 from the README: we set the credentials, so nothing has to be asked for after the fact. These +# lines are the installer's only interface to this script's results — parsed by prefix, never by +# scraping the log above. +echo "OFFICER_RESULT_URL=http://127.0.0.1:${TRANSMISSION_PORT}" +echo "OFFICER_RESULT_PATH=/transmission/rpc" +echo "OFFICER_RESULT_USERNAME=${TRANSMISSION_USER}" +echo "OFFICER_RESULT_SECRET=${TRANSMISSION_PASS}" +echo "==> Done" diff --git a/src/servers/app-store/templates/vaultwarden/docker-compose.yaml b/src/servers/app-store/templates/vaultwarden/docker-compose.yaml new file mode 100644 index 00000000..bcb31d3d --- /dev/null +++ b/src/servers/app-store/templates/vaultwarden/docker-compose.yaml @@ -0,0 +1,35 @@ +# Vaultwarden — the Bitwarden-compatible server behind Officer's vault sidecar. +# +# Every request reaches it through us: app → /api/vault → officer-vault → here. Nothing else should be +# able to, which is why the port below is loopback-only. +name: officer-vault + +services: + vaultwarden: + image: vaultwarden/server:latest + container_name: officer-vault + restart: unless-stopped + + user: '${OFFICER_UID}:${OFFICER_GID}' + + environment: + - TZ=${OFFICER_TZ} + # Argon2 hash of a token setup.sh generated. The plaintext is printed once, to the installer, and + # never written to disk here — a compose file is not a secret store, and this one sits in a + # directory the user is encouraged to read. + - ADMIN_TOKEN=${VAULTWARDEN_ADMIN_TOKEN_HASH} + # Closed by default. An open Vaultwarden on a machine the owner just handed a password manager to + # is the wrong default at every scale, and the owner can open it from the admin page if they want. + - SIGNUPS_ALLOWED=false + # WebSocket notifications, so the Bitwarden clients get live sync rather than polling. + - WEBSOCKET_ENABLED=true + + ports: + # Loopback only, always. This is a password vault: the only thing that should reach it is the + # sidecar on this host, and publishing it on all interfaces would put it on the network. + - '127.0.0.1:${VAULTWARDEN_PORT}:80' + + volumes: + # Holds the SQLite database, the attachments and the RSA keys. Bind-mounted rather than a named + # volume so the owner can see — and back up — exactly where their passwords live. + - ./data:/data diff --git a/src/servers/app-store/templates/vaultwarden/setup.sh b/src/servers/app-store/templates/vaultwarden/setup.sh new file mode 100755 index 00000000..9fb8c5fc --- /dev/null +++ b/src/servers/app-store/templates/vaultwarden/setup.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +# Provision Vaultwarden for Officer. +# +# Shape 2 from ../README.md: nothing is asked for, because the one credential that matters is GENERATED +# here. There is no sensible way for a user to invent an admin token, and asking for one produces a +# weaker secret than `openssl rand` does. +# +# OFFICER_SERVICE_DIR where to write +# OFFICER_UID/GID who the container runs as +# VAULTWARDEN_PORT loopback port (default 8222) +# +# Idempotent, including the token: an existing one is REUSED rather than rotated, because rotating on a +# re-run would lock the owner out of the admin page during a routine resumed install. + +set -euo pipefail + +SERVICE_DIR="${OFFICER_SERVICE_DIR:?OFFICER_SERVICE_DIR is required}" +TEMPLATE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +OFFICER_UID="${OFFICER_UID:-$(id -u)}" +OFFICER_GID="${OFFICER_GID:-$(id -g)}" +OFFICER_TZ="${OFFICER_TZ:-$(cat /etc/timezone 2>/dev/null || echo UTC)}" +VAULTWARDEN_PORT="${VAULTWARDEN_PORT:-8222}" + +command -v docker >/dev/null || { echo "error: docker is not installed" >&2; exit 2; } +command -v openssl >/dev/null || { echo "error: openssl is required to generate the admin token" >&2; exit 2; } + +mkdir -p "$SERVICE_DIR/data" + +# ── The admin token ─────────────────────────────────────────────────────────────────────────────────── +# Kept in a 0600 file inside the service directory rather than only in the database, so the owner can +# still reach the admin page if Officer is down — which is exactly when they might need to. +TOKEN_FILE="$SERVICE_DIR/.admin-token" + +if [ -f "$TOKEN_FILE" ]; then + echo "==> Reusing the existing admin token" + VAULTWARDEN_ADMIN_TOKEN="$(cat "$TOKEN_FILE")" +else + echo "==> Generating an admin token" + VAULTWARDEN_ADMIN_TOKEN="$(openssl rand -base64 48 | tr -d '\n')" + ( umask 077; printf '%s' "$VAULTWARDEN_ADMIN_TOKEN" > "$TOKEN_FILE" ) +fi + +# Vaultwarden accepts a plaintext token but warns loudly and recommends an Argon2 hash; `vaultwarden +# hash` does not exist as a standalone binary, so the hash is produced by the image itself. Falling back +# to plaintext rather than failing: a working install with a warning beats no install at all, and the +# token is only reachable over loopback. +echo "==> Hashing it" +if VAULTWARDEN_ADMIN_TOKEN_HASH="$(printf '%s' "$VAULTWARDEN_ADMIN_TOKEN" \ + | docker run --rm -i vaultwarden/server:latest /vaultwarden hash --preset owasp 2>/dev/null \ + | grep -oE '\$argon2[^ ]*' | head -1)" && [ -n "$VAULTWARDEN_ADMIN_TOKEN_HASH" ]; then + # Compose reads `$` as interpolation, so a literal Argon2 hash must have every `$` doubled or the + # container receives a mangled token and rejects every admin login with no useful error. + VAULTWARDEN_ADMIN_TOKEN_HASH="${VAULTWARDEN_ADMIN_TOKEN_HASH//\$/\$\$}" +else + echo " (could not hash — falling back to a plaintext token, which Vaultwarden will warn about)" + VAULTWARDEN_ADMIN_TOKEN_HASH="$VAULTWARDEN_ADMIN_TOKEN" +fi + +# ── Render and start ────────────────────────────────────────────────────────────────────────────────── +echo "==> Preparing $SERVICE_DIR" +export OFFICER_UID OFFICER_GID OFFICER_TZ VAULTWARDEN_PORT VAULTWARDEN_ADMIN_TOKEN_HASH +envsubst '${OFFICER_UID} ${OFFICER_GID} ${OFFICER_TZ} ${VAULTWARDEN_PORT} ${VAULTWARDEN_ADMIN_TOKEN_HASH}' \ + < "$TEMPLATE_DIR/docker-compose.yaml" > "$SERVICE_DIR/docker-compose.yaml" + +echo "==> Starting the container" +docker compose --project-directory "$SERVICE_DIR" up -d + +echo "==> Waiting for Vaultwarden to answer" +for i in $(seq 1 60); do + code="$(curl -s -o /dev/null -w '%{http_code}' "http://127.0.0.1:${VAULTWARDEN_PORT}/alive" || true)" + if [ "$code" = "200" ]; then echo " up after ${i}s"; break; fi + if [ "$i" = "60" ]; then + echo "error: Vaultwarden did not answer on 127.0.0.1:${VAULTWARDEN_PORT} within 60s" >&2 + echo " check: docker compose --project-directory '$SERVICE_DIR' logs" >&2 + exit 1 + fi + sleep 1 +done + +# The URL is all the platform stores. The admin token is deliberately NOT returned: the vault sidecar +# proxies the Bitwarden protocol and never needs it, and a secret the platform does not hold is a secret +# it cannot leak. It is in $TOKEN_FILE for the owner. +echo "OFFICER_RESULT_URL=http://127.0.0.1:${VAULTWARDEN_PORT}" +echo "==> Done. Admin token: $TOKEN_FILE (0600, not stored by Officer)"