Files
platform/docs/secret-store.md
T
pastilhasandClaude Opus 5 27aa72d923 finish the offscale sweep: two real remnants and eleven stale names
Asked for a thorough check that nothing of offscale is left in the platform. The
extraction was sound — no router, no permission entry, no app-store entry, no
screen, no panel app, no env var anything reads. Two things were genuinely wrong,
and both were in setup rather than in the server.

── The secret store minted a key for a plugin ──

`officer-setup` bootstrapped two purposes: `jwt` and `headscale`. The second was a
key for a PLUGIN, under a name the rename had already replaced, on a machine where
the plugin is not installed and nothing would ever ask for it. offscale encrypts
under `offscale`, and `getKey` creates on demand — which is precisely what lets
core know nothing about plugins. Now core bootstraps `jwt` alone.

── `.env.example` documented variables for a deleted router ──

HEADSCALE_URL, HEADSCALE_API_KEY and HEADSCALE_USER were described as driving
`/api/vpn`. That router was deleted on 2026-08-14 and nothing has read any of them
since. Left as a note saying where the tailnet's credentials live now, because
somebody with an older `.env` will otherwise wonder why setting them does nothing.

── Names that outlived the thing ──

Eleven comments referred to the plugin's route, panels, console or server registry
as "headscale": three sidecar config files describing their own shape by pointing
at it, two screens citing `/headscale` as the example of a Workspace/Panel route,
the terminal's note about the Console, and the photos types. Also `catalogue.ts`,
which still claimed headscale "is CORE now", and `working-on-officer.md`, which
told the reader a core install runs six processes including `officer-headscale`.

Upstream references are deliberately untouched: "Headscale's API", "the Headscale
admin credential", "is this a Headscale server?" all name the thing the plugin
drives, and renaming those would make the sentences false.

`docs/secret-store.md` argued headscale must be core because the tailnet is the
perimeter. Marked superseded rather than deleted, with the distinction that
settles it: the perimeter is the tailnet EXISTING, which machine-setup still
arranges on a core install — not the platform shipping the screen that
administers it.

Verified by counting: live code, setup provisioning, env references, permission
entries, app-store entries, hono mounts, dock defaults, screens and panel apps all
zero. What is left is generated files (Plugins.gen, plugin-schemas.gen), the
bundled catalogue entry, test fixtures using "offscale" as an example app name,
and history.

tsgo clean, frontend builds, 873 pass / 7 fail unchanged, setup scripts parse.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-16 11:24:23 +00:00

14 KiB
Raw Blame History

The secret store

Status: BUILT 2026-08-13. src/databases/officer_db/src/secret-store.ts, with jwt.ts and crypto.ts reading from it and officer-setup.sh section 7 bootstrapping it. Rotation is NOT built — the schema carries retired_at and the API exposes retiredKeys(), but nothing retires or re-encrypts yet.

A small SQLite database holding every encryption and signing key the platform uses. It replaced VAULT_STORE_KEY and JWT_SECRET in .env, and it is the facility a plugin uses instead of inventing its own.

One change from the design below: keys are per PURPOSE, not one key for everything. The original plan moved a single at-rest key into the store. What shipped gives offscale, wallet, photos, jellyfin, invoiceshelf, vault and service-connections a key each, so one leaked key opens one plugin's columns rather than all seven. jwt is the eighth. A core install bootstraps one — jwt — and every other purpose is created when its owner first asks.

(It bootstrapped headscale as well until 2026-08-16. That was a key for a plugin, minted under a name the rename had already replaced, on machines where the plugin was not installed and nothing would ever ask for it. getKey creates on demand, which is what makes core needing to know nothing about plugins work at all.)


What is wrong with today

Nothing is insecure. The separation is already right — the thing worth keeping is stated first so it is not lost in a refactor:

Secrets live in Postgres. The key that opens them does not.

That is why officer_db/src/crypto.ts reads VAULT_STORE_KEY from the environment, and it is what makes keys.ts:26 true: "a stolen database dump is useless without .env, a stolen .env is useless without the passphrase".

What is wrong is narrower, and it is about blast radius across processes.

.env sits in the repository root, and Bun auto-loads it. ecosystem.config.cjs says so in as many words — it is the reason the Anthropic credential was moved out of the main process. So today VAULT_STORE_KEY is present in the environment of all twenty pm2 processes. officer-music holds the key that decrypts wallet seed envelopes. Anything that can read /proc/<pid>/environ for those processes has it, and nineteen of them have no reason to.

The second problem is that changing the key is currently unrecoverable rather than an operation. See Rotation.


What the key actually protects

Worth listing, because it is wider than the name suggests. Everything below is AES-256-GCM ciphertext in Postgres, encrypted through officer_db/src/crypto.ts with a key derived as SHA-256(VAULT_STORE_KEY):

column what it is
headscale_servers.api_key a Headscale admin credential — the schema notes it "can delete every node on a tailnet"
service_connections.secret every upstream credential the app store stores: gitea, memos, slskd, transmission
jellyfin_servers.access_token Jellyfin session token
wallets.config node credentials — macaroon, rune, LNDHub password, NWC URI. Spending authority
wallets.seed_envelope a BIP39 mnemonic, already sealed under an owner passphrase, encrypted again with this key

decryptSecret throws when the key does not verify, so a wrong key is not a degraded mode — it is every one of those becoming unreadable at once.

The seed envelope is the only one protected by a second, independent secret (the owner passphrase, never persisted). Everything else in that table has exactly one lock.


Decisions

1. The store is SQLite, in the install, outside Postgres

Keys cannot live in the database they unlock. A dump would then contain both the ciphertext and the thing that opens it, and the property quoted at the top stops being true. Encrypting the key with a second key only moves the question — eventually exactly one secret has to be readable without any other secret, and the only real decision is where it lives.

SQLite rather than a flat file, for one reason that is not secrecy: rotation needs key versions. A rotation has to decrypt with the old key and re-encrypt with the new, and an interrupted rotation needs both to still exist. That is a table with id, purpose, key, created_at, retired_at, and it is awkward as an environment variable or a single-value file. Concurrent access from several sidecars is the second reason; SQLite's locking is the part a hand-rolled file store gets wrong.

2. It is NOT encrypted at rest — and that decision changed shape

As built, the file IS the secret. Keys are stored as they are used, with no second key unlocking them, because a key sitting beside the store it opens buys nothing: whoever can read one can read the other. The boundary is 0700 on the directory, 0600 on the file, owned by the service user.

That answers open question 4 below — nothing stays outside, and .env holds no secret at all.

The original reasoning for encrypted-values-in-a-plaintext-file is kept below because the SQLCipher finding is still true and still the reason whole-file encryption is not on the table.

The original note

Checked rather than assumed, because PRAGMA key appears to work and does not:

$ bun --eval 'db.exec("PRAGMA key = \"supersecret\""); … insert …'
  read without key: THE-SECRET-VALUE
  strings enc.db | grep THE-SECRET-VALUE  ->  found

Stock SQLite silently ignores unknown pragmas, so PRAGMA key succeeds, encrypts nothing, and the value sits in the file in plaintext. bun:sqlite ships stock SQLite 3.53.0, not SQLCipher.

Whole-file encryption therefore needs SQLCipher, which means a native module — and this project already knows what one of those costs, since node-pty has no Linux prebuild and compiles from source on every machine.

So the store holds encrypted values in an unencrypted file, the same shape as the Postgres columns. What leaks is metadata: which purposes have keys, and when they were rotated. That is an acceptable trade and it is written down here so nobody later assumes the file is opaque.

[open] SQLCipher, if the native-dependency cost ever becomes worth paying.

3. Where the file goes

$OFFICER_ROOT/secrets/officer-keys.db — a sibling of platform/ and data/, decided 2026-08-13.

Not in $OFFICER_ROOT/data/. That directory holds managed homes and attachments — it is the one people back up. A key store that travels in the same tarball as a database dump rebuilds the exact problem this design exists to avoid.

The setup script says so out loud when it creates the store, because "back this up, but not next to the other thing you back up" is not a rule anyone infers.

4. One secret remains outside — none does

Answered 2026-08-13: no secret remains in .env. The store file is the secret, per decision 2.

The point of the exercise still holds, and it was always about blast radius rather than secrecy: N secrets in twenty process environments becomes a file read on demand by the few processes that need it. .env is auto-loaded by bun into every pm2 process, so a key there is readable from /proc/<pid>/environ of twenty processes — officer-music held the key that decrypts wallet seed envelopes. A file opened by the two or three processes that actually use a key does not.

5. What moves in

  • VAULT_STORE_KEY — split into one key per purpose, rather than moved. See the status note at the top: the table above is seven unrelated things, and one key for all of them meant one leak opened all of them.

  • JWT_SECRET — a signing key rather than an encryption key, but it has the same properties: must survive restarts, must never be regenerated silently, and benefits from versioning during a rotation. Leaving one in a store and one in .env would be the scattering this is meant to end.

  • The anthropic proxy secret, purpose anthropic-proxy. Agreed 2026-08-12. Neither an encryption key nor a signing key — a bearer credential, generated once by ensureProxySecret and presented by officer-agent to officer-anthropic-proxy on 127.0.0.1. It qualifies on the same three properties: generated once, shared between two processes, fatal to regenerate silently.

    It is in the store for a sharper reason than the other two, though. It is not in .env today — it is in $DATA_PATH/sidecar/claude-state.json, mixed in with session records. That is the one location decision 3 rules out by name: DATA_PATH is what people back up, so the secret already travels in the same tarball as the data it protects.

    Naming. It is called ANTHROPIC_API_KEY in ensureAnthropicEnv, and that name is wrong in both halves — it is not Anthropic's and it is not an API key. Anthropic's real credential is the OAuth token in ~/.claude/.credentials.json, which the proxy swaps this one for on the way out. Our name for it is anthropic-proxy-secret everywhere we control.

    The exception is the last line before the spawn. claude reads the variable ANTHROPIC_API_KEY and format-checks the sk-ant-api03- prefix, so both are the CLI's contract rather than ours and both stay. That one assignment keeps the CLI's name, with a comment saying why.


Core, and plugins

The store is core infrastructure, created at first boot. It is not a side effect of installing any one sidecar — it exists on a machine that installs nothing, so that a plugin installed in six months finds it already there.

The core is what ecosystem.light.config.cjs runs today — officer, officer-anthropic-proxy, officer-agent, officer-opencode, officer-pty — plus officer-headscale.

Superseded 2026-08-15. The tailnet left the repository and is the offscale plugin; core is the five above. The argument below was not refuted — it was overtaken. What settled it: the perimeter is the tailnet EXISTING, which is a machine-setup concern, and that is not the same as the platform shipping the screen that administers it. machine-setup.sh still walks you onto a tailnet on a core install; only the admin UI became optional. Kept because the reasoning is what a future "should X be core?" argument needs.

Headscale is core for a stated reason rather than by preference: CLAUDE.md says the tailnet is the perimeter — origin checking was removed on 2026-08-13 precisely because the tailnet is what stands in its place, so the tailnet is now load-bearing rather than one layer of two. A security model that rests on the tailnet cannot treat administering the tailnet as an optional extra. Vaultwarden and the wallet are not load-bearing that way — nothing else stops working without them — so they become plugins.

Moving headscale into the light profile also removes it from the app store automatically: catalogue.test.ts asserts the catalogue equals full − light, so the test fails until the entry is deleted. That derivation is doing its job and should not be worked around.

Headscale is then the store's first user, not its creator — headscale_servers.api_key is the first core credential needing a key.


The contract

What a plugin gets, and is bound by. To be written properly when the first one uses it; the shape is:

  • Ask for a key by purpose, not by name. getKey('vault') returns the active key for that purpose, creating one on first use.
  • Never hold it. Read it at the point of use. A key cached in a long-lived process is the process-environment problem in a different container.
  • Never write to another plugin's purpose. Same rule service_connections already has for rows.
  • Tolerate rotation. A key may change between two calls. Anything that decrypts must be prepared to be handed the retired key for data written before a rotation.

Rotation

The feature that makes the store worth building, and the reason versions exist.

Today, changing VAULT_STORE_KEY is not an operation — it is data loss. Every column above becomes unreadable, and for wallets.seed_envelope that is unrecoverable: the owner passphrase does not help, because it opens the inner envelope and the outer one is gone. Unless the mnemonic was written down offline, the coins are gone with it.

Rotation turns that into a supported action:

  1. Mint a new key for the purpose, leaving the old one in the store as retired.
  2. For every ciphertext column belonging to that purpose: decrypt with the retired key, re-encrypt with the new one.
  3. Retire the old key only when every row has moved.

Two properties it must have, both learned from the failure it replaces:

  • Transactional. A half-rotated table is worse than either end state, because nothing afterwards can tell which rows are which.
  • Verify before writing. Every row must decrypt with the retired key before anything is written. A key that is already wrong should fail loudly on row one rather than produce a second layer of unreadable data.

[open] Whether rotation is a UI action, a CLI command, or both. It is a long operation on a large wallet table and it cannot be interrupted safely, which argues for something that reports progress.


What this does not change

  • Secrets stay in Postgres. This moves the keys, not the data.
  • crypto.ts's interface stays — it did not. Per-purpose keys mean the purpose has to be named at the call site, so it is encryptSecret('headscale', plaintext) now and all seven query modules were touched. That was the cost of the split, and it is worth stating plainly because this line originally promised the opposite.
  • The owner passphrase on wallet seeds is untouched and stays out of every store. Two independent secrets is the property that makes a stolen .env insufficient, and it survives this design.

Open questions

  1. Where the file lives, given it must not be swept up by a backup of data/.
  2. Whether the store's own key stays in .env or moves to a file read on demand.
  3. Whether rotation is UI, CLI, or both — and how it reports progress on a table that takes minutes.
  4. SQLCipher, and whether whole-file encryption is ever worth a second native dependency.
  5. What happens to a plugin's keys when it is uninstalled. The app store already decided that uninstalling never deletes data; the same answer probably applies, but "probably" is not a decision.