diff --git a/docs/servers.md b/docs/servers.md index 31070584..68ea5ce4 100644 --- a/docs/servers.md +++ b/docs/servers.md @@ -4,59 +4,105 @@ Every machine the owner runs, how to reach it, and what of Officer is on it. Kep orchestrator ("pull and update every platform", "restart officer everywhere", "which host is behind") has one place to read rather than a fact re-derived per session. -**Started 2026-08-21.** Everything below was verified from `officer-dev` on that date by SSH and -`pm2 jlist`, except where marked assumed. Commit SHAs are a snapshot and go stale immediately — they -are here to show which checkout is live, not as a status board. +**Verified 2026-08-21** from `officer-dev` by SSH and `pm2 jlist`, except where marked assumed. ## How access works -All of it is over the tailnet. There is **no `~/.ssh/config`** — names resolve by MagicDNS and the -login is key-based, so `ssh ` as `pastilhas` is the whole story: +All of it is over the tailnet, key-based, no `~/.ssh/config`, no passwords, no jump hosts, no ports. + +**The login user is NOT always `pastilhas`.** It is per host and it is in the table below. `deadpool` runs +as `br` and always will — the owner's key is authorised for that account. An orchestrator that hard-codes +a username fails on that host and will fail on the next one like it. ```bash -ssh alpha -ssh fofos +ssh alpha # pastilhas, implied +ssh br@deadpool-rezio-bridge # not implied ``` -Nothing here needs a password, a jump host or a port. A host that refuses port 22 is not -misconfigured — see "Not SSH-reachable" below. - ## Officer platforms -| Host | Tailnet IP | Live checkout | Runs `officer` | Git remote | -| --- | --- | --- | --- | --- | -| `officer-dev` | 100.64.0.13 | `~/officer-dev/platform` | yes (this host) | `git@gitea.officer.dev:officerdev/platform.git` | -| `alpha` | 100.64.0.1 | `~/officerdev/platform` | yes | `git@gitea.officer.dev:officerdev/platform.git` | -| `fofos` | 100.64.0.17 | `~/officerdev/platform` | yes | `https://gitea.officer.dev/officerdev/platform.git` | -| `headscale` | 100.64.0.2 | `~/dockers/officerdev/platform` | no | `ssh://git@gitea.pastilhas.dev:2222/officerdev/platform.git` | -| `omega` | 100.64.0.3 | `~/projects/officerdev/platform` | no | none set | +Seven, all running. -Three things an orchestrator has to handle, all verified and none of them cosmetic: +| Host | Tailnet IP | Login | Checkout | Domain | +| ---------------------------- | ----------- | ----------- | ----------------------------------- | --------------------------------- | +| `officer-dev` | 100.64.0.13 | `pastilhas` | `~/officer-dev/platform` | `platform.officer.dev` | +| `alpha` | 100.64.0.1 | `pastilhas` | `~/officerdev/platform` | `officer.pastilhas.dev` | +| `fofos` | 100.64.0.17 | `pastilhas` | `~/officerdev/platform` | `officer.fofosfluffybastards.com` | +| `macbookpro` | 100.64.0.9 | `pastilhas` | `~/projects/officer-suite/platform` | `macbookpro.pastilhas.dev` | +| `edge-pertento-bridge` | 100.64.0.15 | `pastilhas` | `~/officerdev/platform` | `edge.pertento.ai` | +| `production-pertento-bridge` | 100.64.0.12 | `pastilhas` | `~/officerdev/platform` | `officer.pertento.ai` | +| `deadpool-rezio-bridge` | 100.64.0.21 | **`br`** | `/home/br/officerdev/platform` | `officer.rezio.dev` `[pending]` | -- **The checkout path differs per host.** There is no single path to hard-code. Take it from PM2 — - `pm2 jlist` gives the `officer` process's `pm_cwd`, which is the checkout that is actually serving. -- **`alpha` has a SECOND, stale checkout** at `~/dockers/officer.dev/platform` (same remote, different - HEAD). Pulling the wrong one updates nothing and looks like success. PM2's `pm_cwd` is the tiebreak. -- **The remotes are not the same URL.** `fofos` pulls over HTTPS and `headscale` points at a different - Gitea entirely (`gitea.pastilhas.dev:2222`). An orchestrator that assumes SSH to `gitea.officer.dev` - breaks on two of five. +Only `platform.officer.dev` is public (77.42.39.155). The rest resolve to tailnet addresses and are +reachable from inside only. -`headscale` and `omega` hold a checkout but run no `officer` process (no PM2 on either). Whether those -are retired, staging or half-installed is **not established** — ask before treating them as targets. +### Names that disagree with themselves + +Three hosts answer to more than one name, and `tailscale status` is not always the one in use: + +| In `tailscale status` | Also | The machine's own hostname | +| ----------------------- | ---------------------------- | -------------------------- | +| `edge-bridge` | `edge-pertento-bridge` | `edge-pertento` | +| `gitea-bridge` | `production-pertento-bridge` | `PPROD` | +| `deadpool-rezio-bridge` | — | `deadpool` | + +The middle column is what to use. `edge-bridge` and `gitea-bridge` are the old names and are being +retired; anything reading `tailscale status` will still see them. + +### Things that will catch out an orchestrator + +All verified, none cosmetic. + +- **The checkout path differs per host** — four distinct shapes across seven machines. Take it from PM2: + the `officer` process's `pm_cwd`. `alpha` carries a SECOND, stale checkout at + `~/dockers/officer.dev/platform`, and pulling that one succeeds while changing nothing. +- **`pm2` is not on a non-interactive PATH.** It lives under nvm, and nvm's setup is in `.zshrc`, which a + _login_ shell does not read. Use `ssh '$SHELL -ic "pm2 jlist"'`. `-lc` is **not** enough — it + reports "command not found" on machines where pm2 is plainly running, which cost two wrong conclusions + on 2026-08-21. +- **On `macbookpro`, do not help the PATH.** Exporting one to reach `bun` breaks `pm2`, which is only on + the interactive shell's. `$SHELL -ic` and nothing else. +- **The git remotes differ.** `fofos` pulls over HTTPS; the others over SSH. Never assume a URL — pull in + place. +- **A "dirty tree" means TRACKED changes.** `git status --porcelain` counts untracked files, and a stray + script a fast-forward could never touch is not a reason to skip a host. Use `--untracked-files=no`. +- **`macbookpro` is a laptop.** It may be asleep. Unreachable is a skip with a note, not a failed deploy. + +## Not platforms — ignore for now + +A checkout may be present; no `officer` process runs on any of these. Whether they are retired, staging +or half-installed is **not established** — ask before treating them as targets. + +| Host | Tailnet IP | +| ----------- | ---------- | +| `headscale` | 100.64.0.2 | +| `omega` | 100.64.0.3 | +| `gateway` | 100.64.0.4 | ## Other machines on the tailnet -- `gateway` (100.64.0.4) — reachable over SSH as `pastilhas`, no platform checkout, no PM2. -- **Not SSH-reachable** (port 22 refused): `edge-dns` (100.64.0.8), `edge-bridge` (100.64.0.15), - `gitea-bridge` (100.64.0.12), `officer-bridge` (100.64.0.5). Assumed to be service-only - containers rather than hosts to log into — not confirmed. -- **Client devices, not servers:** `macbookpro`, `iphone16`, `andres-ipad`, `pixel9a`, plus other - people's devices in the tailnet (`diogos-ipad`, `iphone-diogo`, `iphone-bruno`, +- **Not SSH-reachable** (port 22 refused): `edge-dns` (100.64.0.8), `officer-bridge` (100.64.0.5). + Assumed service-only containers — not confirmed. `edge-bridge` was listed here until 2026-08-21 and was + wrong: its SSH was simply down at the time of checking. +- **Client devices, not servers:** `iphone16`, `andres-ipad`, `pixel9a`, plus other people's devices in + the tailnet (`diogos-ipad`, `iphone-diogo`, `iphone-bruno`, `fedras-iphone`, `guilhermes-macbook-pro`, `macbook-pro-de-joo`). +## Known wrong, waiting on someone else + +- **`officer.rezio.dev` resolves to 100.64.0.4** — that is `gateway`, not `deadpool` (100.64.0.21). Not + merely unavailable: pointed at a different machine, so it will serve the wrong box the moment gateway + answers on 443. Ports 80 and 443 are closed on `deadpool`, so nothing fronts it yet. Waiting on the + domain owner. + ## Keeping this list true -It is a running list, so it is worth saying what makes it rot: a new server, a moved checkout, a -platform retired, or a remote changed. Re-verify with the same three commands the list was built from -— `tailscale status`, `pm2 jlist` for the `officer` entry's `pm_cwd`, and `git -C remote get-url -origin`. +It rots when a server is added, a checkout moves, a platform is retired, a remote changes or a login +differs. Re-verify with the four commands it was built from: + +```bash +tailscale status +ssh @ '$SHELL -ic "pm2 jlist"' # the officer entry's pm_cwd +ssh @ 'git -C remote get-url origin' +getent hosts +``` diff --git a/prompts/deploy-commander.md b/prompts/deploy-commander.md index 9f18b572..d28fd125 100644 --- a/prompts/deploy-commander.md +++ b/prompts/deploy-commander.md @@ -49,8 +49,13 @@ at, and guessing is how a database gets a schema it was not ready for. **Read `docs/servers.md`.** Do not take a list of hosts from this file — there is deliberately none here, because a copy rots the day a server moves and the copy is always the one somebody trusts. -That document holds every machine, its tailnet name, its checkout and whether a platform runs on it. Read -it at the start of every deploy, and treat the hosts marked as having no platform as out of scope. +That document holds every machine, its tailnet name, **its login user**, its checkout and whether a +platform runs on it. Read it at the start of every deploy, and treat the hosts marked as having no +platform as out of scope. + +**The login is not always `pastilhas`.** `deadpool-rezio-bridge` runs as `br`, permanently, and more will. +Take the user from the table — an orchestrator that hard-codes one fails on that host silently, because +`Permission denied (publickey)` looks like a broken key rather than a wrong username. Four things that will catch you out, all verified rather than assumed: @@ -71,8 +76,10 @@ Four things that will catch you out, all verified rather than assumed: Per host, in this order. Do them **one host at a time** and keep a record as you go — a report you assemble at the end from memory is a report that will be wrong about the host that failed. -1. **Refuse a dirty tree.** `git status --porcelain` — if anything is uncommitted, skip this host and say - so. Never discard someone's work to ship a release. +1. **Refuse a dirty tree — TRACKED changes only.** `git status --porcelain --untracked-files=no`. Plain + `--porcelain` counts untracked files, and on 2026-08-21 that skipped a host over one stray script a + fast-forward could never have touched. Untracked files get a note in the report, not a skip. Never + discard someone's work to ship a release. 2. `git pull --ff-only`. A non-fast-forward means that checkout has diverged: skip and report. 3. **Update the plugins.** `bun run plugins:update`. Sequential by design, one plugin failing does not stop the rest, and it exits non-zero if any did — so a failure is something you report rather than