Files
platform/docs/servers.md
T
pastilhas b0ceb52bf1 Move both Pertento hosts to deliberately-skipped deploy targets
Owner's call, not an incident on either box. Details kept in the
skipped section so re-adding them is a move, not a re-derivation.
2026-09-08 22:10:18 +01:00

189 lines
12 KiB
Markdown

# Servers — the running list
Every machine the owner runs, how to reach it, and what of Officer is on it. Kept here so an
orchestrator ("pull and update every platform", "restart officer everywhere", "which host is behind")
has one place to read rather than a fact re-derived per session.
**Verified 2026-08-21** from `officer-dev` by SSH and `pm2 jlist`, except where marked assumed.
## How access works
All of it is over the tailnet, key-based, no `~/.ssh/config`, no passwords, no jump hosts, no ports.
**The login user is NOT always `pastilhas`.** It is per host and it is in the table below. `deadpool` runs
as `br` and always will — the owner's key is authorised for that account. An orchestrator that hard-codes
a username fails on that host and will fail on the next one like it.
```bash
ssh alpha # pastilhas, implied
ssh br@deadpool-rezio-bridge # not implied
```
## Officer platforms
Six, all running. **Verified 2026-08-27**, minus the two Pertento hosts moved to "Deliberately skipped" on
2026-09-08 — every remaining row below was read off the machine, not recalled.
| Host | Tailnet IP | Login | Port | Checkout | Domain |
| ---------------------------- | ----------- | ----------------------------------------- | -------- | ----------------------------------- | --------------------------------- |
| `officer-dev` | 100.64.0.13 | `pastilhas` | 9000 | `~/officer-dev/platform` | `platform.officer.dev` |
| `alpha` | 100.64.0.1 | `pastilhas` | 9000 | `~/officerdev/platform` | `officer.pastilhas.dev` |
| `fofos` | 100.64.0.17 | `pastilhas` | 9000 | `~/officerdev/platform` | `officer.fofosfluffybastards.com` |
| `macbookpro` | 100.64.0.9 | `pastilhas` | **9010** | `~/projects/officer-suite/platform` | `macbookpro.pastilhas.dev` |
| `macbruno` | 100.64.0.22 | **`brunoalcadapadezmonteirodaconceicao`** | 9000 | `~/officerdev/platform` | — (none yet) |
| `deadpool-rezio-bridge` | 100.64.0.21 | **`br`** | 9000 | `/home/br/officerdev/platform` | `officer.rezio.dev` `[pending]` |
**THE PORT IS NOT ALWAYS 9000.** Two hosts run on 9010 and it is in `OFFICERDEV_PORT` in each checkout's
`.env`. This is not a footnote: on 2026-08-27 a health check against 9000 on `production-pertento-bridge`
hit one of Pertento's own apps, which answered `403`, and that was read as Officer failing after a
deploy. Take the port from the `.env`, never from habit.
**`production-pertento-bridge` also runs eight processes that are NOT Officer** — `Dashboard`, `Tracking`,
`Uptime` and five `Crons *`. They are production-critical and belong to Pertento. Anything sweeping over
`pm2 list` must filter to `officer` and `officer-*`, and a restart there must name its target.
Only `platform.officer.dev` is public (77.42.39.155). The rest resolve to tailnet addresses and are
reachable from inside only.
## `pm2` is not on the PATH of a non-interactive shell
`ssh host 'pm2 jlist'` runs a NON-interactive, NON-login shell. zsh reads `~/.zshenv` for those and reads
neither `~/.zshrc` nor `~/.zprofile` — so a pm2 installed by nvm or Homebrew is invisible to an
orchestrator even though the daemon is running and healthy.
**This read as "pm2 is not installed" three times on 2026-08-27**, and twice sent someone diagnosing a
machine with nothing wrong with it. `claude` on a fresh install has the same shape.
Fixed at the source on the two hosts where it bit — `production-pertento-bridge` (nvm) and `macbruno`
(Homebrew at `/opt/homebrew`, which is on no default PATH on Apple Silicon) — by appending a
marker-guarded block to `~/.zshenv`. Everything else already worked.
If a command comes back empty over SSH, look for the binary on disk before concluding it is absent:
```bash
ssh host 'ls /opt/homebrew/bin/pm2 /usr/local/bin/pm2 ~/.nvm/versions/node/*/bin/pm2 2>/dev/null'
```
## Deliberately skipped
A host here is NOT a deploy target. A rollout names it as skipped and moves on — it is not a failure,
and it must not be reported as one.
**Both Pertento hosts, since 2026-09-08** — `production-pertento-bridge` and `edge-pertento-bridge`. Owner's
call, not a fault found on either box. Details kept here rather than dropped, so putting them back is a
one-line move, not a re-derivation:
| Host | Tailnet IP | Login | Port | Checkout | Domain |
| ----------------------------- | ----------- | ------------ | -------- | ------------------------- | ----------------------- |
| `production-pertento-bridge` | 100.64.0.12 | `pastilhas` | **9010** | `~/officerdev/platform` | `officer.pertento.ai` |
| `edge-pertento-bridge` | 100.64.0.15 | `pastilhas` | 9000 | `~/officerdev/platform` | `edge.pertento.ai` |
`edge-pertento-bridge` was skipped once before, from 2026-08-23 to 2026-08-24 — but that was downtime, not
a choice, and it came back as a deploy target. This is a different reason and there is no return date.
It was down long enough to matter last time, so expect it to be behind whenever it returns: when it came
back on 2026-08-24 it was 215 commits back at `2026.08.22.02`, with no `roles` table and no commanders. A
host coming off this list needs its schema brought forward before it is treated as ordinary.
On 2026-08-23 the deploy agent spent four attempts and up to 60s per try on this host before reporting
it unreachable — TCP on 22 accepted, then no SSH banner. That work was correct and entirely wasted, which
is what this section exists to prevent: a host that is known to be down should cost a line in a report,
not a retry budget.
### Names that disagree with themselves
Three hosts answer to more than one name, and `tailscale status` is not always the one in use:
| In `tailscale status` | Also | The machine's own hostname |
| ----------------------- | ---------------------------- | -------------------------- |
| `edge-bridge` | `edge-pertento-bridge` | `edge-pertento` |
| `gitea-bridge` | `production-pertento-bridge` | `PPROD` |
| `deadpool-rezio-bridge` | — | `deadpool` |
The middle column is what to use. `edge-bridge` and `gitea-bridge` are the old names and are being
retired; anything reading `tailscale status` will still see them.
### Things that will catch out an orchestrator
All verified, none cosmetic.
- **The checkout path differs per host** — four distinct shapes across seven machines. Take it from PM2:
the `officer` process's `pm_cwd`. `alpha` carries a SECOND, stale checkout at
`~/dockers/officer.dev/platform`, and pulling that one succeeds while changing nothing.
- **`pm2` is not on a non-interactive PATH.** It lives under nvm, and nvm's setup is in `.zshrc`, which a
_login_ shell does not read. Use `ssh <host> '$SHELL -ic "pm2 jlist"'`. `-lc` is **not** enough — it
reports "command not found" on machines where pm2 is plainly running, which cost two wrong conclusions
on 2026-08-21.
- **On `macbookpro`, APPEND to the PATH for `bun`. Never replace it.** This said "do not help the PATH"
until 2026-08-21, and that was half right in a way that cost a deploy step: `pm2` is only on the
interactive shell's PATH, so replacing it breaks `pm2` — but `bun` is at `/opt/homebrew/bin/bun` and
`.zshrc` does not put that directory on the interactive PATH either, so `bun run plugins:update` fails
`command not found` before it runs a line. `export PATH=$PATH:/opt/homebrew/bin` for that one command
shadows nothing: `pm2` stays at `~/.local/bin/pm2` and `node` at `node@22`. Run `pm2` as a separate
command.
- **`$SHELL -ic` on `macbookpro` aliases `ls` to `eza`, which is not installed.** A probe using `ls`
reports a missing file that is plainly there. Use `command ls`, or a non-interactive shell for file tests.
- **The git remotes differ.** `fofos` pulls over HTTPS; the others over SSH. Never assume a URL — pull in
place.
- **A "dirty tree" means TRACKED changes.** `git status --porcelain` counts untracked files, and a stray
script a fast-forward could never touch is not a reason to skip a host. Use `--untracked-files=no`.
- **`macbookpro` is a laptop.** It may be asleep. Unreachable is a skip with a note, not a failed deploy.
- **`bun run plugins:update` restarts sidecars itself**, printing `sidecar: started` per plugin — on
`alpha` that is twelve processes bouncing during the plugins step, before any restart the plan asked for.
Not a fault, but a deploy report that lists only `officer` under "restarted" is understating what moved.
- **`fofos` failed its capabilities sync for three deploys, and it was ownership.** Fixed 2026-08-21.
Two errors that read as unrelated problems — `dubious ownership` from git, and
`EACCES … mkdir '…/capabilities/affordances/officerdev'` from the gitea plugin — were one cause in two
dialects: fifty paths under that checkout were owned by `root`, including `.git` and `affordances`, all
stamped `19 Aug 21:43`. Something was run with `sudo` and left them behind. Mode `755` is why the split
looked odd: everything could be READ, so git saw the repo and refused it on ownership while the plugin
saw `affordances` and could not write into it.
One `stat` settled it, and that is the general lesson — `stat` prints the owner rather than inferring
anything, so it either proves ownership is the cause or eliminates it, in one command. Fixed with
`sudo chown -R pastilhas:pastilhas ~/officerdev/capabilities`; verified after by a `mkdir` into the
directory that used to refuse one.
**A `safe.directory` entry was deliberately NOT used**, and that call is worth keeping. It would have
silenced the git message and left the `EACCES` untouched — one symptom of a two-symptom cause hidden,
with the obvious explanation spent.
## Not platforms — ignore for now
A checkout may be present; no `officer` process runs on any of these. Whether they are retired, staging
or half-installed is **not established** — ask before treating them as targets.
| Host | Tailnet IP |
| ----------- | ---------- |
| `headscale` | 100.64.0.2 |
| `omega` | 100.64.0.3 |
| `gateway` | 100.64.0.4 |
## Other machines on the tailnet
- **Not SSH-reachable** (port 22 refused): `edge-dns` (100.64.0.8), `officer-bridge` (100.64.0.5).
Assumed service-only containers — not confirmed. `edge-bridge` was listed here until 2026-08-21 and was
wrong: its SSH was simply down at the time of checking.
- **Client devices, not servers:** `iphone16`, `andres-ipad`, `pixel9a`, plus other people's devices in
the tailnet (`diogos-ipad`, `iphone-diogo`, `iphone-bruno`, `fedras-iphone`,
`guilhermes-macbook-pro`, `macbook-pro-de-joo`).
## Known wrong, waiting on someone else
- **`officer.rezio.dev` resolves to 100.64.0.4** — that is `gateway`, not `deadpool` (100.64.0.21). Not
merely unavailable: pointed at a different machine, so it will serve the wrong box the moment gateway
answers on 443. Ports 80 and 443 are closed on `deadpool`, so nothing fronts it yet. Waiting on the
domain owner.
## Keeping this list true
It rots when a server is added, a checkout moves, a platform is retired, a remote changes or a login
differs. Re-verify with the four commands it was built from:
```bash
tailscale status
ssh <login>@<host> '$SHELL -ic "pm2 jlist"' # the officer entry's pm_cwd
ssh <login>@<host> 'git -C <cwd> remote get-url origin'
getent hosts <domain>
```