platform/docs/deprovision-os-account.md

# Deprovisioning a member's Linux account

**Status:** implemented 2026-08-12 in `src/servers/os-user-deprovision.ts`, called by `deleteUserHandler`.
**Not yet run against a real account** — see "What is still unproven" at the bottom before trusting it.
Written from a manual teardown performed on the production host on 2026-08-11, so the ordering constraints
below are measured rather than reasoned.

The spec is kept as written rather than rewritten in the past tense: it is the reasoning the implementation
has to keep satisfying, and the failure modes it names are still the ones a change would reintroduce.

---

## What happens today

`deleteUserHandler` removes the `users` row and cascades the database. `userdel` never runs. Measured on a
real member (`green`, uid 1002) immediately after deleting them through the UI, before any cleanup:

| | after `deleteUserHandler` |
|---|---|
| `users` row | gone |
| Linux account | alive, uid 1002 |
| Login shell | `id -u` → 1002 — the deleted account still had a working login |
| Rootless Docker | daemon running, `postgres` container `Up 2 hours (healthy)` |
| Home + Docker storage | 454 MB intact |
| linger, `/run/user/1002`, `/etc/subuid`, `/etc/subgid` | all present |

Nothing breaks, which is what makes it dangerous. The account keeps working; only the platform forgets it
exists.

## Why it matters: uid reuse

`useradd` allocates the lowest free uid. Delete a member and the uid is free while their files still carry it,
so the next member created inherits the previous member's home, keys, Docker storage and anything else owned
by that number. By uid, not by any decision anyone made.

This is not hypothetical. `officer_jg` (uid 1001) and `green` (uid 1002) both had login shells pointing at one
home on this host, and the `users` table had no row for `officer_jg` at all — an earlier account for the same
email, deleted from the platform, whose Linux side survived. The adoption rule in `ensureOsUser` was never
bypassed; the account simply outlived the row.

**The invariant this function exists to guarantee:**

> After deprovisioning, no file anywhere is owned by the freed uid **or by any id in its freed subuid range**,
> and no passwd entry, linger marker, runtime directory or process refers to it.

## The subuid half, which is easy to miss

A member's rootless Docker storage is **not** owned by their uid. Container processes map through
`/etc/subuid`, so the files are owned by ids in that range — on this host, `green` had `231072:65536`, and
postgres's data directory was owned by `231141` (231072 + 70, postgres's inner uid in the Alpine image).

`userdel` releases the subuid range along with the uid, and a later account can be allocated the same range.
So a check for "nothing is owned by the freed uid" **passes while hundreds of megabytes are still owned by the
freed subuid range**, and a future member's containers would map onto another member's leftover files.

Any verification has to cover the range, not just the uid.

---

## The sequence

Ordering is load-bearing. Each step explains what breaks if it moves.

### 1. Disable linger, before stopping anything

```
loginctl disable-linger <user>
```

Lingering keeps a systemd user manager alive with no login session. Terminate first and linger can bring it
back; disable first and nothing can re-spawn between the two steps.

*(The manual teardown ran these in the opposite order and worked. This order is specified because it removes a
race rather than because the other one failed.)*

### 2. Terminate the session, then **verify it actually died**

```
loginctl terminate-user <user>
```

**`terminate-user` is not a barrier.** Measured: a `/bin/zsh -i` owned by the member survived it — three hours
old, still running after the session was terminated and `/run/user/<uid>` was removed. `userdel` refuses while
a process owned by the account is alive, so an implementation that trusts `terminate-user` works on a quiet
account and fails on a member who left a shell open, which is the normal case.

Required after terminating:

```
pkill -u <user>            # wait, then re-check
pkill -9 -u <user>         # only if the count is still non-zero
```

with a bounded wait between and a final assertion that the process count is zero. **Do not proceed while it is
not.**

### 3. Sever the data from the uid — *before* releasing it

Two policies. The platform's default is **preserve**:

```
chown -R <service-user>:<service-group> <member-tree>
```

Destroying a member's data because their account was deleted is a separate decision from removing their
access, and the platform has no standing to make it silently. Reassigning ownership severs the uid link while
keeping every byte.

**Destroy** is opt-in, for a deliberate rebuild:

```
rm -rf <member-tree>
```

**Ownership is not the only link.** `confineUserTree` grants the member a NAMED ACL entry on their whole
tree — `u:<uid>:rwx` and a `default:` copy, inherited by everything either party creates. `chown` does not
remove them: they are xattrs rather than ownership, and they store the uid **numerically**. Measured — a
`chown -h -R` to the service user leaves `user:<uid>:rwx` intact on the directory, its children and their
defaults.

So a tree reassigned to the service user still grants the freed uid read and write on every byte, and the next
account allocated that number inherits it: home, SSH keys, `.credentials.json`, transcripts, container
storage. That is the hazard this function exists to prevent, arriving through ACLs instead of ownership.

Severing therefore has two parts:

```
chown -h -R <service-user>:<service-group> <member-tree>
setfacl -R -b <member-tree>          # or -x u:<uid> -x d:u:<uid> to keep the platform's own entry
```

`-b` is the simpler answer for a preserved tree: the service user owns every byte afterwards, so a named
entry granting themselves access is redundant.

**This step must complete before step 4.** That is the one ordering choice the manual teardown got wrong: it
released the uid first and removed the data afterwards, which leaves a window where the uid is free while
files still carry it. If the process dies in that window, the next `useradd` inherits them. Sever first, then
release — the irreversible step goes last, and only once nothing points at it.

### 4. Release the account

```
userdel <user>          # NEVER -r
```

`-r` deletes the home, which contradicts the preserve policy and would make the destroy policy depend on a
flag rather than on an explicit decision. Measured: plain `userdel` removes the passwd, shadow and group
entries **and** the `/etc/subuid` and `/etc/subgid` ranges.

### 5. Verify, and refuse to call it done otherwise

See the checklist below. A deprovision that half-succeeded is worse than one that failed cleanly, because the
uid is free and something still owns files.

---

## Verification: what "clean" means

All of these must hold for the freed uid *and* its freed subuid range:

- `getent passwd <user>` → nothing
- no entry in `/etc/subuid` or `/etc/subgid`
- `find <DATA_PATH> /home -uid <uid>` → nothing
- `find <DATA_PATH> /home -uid <subuid-start> -o ... ` over the freed range → nothing
  *(a range scan, not a single id — the mapped ids are spread across it)*
- `/var/lib/systemd/linger/<user>` absent
- `/run/user/<uid>` absent
- no processes owned by the uid
- **no ACL entry naming the uid** anywhere under `DATA_PATH` — `getfacl -R -n` and look for
  `user:<uid>:` / `default:user:<uid>:`. Ownership checks cannot see these, and `chown` does not clear them.

`scripts/assert-uid-free.sh` implements exactly this, deliberately **outside** the function: a checker the
implementation calls is a restatement of its own beliefs, not an audit. Two modes, and the split matters —

```
./scripts/assert-uid-free.sh --capture green                    # BEFORE: prints "green 1001 165536 65536"
sudo DATA_PATH="$DATA_PATH" ./scripts/assert-uid-free.sh --check green 1001 165536 65536   # AFTER: exit 1 unless clean
```

**Pass `DATA_PATH` through explicitly.** sudo's `env_reset` drops it, so the plain `sudo ./assert-uid-free.sh`
this used to say fell back to a hardcoded default — and every check here reports `ok` on finding nothing, so
a wrong root reports `CLEAN — uid safe to reissue` without having looked at a single member tree. The ACL
check is the one that failed silently and completely, because it is the only one scoped to `DATA_PATH` alone.
The script now refuses to run when a search root is missing rather than passing vacuously.

The range has to be captured **before** deletion, because `userdel` removes the `/etc/subuid` entry with the
account. After that there is no way to ask what range it held — and a check that silently skips that half is
the exact failure this section exists to prevent.

**The subuid check passes vacuously on most accounts, and that is a trap.** Container files are owned by a
mapped id only when a process inside the container runs as a NON-root user; an image whose files are root-owned
maps to the member's own uid and leaves nothing in the range. Measured on green after a night of real use —
`claude` installed, images pulled, transcripts written — the range check found **zero** files and passed
without testing anything.

To build a specimen that actually exercises it, run a container whose process writes as a non-root user. The
`postgres:18-alpine` case from the same night is the natural one: its entrypoint drops to uid 70, and the data
directory came out owned by `subuid_start + 70` on the host. Verify the range check *fails* on that tree before
trusting it to pass on a cleaned one.

**Trap for the verifier:** do not use `sudo -u <user> …` to check anything after step 2. Creating a session
starts a user manager and recreates `/run/user/<uid>`, so the check would undo the step it is verifying.

---

## Behaviour requirements

**Idempotent.** Every step tolerates already-done. Re-running on a clean box is a no-op, and re-running after a
partial failure completes it. The delete handler should be able to call it, fail, and have an operator press
retry.

**Never throws; returns a result.** Same posture as `provisionOsAccount`, `provisionSshAccess` and
`provisionRootlessDocker`.

**But a failed deprovision is not the same as a failed provision.** An account that fails to provision is
merely unusable. An account that fails to *de*provision may have a freed uid with files still owned by it,
which is the hazard itself. So:

- if step 3 (sever) fails, **do not proceed to step 4**. Leaving the account intact is strictly safer than
  freeing a uid that still owns data.
- a partial failure must be surfaced loudly, not warned into a log the way a missing Docker install is.
- the `users` row should not be considered fully deleted while the OS side is in a partial state, or the
  platform forgets about a mess it created.

**Must not:** run `userdel -r`; delete data under the preserve policy; touch any account other than the one
named; run anything as the member after step 2.

---

## Call sites

- `deleteUserHandler` — the reason this exists.
- An admin-triggered retry, for an account left in a partial state.
- Worth considering: a startup reconciliation that reports Linux accounts with `os_user` set and no
  corresponding `users` row. That is exactly how `officer_jg` would have been noticed months earlier, and it
  is a report rather than an action — nothing should be deleted automatically at boot.

## Open questions for whoever implements it

1. **Where does severed data go?** Reassigned in place under the member's old path, or moved somewhere that
   reads as archival? In place is simpler; a `deleted/` location makes it obvious the data is orphaned.
2. **Is destroy ever exposed in the UI**, or is it always a deliberate operator action outside the platform?
3. **Should uid allocation avoid reuse entirely** as defence in depth — a monotonic counter rather than
   `useradd`'s lowest-free? The previous discussion concluded severing is better, and it is, because it also
   fixes orphaned files. The two are not exclusive.
4. **What happens to a member's rootless Docker images and volumes** under preserve? They become unreadable to
   any live account once chowned, which is correct but means the disk stays occupied by data nobody can open.

---

## Provenance

Every measured claim here comes from a real teardown on the production host on 2026-08-11: the surviving
account and container after a UI delete, the shell that outlived `terminate-user`, `userdel` releasing the
subuid ranges, and the final verified-clean state (no accounts ≥ 1000 but the owner, no files owned by 1001 or
1002 anywhere under `DATA_PATH` or `/home`, linger empty, the owner's eight containers untouched).

The one thing not measured is the preserve path. The teardown used `rm -rf`, because the data was a disposable
test database. `chown -R` as a severing mechanism is reasoned, not observed.

---

## What the implementation decided, where the spec left it open

- **Q1, where severed data goes:** in place. A `deleted/` location is a second thing that can fail between
  severing and releasing, and the ordering rule already says nothing may come between them.
- **Q2, destroy in the UI:** no. `policy: 'destroy'` exists and has no call site; `deleteUserHandler` always
  preserves. It is also implemented as *chown, then delete as the service user* rather than `sudo rm -rf`, so
  a recursive delete as root built from a database column does not exist in the codebase at all.
- **Q3, monotonic uid allocation:** not done. Severing addresses the same hazard and also fixes files orphaned
  by any other route; the two are not exclusive and this one is still available later.
- **Q4, a preserved member's Docker images:** unchanged — they stay on disk, owned by the service user,
  readable by nobody who wants them. Correct and wasteful, as the spec predicted.

Two guards were added that the spec did not ask for, both exported and unit-tested:

- `guardDeletable` — the adoption rule from `ensureOsUser` read backwards. An account is only deletable if its
  passwd home is the home the platform would have confined, and its uid is ≥ 1000. Without it, `userdel root`
  is one bad `users.osUser` value away, and nothing else in the sequence would object.
- `guardMemberTree` — the member tree must resolve to a direct child of `DATA_PATH`. The email reaches
  `join()` from a database row and the result is the argument to a recursive `chown`.

`chown` runs with `-h`. Measured on this host that `chown -R` already declines to follow a symlink out of the
tree and re-owns the link itself, but the flag states it in the argv rather than resting on traversal
semantics — and re-owning links is what makes `find -uid` (which uses `lstat`) a meaningful check.

## What is still unproven

The whole thing has been exercised only by unit tests over the pure guards. Nothing has run against a live
account, and this dev machine is deliberately not the place to try it.

To validate on the production host, against a throwaway account:

1. Create a member, open a terminal as them, and **leave a shell running** — that is the case
   `terminate-user` does not handle, and the reap loop is the part most likely to be wrong.
2. Give them a container that writes as a non-root user, so the subuid range is genuinely populated:
   `postgres:18-alpine` drops to uid 70 and its data directory lands on `subuid_start + 70`.
3. `./scripts/assert-uid-free.sh --capture <user>` — **before** deleting, or the range is gone.
4. Confirm the range check *fails* on that tree while the account still exists. A checker that has never
   failed has not been tested.
5. Delete through the UI, then
   `sudo DATA_PATH="$DATA_PATH" ./scripts/assert-uid-free.sh --check <user> <uid> <start> <count>`.

The delete handler logs that exact command line with the captured values after a successful deprovision,
because after `userdel` nothing else on the machine remembers the range.