platform/docs/deprovision-os-account.md

Deprovisioning a member's Linux account

Status: implemented 2026-08-12 in src/servers/os-user-deprovision.ts, called by deleteUserHandler. Run against a real account on 2026-08-12 and verified clean — green, uid 1001, with a live systemd session, a running rootless Docker stack and a shell parented outside the session cgroup. Nine processes reaped in ~60s, then all ten checks of scripts/assert-uid-free.sh passed, and the preserve policy left 316 MB reassigned to the service user with zero ACL entries naming the freed uid. Written from a manual teardown performed on the production host on 2026-08-11, so the ordering constraints below are measured rather than reasoned.

The spec is kept as written rather than rewritten in the past tense: it is the reasoning the implementation has to keep satisfying, and the failure modes it names are still the ones a change would reintroduce.

---

What happens today

deleteUserHandler removes the users row and cascades the database. userdel never runs. Measured on a real member (green, uid 1002) immediately after deleting them through the UI, before any cleanup:

	after deleteUserHandler
users row	gone
Linux account	alive, uid 1002
Login shell	id -u → 1002 — the deleted account still had a working login
Rootless Docker	daemon running, postgres container Up 2 hours (healthy)
Home + Docker storage	454 MB intact
linger, /run/user/1002, /etc/subuid, /etc/subgid	all present

Nothing breaks, which is what makes it dangerous. The account keeps working; only the platform forgets it exists.

Why it matters: uid reuse

useradd allocates the lowest free uid. Delete a member and the uid is free while their files still carry it, so the next member created inherits the previous member's home, keys, Docker storage and anything else owned by that number. By uid, not by any decision anyone made.

This is not hypothetical. officer_jg (uid 1001) and green (uid 1002) both had login shells pointing at one home on this host, and the users table had no row for officer_jg at all — an earlier account for the same email, deleted from the platform, whose Linux side survived. The adoption rule in ensureOsUser was never bypassed; the account simply outlived the row.

The invariant this function exists to guarantee:

After deprovisioning, no file anywhere is owned by the freed uid or by any id in its freed subuid range, and no passwd entry, linger marker, runtime directory or process refers to it.

The subuid half, which is easy to miss

A member's rootless Docker storage is not owned by their uid. Container processes map through /etc/subuid, so the files are owned by ids in that range — on this host, green had 231072:65536, and postgres's data directory was owned by 231141 (231072 + 70, postgres's inner uid in the Alpine image).

userdel releases the subuid range along with the uid, and a later account can be allocated the same range. So a check for "nothing is owned by the freed uid" passes while hundreds of megabytes are still owned by the freed subuid range, and a future member's containers would map onto another member's leftover files.

Any verification has to cover the range, not just the uid.

---

The sequence

Ordering is load-bearing. Each step explains what breaks if it moves.

1. Disable linger, before stopping anything

loginctl disable-linger <user>

Lingering keeps a systemd user manager alive with no login session. Terminate first and linger can bring it back; disable first and nothing can re-spawn between the two steps.

(The manual teardown ran these in the opposite order and worked. This order is specified because it removes a race rather than because the other one failed.)

2. Terminate the session, then verify it actually died

loginctl terminate-user <user>

terminate-user is not a barrier. Measured: a /bin/zsh -i owned by the member survived it — three hours old, still running after the session was terminated and /run/user/<uid> was removed. userdel refuses while a process owned by the account is alive, so an implementation that trusts terminate-user works on a quiet account and fails on a member who left a shell open, which is the normal case.

Required after terminating:

pkill -u <user>            # wait, then re-check
pkill -9 -u <user>         # only if the count is still non-zero

with a bounded wait between and a final assertion that the process count is zero. Do not proceed while it is not.

3. Sever the data from the uid — before releasing it

Two policies. The platform's default is preserve:

chown -R <service-user>:<service-group> <member-tree>

Destroying a member's data because their account was deleted is a separate decision from removing their access, and the platform has no standing to make it silently. Reassigning ownership severs the uid link while keeping every byte.

Destroy is opt-in, for a deliberate rebuild:

rm -rf <member-tree>

Ownership is not the only link. confineUserTree grants the member a NAMED ACL entry on their whole tree — u:<uid>:rwx and a default: copy, inherited by everything either party creates. chown does not remove them: they are xattrs rather than ownership, and they store the uid numerically. Measured — a chown -h -R to the service user leaves user:<uid>:rwx intact on the directory, its children and their defaults.

So a tree reassigned to the service user still grants the freed uid read and write on every byte, and the next account allocated that number inherits it: home, SSH keys, .credentials.json, transcripts, container storage. That is the hazard this function exists to prevent, arriving through ACLs instead of ownership.

Severing therefore has two parts:

chown -h -R <service-user>:<service-group> <member-tree>
setfacl -R -b <member-tree>          # or -x u:<uid> -x d:u:<uid> to keep the platform's own entry

-b is the simpler answer for a preserved tree: the service user owns every byte afterwards, so a named entry granting themselves access is redundant.

This step must complete before step 4. That is the one ordering choice the manual teardown got wrong: it released the uid first and removed the data afterwards, which leaves a window where the uid is free while files still carry it. If the process dies in that window, the next useradd inherits them. Sever first, then release — the irreversible step goes last, and only once nothing points at it.

4. Release the account

userdel <user>          # NEVER -r

-r deletes the home, which contradicts the preserve policy and would make the destroy policy depend on a flag rather than on an explicit decision. Measured: plain userdel removes the passwd, shadow and group entries and the /etc/subuid and /etc/subgid ranges.

5. Verify, and refuse to call it done otherwise

See the checklist below. A deprovision that half-succeeded is worse than one that failed cleanly, because the uid is free and something still owns files.

---

Verification: what "clean" means

All of these must hold for the freed uid and its freed subuid range:

• getent passwd <user> → nothing
• no entry in /etc/subuid or /etc/subgid
• find <DATA_PATH> /home -uid <uid> → nothing
• find <DATA_PATH> /home -uid <subuid-start> -o ... over the freed range → nothing (a range scan, not a single id — the mapped ids are spread across it)
• /var/lib/systemd/linger/<user> absent
• /run/user/<uid> absent
• no processes owned by the uid
• no ACL entry naming the uid anywhere under DATA_PATH — getfacl -R -n and look for user:<uid>: / default:user:<uid>:. Ownership checks cannot see these, and chown does not clear them.

scripts/assert-uid-free.sh implements exactly this, deliberately outside the function: a checker the implementation calls is a restatement of its own beliefs, not an audit. Two modes, and the split matters —

./scripts/assert-uid-free.sh --capture green                    # BEFORE: prints "green 1001 165536 65536"
sudo DATA_PATH="$DATA_PATH" ./scripts/assert-uid-free.sh --check green 1001 165536 65536   # AFTER: exit 1 unless clean

Pass DATA_PATH through explicitly. sudo's env_reset drops it, so the plain sudo ./assert-uid-free.sh this used to say fell back to a hardcoded default — and every check here reports ok on finding nothing, so a wrong root reports CLEAN — uid safe to reissue without having looked at a single member tree. The ACL check is the one that failed silently and completely, because it is the only one scoped to DATA_PATH alone. The script now refuses to run when a search root is missing rather than passing vacuously.

The range has to be captured before deletion, because userdel removes the /etc/subuid entry with the account. After that there is no way to ask what range it held — and a check that silently skips that half is the exact failure this section exists to prevent.

The subuid check passes vacuously on most accounts, and that is a trap. Container files are owned by a mapped id only when a process inside the container runs as a NON-root user; an image whose files are root-owned maps to the member's own uid and leaves nothing in the range. Measured on green after a night of real use — claude installed, images pulled, transcripts written — the range check found zero files and passed without testing anything.

To build a specimen that actually exercises it, run a container whose process writes as a non-root user. The postgres:18-alpine case from the same night is the natural one: its entrypoint drops to uid 70, and the data directory came out owned by subuid_start + 70 on the host. Verify the range check fails on that tree before trusting it to pass on a cleaned one.

Trap for the verifier: do not use sudo -u <user> … to check anything after step 2. Creating a session starts a user manager and recreates /run/user/<uid>, so the check would undo the step it is verifying.

---

Behaviour requirements

Idempotent. Every step tolerates already-done. Re-running on a clean box is a no-op, and re-running after a partial failure completes it. The delete handler should be able to call it, fail, and have an operator press retry.

Never throws; returns a result. Same posture as provisionOsAccount, provisionSshAccess and provisionRootlessDocker.

But a failed deprovision is not the same as a failed provision. An account that fails to provision is merely unusable. An account that fails to deprovision may have a freed uid with files still owned by it, which is the hazard itself. So:

• if step 3 (sever) fails, do not proceed to step 4. Leaving the account intact is strictly safer than freeing a uid that still owns data.
• a partial failure must be surfaced loudly, not warned into a log the way a missing Docker install is.
• the users row should not be considered fully deleted while the OS side is in a partial state, or the platform forgets about a mess it created.

Must not: run userdel -r; delete data under the preserve policy; touch any account other than the one named; run anything as the member after step 2.

---

Call sites

• deleteUserHandler — the reason this exists.
• An admin-triggered retry, for an account left in a partial state.
• Worth considering: a startup reconciliation that reports Linux accounts with os_user set and no corresponding users row. That is exactly how officer_jg would have been noticed months earlier, and it is a report rather than an action — nothing should be deleted automatically at boot.

Open questions for whoever implements it

1. Where does severed data go? Reassigned in place under the member's old path, or moved somewhere that reads as archival? In place is simpler; a deleted/ location makes it obvious the data is orphaned.
2. Is destroy ever exposed in the UI, or is it always a deliberate operator action outside the platform?
3. Should uid allocation avoid reuse entirely as defence in depth — a monotonic counter rather than useradd's lowest-free? The previous discussion concluded severing is better, and it is, because it also fixes orphaned files. The two are not exclusive.
4. What happens to a member's rootless Docker images and volumes under preserve? They become unreadable to any live account once chowned, which is correct but means the disk stays occupied by data nobody can open.

---

Provenance

Every measured claim here comes from a real teardown on the production host on 2026-08-11: the surviving account and container after a UI delete, the shell that outlived terminate-user, userdel releasing the subuid ranges, and the final verified-clean state (no accounts ≥ 1000 but the owner, no files owned by 1001 or 1002 anywhere under DATA_PATH or /home, linger empty, the owner's eight containers untouched).

The one thing not measured is the preserve path. The teardown used rm -rf, because the data was a disposable test database. chown -R as a severing mechanism is reasoned, not observed.

---

What the implementation decided, where the spec left it open

• Q1, where severed data goes: in place. A deleted/ location is a second thing that can fail between severing and releasing, and the ordering rule already says nothing may come between them.
• Q2, destroy in the UI: no. policy: 'destroy' exists and has no call site; deleteUserHandler always preserves. It is also implemented as chown, then delete as the service user rather than sudo rm -rf, so a recursive delete as root built from a database column does not exist in the codebase at all.
• Q3, monotonic uid allocation: not done. Severing addresses the same hazard and also fixes files orphaned by any other route; the two are not exclusive and this one is still available later.
• Q4, a preserved member's Docker images: unchanged — they stay on disk, owned by the service user, readable by nobody who wants them. Correct and wasteful, as the spec predicted.

Two guards were added that the spec did not ask for, both exported and unit-tested:

• guardDeletable — the adoption rule from ensureOsUser read backwards. An account is only deletable if its passwd home is the home the platform would have confined, and its uid is ≥ 1000. Without it, userdel root is one bad users.osUser value away, and nothing else in the sequence would object.
• guardMemberTree — the member tree must resolve to a direct child of DATA_PATH. The email reaches join() from a database row and the result is the argument to a recursive chown.

chown runs with -h. Measured on this host that chown -R already declines to follow a symlink out of the tree and re-owns the link itself, but the flag states it in the argv rather than resting on traversal semantics — and re-owning links is what makes find -uid (which uses lstat) a meaningful check.

What is still unproven

The whole thing has been exercised only by unit tests over the pure guards. Nothing has run against a live account, and this dev machine is deliberately not the place to try it.

To validate on the production host, against a throwaway account:

1. Create a member, open a terminal as them, and leave a shell running — that is the case terminate-user does not handle, and the reap loop is the part most likely to be wrong.
2. Give them a container that writes as a non-root user, so the subuid range is genuinely populated: postgres:18-alpine drops to uid 70 and its data directory lands on subuid_start + 70.
3. ./scripts/assert-uid-free.sh --capture <user> — before deleting, or the range is gone.
4. Confirm the range check fails on that tree while the account still exists. A checker that has never failed has not been tested.
5. Delete through the UI, then sudo DATA_PATH="$DATA_PATH" ./scripts/assert-uid-free.sh --check <user> <uid> <start> <count>.

The delete handler logs that exact command line with the captured values after a successful deprovision, because after userdel nothing else on the machine remembers the range.