officerdev/platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

rootless docker per member — provisioning works, running a container does not yet

Browse Source 
Not finished. Committed because the diagnosis is worth more than the code.

WHY ROOTLESS AND NOT THE DOCKER GROUP. `usermod -aG docker <user>` is the one-line version
and it is root: `docker run -v /:/host -it alpine chroot /host` is a root shell, which reads
.env, every other member's home and the wallet seed. Every boundary from today, bypassed by
one documented command. Rootless gives what was actually asked for — a daemon per account,
containers in that account's user namespace, images in their own home.

VERIFIED on this host: provisioning succeeds, the server reports 29.5.0, the daemon runs as
the member, `docker pull` puts 403 MB under their own home, and `docker ps -a` shows nothing
while the owner has four containers. That last line is the isolation, measured.

NOT VERIFIED: actually running a container. It failed, and the cause is an interaction
between two things built today:

  failed to copy xattrs: failed to set xattr "system.posix_acl_default" on …/volumes/…/_data

Creating a volume copies xattrs, and the DEFAULT ACLs on a member's home — added so the file
browser could read their files — are inherited by Docker's storage, where a mapped id inside
a user namespace is not a valid id to set. Both features correct alone. The fix here strips
default ACLs from ~/.local/share/docker only, leaving the access ACLs the file browser needs.

That fix is UNPROVEN. The re-test failed for a different, environmental reason: probe users
recycle uid 1001, and a stale lingering systemd user manager from a previous probe answered
`systemctl --user`, so the unit appeared not to exist. Cleaned with `loginctl terminate-user`.
Retest on a machine that has not had a uid-1001 user, or on a fresh uid.

Also worth knowing before this ships: uid reuse after deleting a member is a real hazard, not
just a test artefact — the next member gets the previous member's uid, and anything left
lingering belongs to them.

setup.sh gains uidmap and dbus-user-session as core packages; the shell template exports
DOCKER_HOST from $XDG_RUNTIME_DIR when the socket exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
pastilhas
2026-08-11 20:01:17 +00:00
co-authored by Claude Opus 5
parent 71589aee99
commit 3bea46f2d7
4 changed files with 246 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/servers/os-user-docker.ts
+198
View File
@@ -0,0 +1,198 @@
import { existsSync } from 'node:fs';
import { runAs } from './os-user';
// A member's own Docker: their daemon, their images, their containers, running as their uid.
//
// ── Why rootless, and why the alternative is not on the table ──
//
// The one-line version of "give the user Docker" is `usermod -aG docker <user>`, and it is root. Membership of
// that group means talking to the host daemon, which runs as root, so:
//
// docker run -v /:/host -it alpine chroot /host
//
// is a root shell on the machine. It reads the platform's `.env`, every other member's home, the wallet seed
// — every boundary in docs/per-user-linux-accounts.md, bypassed by one documented command. The group is not
// "access to Docker", it is "root, by a longer route".
//
// Rootless gives the thing that was actually wanted: a daemon per account, containers in that account's user
// namespace, images in their own home. Root inside their container is their uid outside it, which is nobody.
// They cannot see the owner's containers and the owner cannot break theirs.
//
// ── What it needs from the host ──
//
// uidmap newuidmap/newgidmap, to map subordinate ids. Rootless cannot start without them.
// /etc/subuid,gid a range per account. `useradd` allocates one automatically wherever login.defs sets
// SUB_UID_COUNT (Ubuntu does), and `userdel` reclaims it — verified on this host.
// linger `loginctl enable-linger`, or the daemon dies with the session. Officer's shells are
// NOT login sessions, so without this a member's Docker would stop the moment their
// terminal closed, which is the opposite of a daemon.
// dbus-user-session systemd --user needs a bus to talk to.
//
// ── The costs, stated rather than discovered ──
//
// Each account has its own image cache, so three members pulling postgres:16 store it three times. Ports
// below 1024 need an explicit capability grant. Both are acceptable for what this buys; neither is hidden.
/** Their own daemon's socket. The value `DOCKER_HOST` must point at. */
export const dockerSocketFor = (uid: number): string => `/run/user/${uid}/docker.sock`;
type Result = { ok: true; alreadyInstalled: boolean } | { ok: false; error: string };
async function sudo(args: string[]): Promise<{ ok: boolean; out: string }> {
const proc = Bun.spawn(['sudo', '-n', ...args], { stdout: 'pipe', stderr: 'pipe' });
const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]);
return { ok: (await proc.exited) === 0, out: `${out}${err}`.trim() };
}
/**
* Run something as the member with a systemd user session in scope.
*
* `runAs` clears the environment, which is right everywhere else and fatal here: `systemctl --user` and the
* rootless setup tool locate the user manager through `XDG_RUNTIME_DIR` and `DBUS_SESSION_BUS_ADDRESS`. With
* those unset the tool reports "systemd not detected" and installs nothing, successfully.
*/
function asMemberWithSession(osUser: string, uid: number, command: string[]) {
const runtime = `/run/user/${uid}`;
return runAs(osUser, [
'env',
`XDG_RUNTIME_DIR=${runtime}`,
`DBUS_SESSION_BUS_ADDRESS=unix:path=${runtime}/bus`,
// The tool shells out to newuidmap, rootlesskit and dockerd, and --reset-env left PATH at the passwd
// default which does not include /usr/sbin.
'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
...command,
]);
}
const text = async (proc: ReturnType<typeof runAs>) =>
`${await new Response(proc.stdout).text()}${await new Response(proc.stderr).text()}`.trim();
/** Everything that must be true of the HOST before any account can have rootless Docker. */
export function checkDockerPrerequisites(): { ok: true } | { ok: false; error: string } {
const missing: string[] = [];
if (!existsSync('/usr/bin/newuidmap') || !existsSync('/usr/bin/newgidmap')) missing.push('uidmap');
if (!existsSync('/usr/bin/dockerd-rootless-setuptool.sh')) missing.push('docker-ce-rootless-extras');
if (missing.length) {
return {
ok: false,
error: `rootless Docker needs these packages on the host: ${missing.join(', ')} (apt install ${missing.join(' ')})`,
};
}
return { ok: true };
}
/**
* Give the account its own rootless Docker daemon, and start it.
*
* Idempotent: the setup tool is safe to re-run, `enable-linger` on a lingering account is a no-op, and an
* already-running daemon is reported as `alreadyInstalled` rather than restarted — a reprovision must not
* bounce a member's containers.
*
* Never throws. Docker is the least essential of the provisioning steps: without it the account still has a
* shell, a home and keys.
*/
export async function provisionRootlessDocker(params: { osUser: string; uid: number; home: string }): Promise<Result> {
const prereq = checkDockerPrerequisites();
if (!prereq.ok) return prereq;
// Subordinate id range. `useradd` allocates one, so a missing entry means either an unusual login.defs or
// an account made by hand — worth naming rather than letting rootlesskit fail obscurely later.
const subuid = await sudo(['grep', '-q', `^${params.osUser}:`, '/etc/subuid']);
if (!subuid.ok) {
return {
ok: false,
error: `${params.osUser} has no /etc/subuid range, which rootless Docker requires. Add one with: sudo usermod --add-subuids 100000-165535 ${params.osUser}`,
};
}
// Linger FIRST: it is what creates /run/user/<uid> and starts the user manager, and everything below needs
// both to exist.
const linger = await sudo(['loginctl', 'enable-linger', params.osUser]);
if (!linger.ok) return { ok: false, error: `could not enable linger for ${params.osUser}: ${linger.out}` };
// The user manager appears asynchronously. Waiting beats a bare sleep, and the failure below is clearer
// than "systemd not detected" from the setup tool.
for (let attempt = 0; attempt < 25 && !existsSync(`/run/user/${params.uid}`); attempt++) {
await Bun.sleep(200);
}
if (!existsSync(`/run/user/${params.uid}`)) {
return {
ok: false,
error: `/run/user/${params.uid} never appeared, so ${params.osUser} has no systemd user session`,
};
}
const already = existsSync(dockerSocketFor(params.uid));
// ── The setup tool's exit code is deliberately NOT the gate ──
//
// It writes `~/.config/systemd/user/docker.service` and then runs `systemctl --user start docker.service`
// itself — which fails with "Unit docker.service not found" on a manager that was already running when the
// file appeared, because nothing reloaded it. Measured here: the unit was written correctly and the tool
// still exited 1.
//
// So: run it, reload, and start it ourselves. The tool's output is kept for the error message if the start
// then genuinely fails, since its diagnosis (a missing kernel module, an unsupported filesystem) is better
// than anything paraphrased.
const install = asMemberWithSession(params.osUser, params.uid, ['dockerd-rootless-setuptool.sh', 'install']);
const installOut = await text(install);
await install.exited;
const reload = asMemberWithSession(params.osUser, params.uid, ['systemctl', '--user', 'daemon-reload']);
await reload.exited;
const enable = asMemberWithSession(params.osUser, params.uid, ['systemctl', '--user', 'enable', '--now', 'docker']);
const enableOut = await text(enable);
if ((await enable.exited) !== 0) {
// Both halves, because the useful sentence is usually in the setup tool's output rather than systemd's.
const detail = [installOut, enableOut]
.map((s) => s.split('\n').slice(-3).join(' ').trim())
.filter(Boolean)
.join(' | ');
return { ok: false, error: `could not start ${params.osUser}'s Docker: ${detail}` };
}
// ── Undo our own ACLs, for this one subtree ──
//
// The home carries DEFAULT ACLs so the file browser can read a member's files (os-user.ts explains why).
// Docker inherits them under `~/.local/share/docker`, then fails every `docker run` with:
//
// failed to copy xattrs: failed to set xattr "system.posix_acl_default" on …/volumes/…/_data:
// invalid argument
//
// Creating a volume copies xattrs, and inside a rootless user namespace the mapped id in an inherited
// default ACL is not a valid id, so setting it is EINVAL. Two features built the same day, each correct
// alone. Measured: the image pulled fine — 403 MB into their home — and every container failed to start.
//
// `-k` removes DEFAULT entries only, so nothing inside Docker's storage inherits them from here on. The
// access ACLs on the home itself are untouched, which is what the file browser depends on. Losing the
// platform's reach into Docker's internal storage is no loss: it is image layers and volume data, read
// through `docker` or not at all.
const dockerData = `${params.home}/.local/share/docker`;
if (existsSync(dockerData)) {
const stripped = await sudo(['setfacl', '-R', '-k', dockerData]);
if (!stripped.ok) {
return { ok: false, error: `could not clear inherited ACLs from ${dockerData}: ${stripped.out}` };
}
}
// Proof, not assumption: ask their daemon who it is. `docker version --format` on the SERVER half only
// answers if the socket is live and talking.
const verify = asMemberWithSession(params.osUser, params.uid, [
'env',
`DOCKER_HOST=unix://${dockerSocketFor(params.uid)}`,
'docker',
'version',
'--format',
'{{.Server.Version}}',
]);
const version = await text(verify);
if ((await verify.exited) !== 0) {
return {
ok: false,
error: `${params.osUser}'s Docker did not answer: ${version.split('\n').slice(-2).join(' ').trim()}`,
};
}
return { ok: true, alreadyInstalled: already };
}