platform/docs/DOCKERIZATION_PLAN.md

Officer Platform — Full Dockerization Plan

Current State

The platform runs directly on an Ubuntu host with the following components:

Processes (managed by PM2)

Process	Runtime	Entry Point
officer (main API)	bun	src/server.tsx
officer-claude	bun	src/servers/sidecar/claude/index.ts
officer-pi	bun	src/servers/sidecar/pi/index.ts
officer-email	bun	src/servers/sidecar/email/index.ts
officer-pty	node	src/servers/api/terminal/pty-sidecar.mjs
officer-vnc	bun	src/servers/sidecar/vnc/index.ts

Already Dockerized (docker-compose.yaml)

• Nginx Proxy Manager
• PostgreSQL 18
• Mailhog
• Redis
• SearXNG
• static-files (nginx)
• landing-page (nginx)

System Dependencies

• Runtimes: Bun, Node.js 22, Go, Rust
• CLI tools: pi, claude, ffmpeg, yt-dlp, git, mbsync, cliamp
• Audio: PulseAudio, pactl, libasound2-dev, libvorbis-dev, libogg-dev, libflac-dev
• Desktop: XFCE4, TigerVNC, Brave browser, dbus-x11
• Shell: zsh, oh-my-zsh, starship, neovim, ripgrep, fd, tmux, htop, btop
• Build: build-essential, pkg-config, python3
• Browser: Chrome (Puppeteer, for WhatsApp bot)

---

Target State

Everything runs via docker compose up. Migration = copy volumes + docker compose up on new host.

---

Proposed Container Architecture

Container 1: officer-core

What: Main API server + all sidecars (claude, pi, email) Why combined: Sidecars communicate with the main server over localhost WebSocket. Splitting them into separate containers adds networking complexity with no real benefit — they share the same codebase and dependencies.

Base image: ubuntu:24.04 (not Alpine — too many native deps)

Includes:

• Bun runtime
• Node.js 22 (for PTY sidecar, Pi agent)
• Pi coding agent (@mariozechner/pi-coding-agent)
• Claude Code (@anthropic-ai/claude-code)
• ffmpeg, git, zsh, ripgrep, fd, jq
• Chrome for Testing (Puppeteer — WhatsApp bot)
• mbsync (Gmail IMAP sync)
• yt-dlp
• build-essential, pkg-config, python3 (for native modules like node-pty, argon2)
• PM2 (process manager for main + sidecars)

Entry point: pm2-runtime ecosystem.config.cjs

Ports:

• 9010 — Main API + WebSocket
• 18792 — Browser relay

Volumes:

• ./data:/app/data — DATA_PATH (all user data, sessions, attachments)
• ./pi-config:/root/.pi/agent — Pi agent auth + models config

Environment:

• PORT, JWT_SECRET, POSTGRES_URL, MAIL_TRANSPORT, PUBLIC_URL, DATA_PATH=/app/data, HOME_DIR=/app/data/superadmin/home

Container 2: officer-pty

What: Terminal/PTY sidecar Why separate: This is the only component that needs sudo and Linux user isolation. It creates real Linux users and spawns shells as those users. Running this inside the core container would require the core container to run as root with full user management capabilities.

Base image: ubuntu:24.04

Includes:

• Node.js 22 (node-pty requires Node, not Bun)
• zsh, oh-my-zsh, starship, neovim
• Shell utilities (ripgrep, fd, tmux, htop, btop, tree, git)
• Go, Rust (available in user shells)
• sudo, useradd (user provisioning)

Entry point: node src/servers/api/terminal/pty-sidecar.mjs

Privileged: Yes — needs to create users, spawn shells as different users

Volumes:

• ./data:/app/data — Shared DATA_PATH (user home dirs live here)

Network: Connects to officer-core via WebSocket (ws://officer-core:9010/api/sidecar/register)

Container 3: officer-vnc

What: VNC/remote desktop sidecar Why separate: Requires a full X11 stack (XFCE4, TigerVNC, Brave, dbus). This is ~2GB+ of packages. Keeping it separate means the core image stays lean, and you can skip this container entirely if desktop isn't needed.

Base image: ubuntu:24.04

Includes:

• XFCE4, xfce4-goodies
• TigerVNC server
• Brave browser
• dbus-x11
• Bun (to run the sidecar code)
• sudo, useradd (VNC sessions run as individual users)

Entry point: bun run src/servers/sidecar/vnc/index.ts

Privileged: Yes — spawns VNC servers as different users

Ports:

• 5901-5999 — VNC display ports (dynamic)

Volumes:

• ./data:/app/data — User home dirs (VNC configs stored per-user)

Network: Connects to officer-core via WebSocket

Container 4: officer-audio

What: PulseAudio + cliamp audio sidecar Why separate: PulseAudio daemon needs to run persistently. Audio dependencies (libvorbis, libogg, libflac, libasound) are specific to this feature.

Base image: ubuntu:24.04

Includes:

• PulseAudio (headless, null sink)
• cliamp binary (pre-built or built from source with Go)
• libasound2-dev, libvorbis-dev, libogg-dev, libflac-dev
• script command (bsdutils — for PTY wrapping)

Entry point: Start PulseAudio daemon, then expose audio endpoint

Note: This is the trickiest container. See pain points below.

---

Pain Points & Challenges

1. User Isolation (PTY + VNC)

Problem: The platform creates real Linux users (useradd) and spawns shells/VNC sessions as those users via sudo -u. Inside Docker, this requires:

• Running as root
• The --privileged flag or specific capabilities
• Shared /etc/passwd, /etc/shadow between PTY and VNC containers (both need to know about the same users)

Options:

• A) Shared user database: Mount a shared /etc/passwd and /etc/shadow between PTY and VNC containers. Both can create users there. Risk: concurrent writes.
• B) User provisioning service: A small API that manages Linux users, called by both PTY and VNC sidecars.
• C) Pre-provision users: At startup, scan DATA_PATH for existing user directories and create Linux users for each. New users get created on-demand.
• D) Drop real user isolation: Map all users to a single Linux user, rely on application-level sandboxing only. Simpler but less secure.

Recommendation: Option C with a shared init script. Both containers run the same user-provisioning logic at startup.

2. PulseAudio in Docker

Problem: PulseAudio expects to manage audio hardware. In a headless Docker container, there's no hardware.

Options:

• A) Null sink approach (current): Already works — pactl load-module module-null-sink. The container just needs PulseAudio running with a null sink. cliamp writes to it, and the audio is captured and streamed over WebSocket.
• B) PulseAudio socket sharing: Run PulseAudio on the host, mount the socket into the container.

Recommendation: Option A. It already works headless. Just ensure the Dockerfile starts PulseAudio before cliamp.

3. Audio Integration with Core

Problem: Currently, the /api/cliamp/audio/ws endpoint is part of the main server (src/server.tsx). It spawns cliamp as a child process. If audio moves to a separate container, the WebSocket endpoint and the cliamp process are in different containers.

Options:

• A) Keep audio in core container: Don't split it out. Add PulseAudio + cliamp deps to the core image. Simplest, but adds ~200MB to the core image.
• B) Audio sidecar with WebSocket proxy: The audio container runs its own WebSocket server. The core container proxies /api/cliamp/* to it. More complex.
• C) Audio as a registered sidecar: Like the other sidecars, audio registers over WebSocket and handles commands. The cleanest architecture but requires refactoring the cliamp code.

Recommendation: Option A for now (keep in core). Refactor to Option C later if the image size becomes a problem.

4. Chrome/Puppeteer (WhatsApp Bot)

Problem: Chrome needs specific flags (--no-sandbox, --disable-dev-shm-usage) and a larger /dev/shm in Docker.

Solution: Well-understood. Add to docker-compose:

shm_size: '2gb'

And install Chrome for Testing via Puppeteer's CLI in the Dockerfile.

5. VNC Port Range

Problem: VNC dynamically allocates ports 5901-5999 based on available displays. Docker needs these ports mapped.

Options:

• A) Fixed port range: Map 5901-5910 (10 concurrent sessions max). Simple.
• B) Host networking: Use network_mode: host for the VNC container. No port mapping needed but less isolation.
• C) noVNC proxy: Add a noVNC WebSocket proxy. Clients connect via a single HTTP port, no VNC port range needed. Better for firewalls/reverse proxies.

Recommendation: Option C (noVNC) is the best long-term solution. Option A works for now.

6. Shared Source Code

Problem: All containers need the same source code (sidecars import from the same codebase). Building separate images from the same repo means either:

• Copying the full source into each image (duplicated, large)
• Using a multi-stage build with a shared base
• Mounting the source as a volume (dev only, not portable)

Recommendation: Multi-stage Dockerfile. One base stage installs OS deps + runtimes + bun install. Subsequent stages extend it with container-specific deps (XFCE for VNC, PulseAudio for audio, etc.).

7. DATA_PATH Permissions

Problem: Multiple containers write to the same DATA_PATH volume. If containers run as different UIDs, file permissions conflict.

Solution: All containers should run processes as the same UID (e.g., 1000). The PTY and VNC containers additionally need root to manage sub-users, but the data files should be owned by a consistent UID.

8. Pi and Claude Config Directories

Problem: Pi reads ~/.pi/agent/auth.json and ~/.pi/agent/models.json. Claude reads ~/.claude/. These are in the container's home directory, which is ephemeral.

Solution: Mount these as volumes:

volumes:
  - ./config/pi:/home/officer/.pi/agent
  - ./config/claude:/home/officer/.claude

9. Native Modules (node-pty, argon2)

Problem: node-pty and argon2 have native C/C++ bindings. They need to be compiled for the container's architecture, not the host's.

Solution: Run bun install inside the Dockerfile (not on the host). The Dockerfile must include build-essential, python3, and pkg-config.

10. Image Size

Concern: The core image will be large due to Bun + Node.js + Chrome + native deps. The VNC image will be even larger (XFCE alone is ~1.5GB).

Estimated sizes:

• officer-core: ~2-3 GB (Bun, Node, Chrome, ffmpeg, native deps)
• officer-pty: ~1-1.5 GB (Node, shell tools, Go, Rust)
• officer-vnc: ~2.5-3 GB (XFCE, TigerVNC, Brave, Bun)
• officer-audio: ~500 MB (PulseAudio, cliamp, Go)

Mitigation: Multi-stage builds, .dockerignore, shared base layers.

---

Proposed docker-compose.yaml Structure

services:
  # --- Infrastructure (already dockerized) ---
  nginx-proxy-manager:
    image: jc21/nginx-proxy-manager:latest
    # ... (unchanged)

  postgres:
    image: postgres:18-alpine
    # ... (unchanged)

  redis:
    image: redis:alpine
    # ... (unchanged)

  mailhog:
    image: mailhog/mailhog:latest
    # ... (unchanged)

  searxng:
    image: searxng/searxng:latest
    # ... (unchanged)

  static-files:
    image: nginx:alpine
    # ... (unchanged)

  landing-page:
    image: nginx:alpine
    # ... (unchanged)

  # --- Application ---
  officer-core:
    build:
      context: .
      dockerfile: docker/Dockerfile.core
    container_name: officer-core
    restart: unless-stopped
    ports:
      - "9010:9010"
      - "18792:18792"
    environment:
      - PORT=9010
      - NODE_ENV=production
      - DATA_PATH=/app/data
      - HOME_DIR=/app/data/superadmin/home
      - POSTGRES_URL=postgresql://postgres:${PG_PASSWORD}@postgres:5432/officer_dev
      - MAIL_TRANSPORT=smtp://mailhog:1025
      - PUBLIC_URL=${PUBLIC_URL}
      - JWT_SECRET=${JWT_SECRET}
      - BROWSER_RELAY_PORT=18792
      - DISCORD_BUG_REPORT_WEBHOOK=${DISCORD_BUG_REPORT_WEBHOOK}
    volumes:
      - officer-data:/app/data
      - pi-config:/home/officer/.pi/agent
    shm_size: '2gb'
    networks:
      - services
    depends_on:
      - postgres
      - redis

  officer-pty:
    build:
      context: .
      dockerfile: docker/Dockerfile.pty
    container_name: officer-pty
    restart: unless-stopped
    privileged: true
    environment:
      - API_URL=ws://officer-core:9010/api/sidecar/register
      - DATA_PATH=/app/data
    volumes:
      - officer-data:/app/data
    networks:
      - services
    depends_on:
      - officer-core

  officer-vnc:
    build:
      context: .
      dockerfile: docker/Dockerfile.vnc
    container_name: officer-vnc
    restart: unless-stopped
    privileged: true
    ports:
      - "5901-5910:5901-5910"
    environment:
      - API_URL=ws://officer-core:9010/api/sidecar/register
      - DATA_PATH=/app/data
    volumes:
      - officer-data:/app/data
    networks:
      - services
    depends_on:
      - officer-core

volumes:
  officer-data:
  pi-config:

networks:
  services:
    external: true

---

Migration Path

Phase 1: Core container

1. Write Dockerfile.core — Bun, Node, Pi, Claude, Chrome, ffmpeg
2. Move the .env config to docker-compose environment variables
3. Update POSTGRES_URL to use Docker service name (postgres not localhost)
4. Update MAIL_TRANSPORT to use Docker service name (mailhog)
5. Test: main server + claude/pi/email sidecars work in container
6. Test: WhatsApp bot (Puppeteer/Chrome) works

Phase 2: PTY container

1. Write Dockerfile.pty — Node, shell tools, user management
2. Refactor PTY sidecar to connect to remote API_URL (currently assumes localhost)
3. Implement user provisioning in container startup
4. Test: terminal sessions work from the web UI

Phase 3: VNC container

1. Write Dockerfile.vnc — XFCE, TigerVNC, Brave
2. Refactor VNC sidecar for remote API_URL
3. Shared user provisioning with PTY container
4. Test: remote desktop sessions work

Phase 4: Audio (optional)

1. If kept in core: add PulseAudio + cliamp deps to Dockerfile.core
2. If separate: write Dockerfile.audio and refactor to sidecar protocol
3. Test: music playback works

Phase 5: Cleanup

1. Remove scripts/setup.sh system-level deps (no longer needed)
2. Write backup/restore scripts for Docker volumes
3. Document the new deployment process
4. CI/CD: build images and push to registry

---

What Stays Outside Docker

• Nginx Proxy Manager — already dockerized, unchanged
• DNS / Tailscale — host-level networking
• UFW — host firewall
• Hetzner backups — host-level
• Cron jobs (pg_dump) — can move into a container or stay on host
• SSL certificates — managed by NPM

---

Open Questions

1. Should Go and Rust be in the PTY container? Users may want to compile Go/Rust projects from the terminal. But it adds significant image size. Could be made optional via a build arg.

2. Should cliamp be pre-built or built from source? Building from source requires Go + audio dev headers in the image. A pre-built binary would be smaller but architecture-specific.

3. How to handle Pi/Claude version updates? Currently npm install -g on the host. In Docker, this means rebuilding the image. Could mount a volume for global npm packages, but that adds complexity.

4. Do we need a container registry? For single-server deployments, docker compose build is fine. For multi-server, you'd want to push images to a registry (GitHub Container Registry, Docker Hub, self-hosted).

5. Dev workflow: How to develop locally? Mount source code as volume + hot reload? Or just develop on host and only use Docker for deployment?