officerdev/platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

dockerization plan

Browse Source
This commit is contained in:
pastilhas
2026-03-07 08:22:46 +00:00
parent 6c4595279a
commit 70442ae277
1 changed files with 404 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/DOCKERIZATION_PLAN.md
+404
View File
@@ -0,0 +1,404 @@
# Officer Platform — Full Dockerization Plan
## Current State
The platform runs directly on an Ubuntu host with the following components:
### Processes (managed by PM2)
| Process | Runtime | Entry Point |
|---------|---------|-------------|
| officer (main API) | bun | `src/server.tsx` |
| officer-claude | bun | `src/servers/sidecar/claude/index.ts` |
| officer-pi | bun | `src/servers/sidecar/pi/index.ts` |
| officer-email | bun | `src/servers/sidecar/email/index.ts` |
| officer-pty | node | `src/servers/api/terminal/pty-sidecar.mjs` |
| officer-vnc | bun | `src/servers/sidecar/vnc/index.ts` |
### Already Dockerized (docker-compose.yaml)
- Nginx Proxy Manager
- PostgreSQL 18
- Mailhog
- Redis
- SearXNG
- static-files (nginx)
- landing-page (nginx)
### System Dependencies
- **Runtimes:** Bun, Node.js 22, Go, Rust
- **CLI tools:** pi, claude, ffmpeg, yt-dlp, git, mbsync, cliamp
- **Audio:** PulseAudio, pactl, libasound2-dev, libvorbis-dev, libogg-dev, libflac-dev
- **Desktop:** XFCE4, TigerVNC, Brave browser, dbus-x11
- **Shell:** zsh, oh-my-zsh, starship, neovim, ripgrep, fd, tmux, htop, btop
- **Build:** build-essential, pkg-config, python3
- **Browser:** Chrome (Puppeteer, for WhatsApp bot)
---
## Target State
Everything runs via `docker compose up`. Migration = copy volumes + `docker compose up` on new host.
---
## Proposed Container Architecture
### Container 1: `officer-core`
**What:** Main API server + all sidecars (claude, pi, email)
**Why combined:** Sidecars communicate with the main server over localhost WebSocket. Splitting them into separate containers adds networking complexity with no real benefit — they share the same codebase and dependencies.
**Base image:** `ubuntu:24.04` (not Alpine — too many native deps)
**Includes:**
- Bun runtime
- Node.js 22 (for PTY sidecar, Pi agent)
- Pi coding agent (`@mariozechner/pi-coding-agent`)
- Claude Code (`@anthropic-ai/claude-code`)
- ffmpeg, git, zsh, ripgrep, fd, jq
- Chrome for Testing (Puppeteer — WhatsApp bot)
- mbsync (Gmail IMAP sync)
- yt-dlp
- build-essential, pkg-config, python3 (for native modules like node-pty, argon2)
- PM2 (process manager for main + sidecars)
**Entry point:** `pm2-runtime ecosystem.config.cjs`
**Ports:**
- `9010` — Main API + WebSocket
- `18792` — Browser relay
**Volumes:**
- `./data:/app/data` — DATA_PATH (all user data, sessions, attachments)
- `./pi-config:/root/.pi/agent` — Pi agent auth + models config
**Environment:**
- `PORT`, `JWT_SECRET`, `POSTGRES_URL`, `MAIL_TRANSPORT`, `PUBLIC_URL`, `DATA_PATH=/app/data`, `HOME_DIR=/app/data/superadmin/home`
### Container 2: `officer-pty`
**What:** Terminal/PTY sidecar
**Why separate:** This is the only component that needs `sudo` and Linux user isolation. It creates real Linux users and spawns shells as those users. Running this inside the core container would require the core container to run as root with full user management capabilities.
**Base image:** `ubuntu:24.04`
**Includes:**
- Node.js 22 (node-pty requires Node, not Bun)
- zsh, oh-my-zsh, starship, neovim
- Shell utilities (ripgrep, fd, tmux, htop, btop, tree, git)
- Go, Rust (available in user shells)
- sudo, useradd (user provisioning)
**Entry point:** `node src/servers/api/terminal/pty-sidecar.mjs`
**Privileged:** Yes — needs to create users, spawn shells as different users
**Volumes:**
- `./data:/app/data` — Shared DATA_PATH (user home dirs live here)
**Network:** Connects to officer-core via WebSocket (`ws://officer-core:9010/api/sidecar/register`)
### Container 3: `officer-vnc`
**What:** VNC/remote desktop sidecar
**Why separate:** Requires a full X11 stack (XFCE4, TigerVNC, Brave, dbus). This is ~2GB+ of packages. Keeping it separate means the core image stays lean, and you can skip this container entirely if desktop isn't needed.
**Base image:** `ubuntu:24.04`
**Includes:**
- XFCE4, xfce4-goodies
- TigerVNC server
- Brave browser
- dbus-x11
- Bun (to run the sidecar code)
- sudo, useradd (VNC sessions run as individual users)
**Entry point:** `bun run src/servers/sidecar/vnc/index.ts`
**Privileged:** Yes — spawns VNC servers as different users
**Ports:**
- `5901-5999` — VNC display ports (dynamic)
**Volumes:**
- `./data:/app/data` — User home dirs (VNC configs stored per-user)
**Network:** Connects to officer-core via WebSocket
### Container 4: `officer-audio`
**What:** PulseAudio + cliamp audio sidecar
**Why separate:** PulseAudio daemon needs to run persistently. Audio dependencies (libvorbis, libogg, libflac, libasound) are specific to this feature.
**Base image:** `ubuntu:24.04`
**Includes:**
- PulseAudio (headless, null sink)
- cliamp binary (pre-built or built from source with Go)
- libasound2-dev, libvorbis-dev, libogg-dev, libflac-dev
- `script` command (bsdutils — for PTY wrapping)
**Entry point:** Start PulseAudio daemon, then expose audio endpoint
**Note:** This is the trickiest container. See pain points below.
---
## Pain Points & Challenges
### 1. User Isolation (PTY + VNC)
**Problem:** The platform creates real Linux users (`useradd`) and spawns shells/VNC sessions as those users via `sudo -u`. Inside Docker, this requires:
- Running as root
- The `--privileged` flag or specific capabilities
- Shared `/etc/passwd`, `/etc/shadow` between PTY and VNC containers (both need to know about the same users)
**Options:**
- **A) Shared user database:** Mount a shared `/etc/passwd` and `/etc/shadow` between PTY and VNC containers. Both can create users there. Risk: concurrent writes.
- **B) User provisioning service:** A small API that manages Linux users, called by both PTY and VNC sidecars.
- **C) Pre-provision users:** At startup, scan DATA_PATH for existing user directories and create Linux users for each. New users get created on-demand.
- **D) Drop real user isolation:** Map all users to a single Linux user, rely on application-level sandboxing only. Simpler but less secure.
**Recommendation:** Option C with a shared init script. Both containers run the same user-provisioning logic at startup.
### 2. PulseAudio in Docker
**Problem:** PulseAudio expects to manage audio hardware. In a headless Docker container, there's no hardware.
**Options:**
- **A) Null sink approach (current):** Already works — `pactl load-module module-null-sink`. The container just needs PulseAudio running with a null sink. cliamp writes to it, and the audio is captured and streamed over WebSocket.
- **B) PulseAudio socket sharing:** Run PulseAudio on the host, mount the socket into the container.
**Recommendation:** Option A. It already works headless. Just ensure the Dockerfile starts PulseAudio before cliamp.
### 3. Audio Integration with Core
**Problem:** Currently, the `/api/cliamp/audio/ws` endpoint is part of the main server (`src/server.tsx`). It spawns `cliamp` as a child process. If audio moves to a separate container, the WebSocket endpoint and the cliamp process are in different containers.
**Options:**
- **A) Keep audio in core container:** Don't split it out. Add PulseAudio + cliamp deps to the core image. Simplest, but adds ~200MB to the core image.
- **B) Audio sidecar with WebSocket proxy:** The audio container runs its own WebSocket server. The core container proxies `/api/cliamp/*` to it. More complex.
- **C) Audio as a registered sidecar:** Like the other sidecars, audio registers over WebSocket and handles commands. The cleanest architecture but requires refactoring the cliamp code.
**Recommendation:** Option A for now (keep in core). Refactor to Option C later if the image size becomes a problem.
### 4. Chrome/Puppeteer (WhatsApp Bot)
**Problem:** Chrome needs specific flags (`--no-sandbox`, `--disable-dev-shm-usage`) and a larger `/dev/shm` in Docker.
**Solution:** Well-understood. Add to docker-compose:
```yaml
shm_size: '2gb'
```
And install Chrome for Testing via Puppeteer's CLI in the Dockerfile.
### 5. VNC Port Range
**Problem:** VNC dynamically allocates ports 5901-5999 based on available displays. Docker needs these ports mapped.
**Options:**
- **A) Fixed port range:** Map `5901-5910` (10 concurrent sessions max). Simple.
- **B) Host networking:** Use `network_mode: host` for the VNC container. No port mapping needed but less isolation.
- **C) noVNC proxy:** Add a noVNC WebSocket proxy. Clients connect via a single HTTP port, no VNC port range needed. Better for firewalls/reverse proxies.
**Recommendation:** Option C (noVNC) is the best long-term solution. Option A works for now.
### 6. Shared Source Code
**Problem:** All containers need the same source code (sidecars import from the same codebase). Building separate images from the same repo means either:
- Copying the full source into each image (duplicated, large)
- Using a multi-stage build with a shared base
- Mounting the source as a volume (dev only, not portable)
**Recommendation:** Multi-stage Dockerfile. One `base` stage installs OS deps + runtimes + `bun install`. Subsequent stages extend it with container-specific deps (XFCE for VNC, PulseAudio for audio, etc.).
### 7. DATA_PATH Permissions
**Problem:** Multiple containers write to the same DATA_PATH volume. If containers run as different UIDs, file permissions conflict.
**Solution:** All containers should run processes as the same UID (e.g., 1000). The PTY and VNC containers additionally need root to manage sub-users, but the data files should be owned by a consistent UID.
### 8. Pi and Claude Config Directories
**Problem:** Pi reads `~/.pi/agent/auth.json` and `~/.pi/agent/models.json`. Claude reads `~/.claude/`. These are in the container's home directory, which is ephemeral.
**Solution:** Mount these as volumes:
```yaml
volumes:
- ./config/pi:/home/officer/.pi/agent
- ./config/claude:/home/officer/.claude
```
### 9. Native Modules (node-pty, argon2)
**Problem:** `node-pty` and `argon2` have native C/C++ bindings. They need to be compiled for the container's architecture, not the host's.
**Solution:** Run `bun install` inside the Dockerfile (not on the host). The Dockerfile must include `build-essential`, `python3`, and `pkg-config`.
### 10. Image Size
**Concern:** The core image will be large due to Bun + Node.js + Chrome + native deps. The VNC image will be even larger (XFCE alone is ~1.5GB).
**Estimated sizes:**
- `officer-core`: ~2-3 GB (Bun, Node, Chrome, ffmpeg, native deps)
- `officer-pty`: ~1-1.5 GB (Node, shell tools, Go, Rust)
- `officer-vnc`: ~2.5-3 GB (XFCE, TigerVNC, Brave, Bun)
- `officer-audio`: ~500 MB (PulseAudio, cliamp, Go)
**Mitigation:** Multi-stage builds, `.dockerignore`, shared base layers.
---
## Proposed docker-compose.yaml Structure
```yaml
services:
# --- Infrastructure (already dockerized) ---
nginx-proxy-manager:
image: jc21/nginx-proxy-manager:latest
# ... (unchanged)
postgres:
image: postgres:18-alpine
# ... (unchanged)
redis:
image: redis:alpine
# ... (unchanged)
mailhog:
image: mailhog/mailhog:latest
# ... (unchanged)
searxng:
image: searxng/searxng:latest
# ... (unchanged)
static-files:
image: nginx:alpine
# ... (unchanged)
landing-page:
image: nginx:alpine
# ... (unchanged)
# --- Application ---
officer-core:
build:
context: .
dockerfile: docker/Dockerfile.core
container_name: officer-core
restart: unless-stopped
ports:
- "9010:9010"
- "18792:18792"
environment:
- PORT=9010
- NODE_ENV=production
- DATA_PATH=/app/data
- HOME_DIR=/app/data/superadmin/home
- POSTGRES_URL=postgresql://postgres:${PG_PASSWORD}@postgres:5432/officer_dev
- MAIL_TRANSPORT=smtp://mailhog:1025
- PUBLIC_URL=${PUBLIC_URL}
- JWT_SECRET=${JWT_SECRET}
- BROWSER_RELAY_PORT=18792
- DISCORD_BUG_REPORT_WEBHOOK=${DISCORD_BUG_REPORT_WEBHOOK}
volumes:
- officer-data:/app/data
- pi-config:/home/officer/.pi/agent
shm_size: '2gb'
networks:
- services
depends_on:
- postgres
- redis
officer-pty:
build:
context: .
dockerfile: docker/Dockerfile.pty
container_name: officer-pty
restart: unless-stopped
privileged: true
environment:
- API_URL=ws://officer-core:9010/api/sidecar/register
- DATA_PATH=/app/data
volumes:
- officer-data:/app/data
networks:
- services
depends_on:
- officer-core
officer-vnc:
build:
context: .
dockerfile: docker/Dockerfile.vnc
container_name: officer-vnc
restart: unless-stopped
privileged: true
ports:
- "5901-5910:5901-5910"
environment:
- API_URL=ws://officer-core:9010/api/sidecar/register
- DATA_PATH=/app/data
volumes:
- officer-data:/app/data
networks:
- services
depends_on:
- officer-core
volumes:
officer-data:
pi-config:
networks:
services:
external: true
```
---
## Migration Path
### Phase 1: Core container
1. Write `Dockerfile.core` — Bun, Node, Pi, Claude, Chrome, ffmpeg
2. Move the `.env` config to docker-compose environment variables
3. Update `POSTGRES_URL` to use Docker service name (`postgres` not `localhost`)
4. Update `MAIL_TRANSPORT` to use Docker service name (`mailhog`)
5. Test: main server + claude/pi/email sidecars work in container
6. Test: WhatsApp bot (Puppeteer/Chrome) works
### Phase 2: PTY container
1. Write `Dockerfile.pty` — Node, shell tools, user management
2. Refactor PTY sidecar to connect to remote API_URL (currently assumes localhost)
3. Implement user provisioning in container startup
4. Test: terminal sessions work from the web UI
### Phase 3: VNC container
1. Write `Dockerfile.vnc` — XFCE, TigerVNC, Brave
2. Refactor VNC sidecar for remote API_URL
3. Shared user provisioning with PTY container
4. Test: remote desktop sessions work
### Phase 4: Audio (optional)
1. If kept in core: add PulseAudio + cliamp deps to `Dockerfile.core`
2. If separate: write `Dockerfile.audio` and refactor to sidecar protocol
3. Test: music playback works
### Phase 5: Cleanup
1. Remove `scripts/setup.sh` system-level deps (no longer needed)
2. Write backup/restore scripts for Docker volumes
3. Document the new deployment process
4. CI/CD: build images and push to registry
---
## What Stays Outside Docker
- **Nginx Proxy Manager** — already dockerized, unchanged
- **DNS / Tailscale** — host-level networking
- **UFW** — host firewall
- **Hetzner backups** — host-level
- **Cron jobs (pg_dump)** — can move into a container or stay on host
- **SSL certificates** — managed by NPM
---
## Open Questions
1. **Should Go and Rust be in the PTY container?** Users may want to compile Go/Rust projects from the terminal. But it adds significant image size. Could be made optional via a build arg.
2. **Should cliamp be pre-built or built from source?** Building from source requires Go + audio dev headers in the image. A pre-built binary would be smaller but architecture-specific.
3. **How to handle Pi/Claude version updates?** Currently `npm install -g` on the host. In Docker, this means rebuilding the image. Could mount a volume for global npm packages, but that adds complexity.
4. **Do we need a container registry?** For single-server deployments, `docker compose build` is fine. For multi-server, you'd want to push images to a registry (GitHub Container Registry, Docker Hub, self-hosted).
5. **Dev workflow:** How to develop locally? Mount source code as volume + hot reload? Or just develop on host and only use Docker for deployment?