1209 lines
48 KiB
Markdown
1209 lines
48 KiB
Markdown
# Officer Mobile App Plan
|
|
|
|
> This document is written by Claude for Claude. It contains the full plan for native mobile apps (Kotlin/Android + Swift/iOS) for the Officer platform, based on a thorough study of the monorepo. When resuming this work, read this document first, then explain the plan to the user before starting implementation.
|
|
|
|
---
|
|
|
|
## Table of Contents
|
|
|
|
1. [Current Web App Summary](#1-current-web-app-summary)
|
|
2. [What to Include vs Exclude](#2-what-to-include-vs-exclude)
|
|
3. [Architecture Overview](#3-architecture-overview)
|
|
4. [Authentication](#4-authentication)
|
|
5. [Site Map](#5-site-map)
|
|
6. [Screen-by-Screen Breakdown](#6-screen-by-screen-breakdown)
|
|
7. [Tablet Adaptations](#7-tablet-adaptations)
|
|
8. [Technical Challenges](#8-technical-challenges)
|
|
9. [API Integration Notes](#9-api-integration-notes)
|
|
10. [Shared Design Language](#10-shared-design-language)
|
|
11. [Implementation Phases](#11-implementation-phases)
|
|
|
|
---
|
|
|
|
## 1. Current Web App Summary
|
|
|
|
Officer is an AI-assisted personal backend for life management. The web app (`officer-web`) is a React 19 SPA talking to a Hono API server (port 5000). Key features:
|
|
|
|
| Feature | Description |
|
|
|---------|-------------|
|
|
| **Chat** | Multi-provider AI chat (Claude, OpenCode, Pi-Mono) via WebSockets. Sessions persisted as JSON files. Supports images, file attachments, model selection, working directory context. |
|
|
| **File Browser** | Full file manager with upload/download, rename, copy/move, search, archive extraction, git clone, yt-dlp download, OCR, TTS, transcription, audio extraction, video transcoding. Multiple "roots" (home, ~, officer.dev). |
|
|
| **Workspaces** | Multi-panel layouts combining chat, files, terminal, code editor. Panels can be split horizontally/vertically. State persisted per workspace. |
|
|
| **Terminal** | Sandboxed Docker terminals and host terminals (Super Admin). WebSocket-based PTY. |
|
|
| **Code Editor** | Text editor with syntax highlighting (PrismJS). |
|
|
| **Dashboard** | Widget panel (clock, weather, pomodoro, daily goals, quick notes) + workspace launcher + chat launcher. |
|
|
| **Automation** | Builder for workflows, pipelines, processes, services, cron jobs, tasks, skills. |
|
|
| **Tasks** | File-based task definitions (TASK.md) that can be triggered on files/directories. Native/global/user scopes. |
|
|
| **Skills** | AI skill library (SKILL.md files) injected into chat system prompts. |
|
|
| **Settings** | User profile, system settings, server resources, AI harness configuration. |
|
|
| **Auth** | Email/password + passkey (WebAuthn). JWT tokens. Role-based access (Member, Admin, Owner, Super Admin). |
|
|
|
|
The API is at `/api/*` with JWT auth on all protected routes. WebSockets for chat (`/api/harness/*/ws`) and terminal (`/api/terminal/ws`).
|
|
|
|
---
|
|
|
|
## 2. What to Include vs Exclude
|
|
|
|
### Include (core mobile experience)
|
|
|
|
| Feature | Rationale |
|
|
|---------|-----------|
|
|
| **Chat** | The #1 use case. Having an AI assistant in your pocket is the killer feature. Voice input makes it even better on mobile. |
|
|
| **File Browser** | Essential for viewing/managing files. Mobile adds camera upload, share sheet integration. Simplified but fully functional. |
|
|
| **Dashboard / Widgets** | Perfect for mobile - quick glance at clock, weather, goals, notes. Widget-style info is native to mobile platforms. |
|
|
| **Notifications** | Mobile-exclusive feature. Push notifications for chat completions, task results, long downloads finishing. |
|
|
| **Settings** | Profile, preferences. Subset of web settings. |
|
|
| **Session History** | Browse and resume past chat sessions. |
|
|
| **Quick Actions** | Share sheet integration (share URL/text/image to Officer for chat or file upload). |
|
|
|
|
### Include (reduced scope)
|
|
|
|
| Feature | What changes |
|
|
|---------|-------------|
|
|
| **File Viewer** | View files (text, images, PDFs, audio, video) but no full code editor. Read-only text with syntax highlighting. |
|
|
| **Tasks** | View and trigger tasks, see logs. No task creation/editing (too complex for mobile). |
|
|
| **Skills** | Browse skill library. No editing. |
|
|
| **Workspaces** | View workspace list, open a workspace (but as a single-panel experience, not multi-panel). |
|
|
|
|
### Exclude (not suitable for phone)
|
|
|
|
| Feature | Rationale |
|
|
|---------|-----------|
|
|
| **Terminal** | Typing shell commands on a phone keyboard is miserable. The sandboxed Docker terminal is a power-user web feature. |
|
|
| **Code Editor (write mode)** | Writing code on a phone is impractical. Read-only viewing with syntax highlighting is fine. |
|
|
| **Multi-panel workspaces** | Phone screens can't meaningfully split into panels. Tablet gets a simplified version (see section 7). |
|
|
| **Automation Builder** | Complex drag-and-drop/form-heavy workflow builder. Keep this web-only. |
|
|
| **Server/Resource Management** | Admin infrastructure management (installing databases, configuring services). Web-only. |
|
|
| **System Settings** | Server configuration, AI harness setup, plugin management. Web-only. |
|
|
|
|
### Mobile-Exclusive Features (not on web)
|
|
|
|
| Feature | Description |
|
|
|---------|-------------|
|
|
| **Push Notifications** | FCM (Android) / APNs (iOS) for chat completions, task results, downloads. |
|
|
| **Voice Input** | Native speech-to-text for chat messages. Much better than web speech API. |
|
|
| **Camera Integration** | Take photo and send to chat or upload to file browser directly. |
|
|
| **Share Sheet** | Receive shared content (URLs, text, images) from other apps. |
|
|
| **Biometric Auth** | Fingerprint/Face ID as alternative to passkey/password. |
|
|
| **Home Screen Widgets** | Android widgets / iOS widgets for quick notes, pomodoro, weather. |
|
|
| **Offline Queue** | Queue chat messages when offline, send when reconnected. |
|
|
| **Background Downloads** | File downloads and yt-dlp downloads continue in background. |
|
|
|
|
---
|
|
|
|
## 3. Architecture Overview
|
|
|
|
### Platform Strategy
|
|
|
|
Two fully native apps sharing no code, but sharing identical:
|
|
- API contract (same REST + WebSocket endpoints)
|
|
- UX flows and information architecture
|
|
- Design language (adapted to each platform's conventions)
|
|
|
|
```
|
|
┌──────────────┐ ┌──────────────┐
|
|
│ Android App │ │ iOS App │
|
|
│ (Kotlin) │ │ (Swift) │
|
|
│ │ │ │
|
|
│ Jetpack │ │ SwiftUI │
|
|
│ Compose │ │ │
|
|
│ Material 3 │ │ Native iOS │
|
|
│ │ │ Components │
|
|
└──────┬───────┘ └──────┬───────┘
|
|
│ │
|
|
└────────┬───────────┘
|
|
│
|
|
┌──────┴──────┐
|
|
│ Officer API │
|
|
│ (Hono) │
|
|
│ port 5000 │
|
|
└─────────────┘
|
|
```
|
|
|
|
### Android Stack
|
|
|
|
| Layer | Technology |
|
|
|-------|-----------|
|
|
| UI | Jetpack Compose + Material 3 |
|
|
| Navigation | Compose Navigation |
|
|
| Networking | Ktor Client (HTTP + WebSocket) |
|
|
| State | ViewModel + StateFlow |
|
|
| DI | Hilt |
|
|
| Storage | DataStore (preferences), Room (offline cache) |
|
|
| Images | Coil |
|
|
| Push | Firebase Cloud Messaging |
|
|
| Auth | BiometricPrompt API |
|
|
| Background | WorkManager |
|
|
|
|
### iOS Stack
|
|
|
|
| Layer | Technology |
|
|
|-------|-----------|
|
|
| UI | SwiftUI |
|
|
| Navigation | NavigationStack (iOS 16+) |
|
|
| Networking | URLSession + native WebSocket (URLSessionWebSocketTask) |
|
|
| State | @Observable (iOS 17+) / ObservableObject |
|
|
| DI | Swift native (Environment, protocol-based) |
|
|
| Storage | UserDefaults, SwiftData (offline cache) |
|
|
| Images | AsyncImage + custom cache |
|
|
| Push | APNs |
|
|
| Auth | LocalAuthentication (Face ID / Touch ID) |
|
|
| Background | BGTaskScheduler |
|
|
|
|
### Minimum OS Versions
|
|
|
|
- **Android**: API 26 (Android 8.0) - covers 95%+ of devices, gives us WorkManager, BiometricPrompt, etc.
|
|
- **iOS**: iOS 16 - gives us NavigationStack, modern SwiftUI features. iOS 17 for @Observable if we want it.
|
|
|
|
---
|
|
|
|
## 4. Authentication
|
|
|
|
### Flow
|
|
|
|
```
|
|
App Launch
|
|
│
|
|
├─ Has stored JWT? ──Yes──► Validate token (GET /api/auth/me)
|
|
│ │
|
|
│ Valid? ──Yes──► Home Screen
|
|
│ │
|
|
│ No ──► Login Screen
|
|
│
|
|
└─ No ──► Login Screen
|
|
│
|
|
├─ Email/Password
|
|
│ POST /api/auth/signin
|
|
│ ► Store JWT securely
|
|
│ ► Home Screen
|
|
│
|
|
├─ Passkey (WebAuthn)
|
|
│ ► Platform passkey flow
|
|
│ ► Store JWT securely
|
|
│
|
|
└─ Biometric Unlock (returning user)
|
|
► Decrypt stored JWT
|
|
► Validate & proceed
|
|
```
|
|
|
|
### Token Storage
|
|
|
|
- **Android**: EncryptedSharedPreferences (AndroidX Security)
|
|
- **iOS**: Keychain Services
|
|
|
|
### Biometric Integration
|
|
|
|
On first successful login, offer to enable biometric unlock:
|
|
1. User logs in with email/password or passkey
|
|
2. App prompts: "Enable Face ID / Fingerprint for quick access?"
|
|
3. If yes: encrypt JWT with biometric-bound key, store in Keychain/Keystore
|
|
4. On next launch: biometric prompt → decrypt JWT → validate → proceed
|
|
|
|
### Server URL Configuration
|
|
|
|
The mobile app needs to know where the Officer API lives. First-launch flow:
|
|
|
|
1. "Enter your Officer server URL" input
|
|
2. Validate by hitting `GET /api/server-settings/plugins` (public endpoint)
|
|
3. Store URL in app preferences
|
|
4. Proceed to login
|
|
|
|
This is important because Officer is self-hosted - there's no central server.
|
|
|
|
---
|
|
|
|
## 5. Site Map
|
|
|
|
```
|
|
App
|
|
├── Login
|
|
│ ├── Server URL Setup (first launch)
|
|
│ ├── Email/Password
|
|
│ ├── Passkey
|
|
│ └── Biometric Unlock
|
|
│
|
|
├── Home (Tab 1)
|
|
│ ├── Widget Dashboard
|
|
│ │ ├── Clock
|
|
│ │ ├── Weather
|
|
│ │ ├── Pomodoro Timer
|
|
│ │ ├── Daily Goals
|
|
│ │ └── Quick Notes
|
|
│ └── Workspace Quick Access
|
|
│
|
|
├── Chat (Tab 2)
|
|
│ ├── Session List
|
|
│ │ ├── Active Sessions
|
|
│ │ └── Archived Sessions
|
|
│ ├── New Chat
|
|
│ └── Chat Session
|
|
│ ├── Message Thread
|
|
│ ├── Model Selector
|
|
│ ├── Image Attachment
|
|
│ ├── Voice Input
|
|
│ └── Tool Use Display
|
|
│
|
|
├── Files (Tab 3)
|
|
│ ├── Directory Browser
|
|
│ │ ├── Grid/List View
|
|
│ │ ├── Search
|
|
│ │ ├── Sort/Filter
|
|
│ │ └── Context Actions
|
|
│ │ ├── Open/Preview
|
|
│ │ ├── Share
|
|
│ │ ├── Rename
|
|
│ │ ├── Copy/Move
|
|
│ │ ├── Delete
|
|
│ │ ├── Download to Device
|
|
│ │ └── Upload (camera, gallery, files)
|
|
│ └── File Viewer
|
|
│ ├── Text (syntax highlighted, read-only)
|
|
│ ├── Image
|
|
│ ├── PDF
|
|
│ ├── Audio Player
|
|
│ ├── Video Player
|
|
│ └── AI Actions (OCR, TTS, Transcribe)
|
|
│
|
|
├── Activity (Tab 4)
|
|
│ ├── Task List
|
|
│ │ ├── Native Tasks
|
|
│ │ ├── Global Tasks
|
|
│ │ └── User Tasks
|
|
│ ├── Task Logs
|
|
│ └── Running Processes
|
|
│
|
|
└── Settings (Tab 5 / Profile)
|
|
├── Profile
|
|
│ ├── Name, Username, Avatar
|
|
│ ├── Password Change
|
|
│ └── Passkey Management
|
|
├── App Preferences
|
|
│ ├── Theme (Light/Dark/System)
|
|
│ ├── Notification Settings
|
|
│ ├── Default Chat Model
|
|
│ ├── Biometric Lock
|
|
│ └── Server URL
|
|
└── About / Logout
|
|
```
|
|
|
|
### Navigation Pattern
|
|
|
|
**Bottom Tab Bar** with 4 primary tabs + profile access:
|
|
|
|
```
|
|
┌─────────────────────────────────────┐
|
|
│ │
|
|
│ [Screen Content] │
|
|
│ │
|
|
├─────┬─────┬─────┬─────┬────────────┤
|
|
│ Home│ Chat│Files│ Act.│ Settings │
|
|
│ 🏠 │ 💬 │ 📁 │ ⚡ │ ⚙️ │
|
|
└─────┴─────┴─────┴─────┴────────────┘
|
|
```
|
|
|
|
- **Home**: Dashboard with widgets
|
|
- **Chat**: Session list → chat view
|
|
- **Files**: File browser → file viewer
|
|
- **Activity**: Tasks, logs, processes
|
|
- **Settings**: Profile and preferences
|
|
|
|
On Android: Material 3 NavigationBar. On iOS: UITabBarController / SwiftUI TabView.
|
|
|
|
---
|
|
|
|
## 6. Screen-by-Screen Breakdown
|
|
|
|
### 6.1 Login Screen
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ │
|
|
│ [Officer Logo] │
|
|
│ │
|
|
│ ┌───────────────────┐ │
|
|
│ │ Server URL │ │ ← Only on first launch or
|
|
│ └───────────────────┘ │ tap "Change server"
|
|
│ │
|
|
│ ┌───────────────────┐ │
|
|
│ │ Email │ │
|
|
│ └───────────────────┘ │
|
|
│ ┌───────────────────┐ │
|
|
│ │ Password │ │
|
|
│ └───────────────────┘ │
|
|
│ │
|
|
│ [ Sign In ] │
|
|
│ │
|
|
│ ── or ── │
|
|
│ │
|
|
│ [ Sign in with Passkey ] │
|
|
│ │
|
|
│ Forgot password? │
|
|
│ │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
**Behavior:**
|
|
- Server URL field shown on first launch or when tapped
|
|
- Email/password fields with proper keyboard types
|
|
- "Sign in with Passkey" triggers platform WebAuthn flow
|
|
- On success: store JWT, navigate to Home
|
|
- On returning user with biometric enabled: show biometric prompt immediately, with "Use password instead" fallback
|
|
|
|
---
|
|
|
|
### 6.2 Home / Dashboard
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ Officer [Avatar]│
|
|
├─────────────────────────┤
|
|
│ │
|
|
│ ┌──────┐ ┌──────────┐│
|
|
│ │ 14:32│ │ Weather ││
|
|
│ │ Fri │ │ 22° ☀️ ││
|
|
│ └──────┘ └──────────┘│
|
|
│ │
|
|
│ ┌──────────────────┐ │
|
|
│ │ 🍅 Pomodoro │ │
|
|
│ │ 18:42 remaining │ │
|
|
│ │ [Start] [Reset] │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ ┌──────────────────┐ │
|
|
│ │ Daily Goals │ │
|
|
│ │ ☑ Exercise │ │
|
|
│ │ ☐ Read 30min │ │
|
|
│ │ ☐ Journal │ │
|
|
│ │ [+ Add goal] │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ ┌──────────────────┐ │
|
|
│ │ Quick Notes │ │
|
|
│ │ Tap to add... │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ Workspaces │
|
|
│ ┌────┐ ┌────┐ ┌────┐ │
|
|
│ │ WS1│ │ WS2│ │ WS3│ │
|
|
│ └────┘ └────┘ └────┘ │
|
|
│ │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
**Behavior:**
|
|
- Scrollable vertical layout
|
|
- Widgets are cards that can be reordered (long-press drag)
|
|
- Widget data syncs with web app via user settings API
|
|
- Workspace cards show name + icon, tap opens workspace (single-panel mode on phone)
|
|
- Pull-to-refresh updates widget data
|
|
- Avatar tap opens Settings
|
|
|
|
**Widget Implementation:**
|
|
- Clock: Local device time (no API needed)
|
|
- Weather: Can use device location + weather API, or sync from web app settings
|
|
- Pomodoro: Local timer with state synced to server (so it persists across devices)
|
|
- Daily Goals: Stored in user settings, synced
|
|
- Quick Notes: Stored in user settings, synced
|
|
|
|
---
|
|
|
|
### 6.3 Chat - Session List
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ Chat [+ New]│
|
|
├─────────────────────────┤
|
|
│ [🔍 Search sessions...] │
|
|
├─────────────────────────┤
|
|
│ │
|
|
│ Today │
|
|
│ ┌──────────────────┐ │
|
|
│ │ 🤖 Debug login │ │
|
|
│ │ Claude · 2h ago │ │
|
|
│ │ Last: "The issue │ │
|
|
│ │ was in the..." │ │
|
|
│ └──────────────────┘ │
|
|
│ ┌──────────────────┐ │
|
|
│ │ 🤖 Refactor API │ │
|
|
│ │ Pi-Mono · 5h ago │ │
|
|
│ │ Last: "I've upda- │ │
|
|
│ │ ted the router..." │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ Yesterday │
|
|
│ ┌──────────────────┐ │
|
|
│ │ 🤖 Recipe ideas │ │
|
|
│ │ Claude · 1d ago │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ Archived ▸ │
|
|
│ │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
**Behavior:**
|
|
- Sessions fetched from `GET /api/sessions?provider=all`
|
|
- Grouped by date (Today, Yesterday, This Week, Older)
|
|
- Each card shows: session name, provider icon, time, last message preview
|
|
- Swipe left to delete, swipe right to archive
|
|
- Long-press for context menu (rename, archive, delete)
|
|
- "Archived" section expandable
|
|
- Search filters sessions by name
|
|
- FAB or top-right button for new chat
|
|
|
|
**Data Source:**
|
|
- `GET /api/sessions?provider=claude` (and opencode, pi-mono)
|
|
- Sessions are JSON files on the server, API returns metadata
|
|
- Message preview from last assistant message
|
|
|
|
---
|
|
|
|
### 6.4 Chat - Conversation
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ ← Debug login [···] │
|
|
│ Claude 3.5 Sonnet ▾ │
|
|
├─────────────────────────┤
|
|
│ │
|
|
│ ┌─ You ────────────┐ │
|
|
│ │ The login page │ │
|
|
│ │ throws a 401... │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ ┌─ Claude ─────────┐ │
|
|
│ │ Looking at the │ │
|
|
│ │ auth middleware... │ │
|
|
│ │ │ │
|
|
│ │ ┌─ Tool Use ────┐ │ │
|
|
│ │ │ 📄 Read file │ │ │
|
|
│ │ │ auth/jwt.ts │ │ │
|
|
│ │ └──────────────┘ │ │
|
|
│ │ │ │
|
|
│ │ The issue is in │ │
|
|
│ │ the token verify │ │
|
|
│ │ ```ts │ │
|
|
│ │ if (!jti) ... │ │
|
|
│ │ ``` │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
├─────────────────────────┤
|
|
│ ┌───────────────┐ 📎 🎤│
|
|
│ │ Message... │ [Send]│
|
|
│ └───────────────┘ │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
**Behavior:**
|
|
- WebSocket connection to `/api/harness/{provider}/ws`
|
|
- Messages stream in real-time (assistant:partial → assistant:text)
|
|
- Tool use blocks shown as collapsible cards (collapsed by default)
|
|
- Code blocks with syntax highlighting and copy button
|
|
- Markdown rendering for assistant messages
|
|
- Model selector dropdown in the header (or a bottom sheet)
|
|
- Attachment button (📎) opens picker: Camera, Photo Library, Files
|
|
- Microphone button (🎤) for voice input (speech-to-text → send as text)
|
|
- Send button disabled while assistant is generating
|
|
- "Stop" button appears during generation
|
|
- Pull-down to load older messages (if session has many)
|
|
- `[···]` menu: Rename session, View in files, Delete, Share
|
|
|
|
**WebSocket Message Protocol:**
|
|
```
|
|
Client sends:
|
|
{ type: 'chat', prompt, sessionId, model, images?, attachmentIds? }
|
|
{ type: 'resume', sessionId }
|
|
{ type: 'stop' }
|
|
|
|
Server sends:
|
|
{ type: 'session:init', sessionId, name }
|
|
{ type: 'assistant:partial', text }
|
|
{ type: 'assistant:text', text }
|
|
{ type: 'tool:use', name, input }
|
|
{ type: 'tool:result', result }
|
|
{ type: 'result', cost?, duration?, turns? }
|
|
{ type: 'error', message }
|
|
{ type: 'stopped' }
|
|
```
|
|
|
|
**Offline Behavior:**
|
|
- If offline when sending, queue message locally
|
|
- Show "Queued - will send when online" indicator
|
|
- On reconnect, send queued messages in order
|
|
- Previously loaded messages remain visible offline (cached in local DB)
|
|
|
|
**Voice Input:**
|
|
- Tap and hold microphone for continuous recording
|
|
- Or tap once to start, tap again to stop
|
|
- Platform native speech recognition
|
|
- Transcribed text placed in input field for review before sending
|
|
- User can edit before sending
|
|
|
|
---
|
|
|
|
### 6.5 Files - Directory Browser
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ ← Files [···] │
|
|
│ / > Documents > Work │
|
|
├─────────────────────────┤
|
|
│ [🔍 Search files...] │
|
|
├─────────────────────────┤
|
|
│ Sort: Name ▾ [≡] [⊞] │
|
|
├─────────────────────────┤
|
|
│ │
|
|
│ 📁 Projects → │
|
|
│ 📁 Reports → │
|
|
│ 📄 notes.md 3KB │
|
|
│ 🖼️ photo.jpg 1.2MB │
|
|
│ 🎵 song.mp3 4.5MB │
|
|
│ 📹 demo.mp4 45.2MB │
|
|
│ 📦 backup.zip 12.0MB │
|
|
│ │
|
|
│ [+ Upload] │
|
|
│ │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
**Behavior:**
|
|
- Breadcrumb navigation at top (scrollable horizontal)
|
|
- Tap folder to navigate in, tap file to open viewer
|
|
- Long-press for context menu:
|
|
- Open / Preview
|
|
- Share (native share sheet)
|
|
- Rename
|
|
- Copy / Move (enter selection mode)
|
|
- Delete
|
|
- Download to device
|
|
- AI actions (OCR, TTS, Transcribe - based on file type)
|
|
- Pull-to-refresh
|
|
- FAB or bottom button for upload (Camera, Photo Library, Files picker)
|
|
- Multi-select mode (long-press first item, then tap others)
|
|
- Swipe actions: left = delete, right = share
|
|
- Grid/List view toggle
|
|
- Sort by name/date/size
|
|
- Search with debounce (calls `/api/file-browser/search`)
|
|
|
|
**Root Selection:**
|
|
- Dropdown or segmented control at top for root selection
|
|
- Only show non-home roots for Super Admin users
|
|
- Default to "home"
|
|
|
|
**Upload Sources:**
|
|
- Device file picker
|
|
- Camera (take photo)
|
|
- Photo library
|
|
- Share sheet (from other apps)
|
|
- Drag & drop (tablet only)
|
|
|
|
---
|
|
|
|
### 6.6 Files - File Viewer
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ ← notes.md [↗] [···]│
|
|
├─────────────────────────┤
|
|
│ │
|
|
│ # Meeting Notes │
|
|
│ │
|
|
│ ## Action Items │
|
|
│ - Fix login bug │
|
|
│ - Update docs │
|
|
│ - Review PR #42 │
|
|
│ │
|
|
│ ## Discussion │
|
|
│ We talked about the │
|
|
│ new mobile app and... │
|
|
│ │
|
|
│ │
|
|
│ │
|
|
│ │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
**Viewer Types:**
|
|
|
|
| File Type | Viewer | Notes |
|
|
|-----------|--------|-------|
|
|
| `.md` | Rendered Markdown | With proper styling |
|
|
| `.txt`, `.json`, `.ts`, `.js`, etc. | Syntax-highlighted text | Read-only, with line numbers |
|
|
| `.jpg`, `.png`, `.gif`, `.svg`, `.webp` | Image viewer | Pinch to zoom, pan |
|
|
| `.pdf` | PDF viewer | Native PDF rendering |
|
|
| `.mp3`, `.wav`, `.flac`, `.ogg` | Audio player | Play/pause, seek, speed control |
|
|
| `.mp4`, `.webm`, `.mov` | Video player | Native player, fullscreen support |
|
|
| `.mkv`, `.avi` | Video player (transcoded) | Uses `/api/file-browser/transcode` endpoint |
|
|
| Other | Download prompt | "This file type can't be previewed. Download?" |
|
|
|
|
**Header Actions:**
|
|
- [↗] Share via native share sheet
|
|
- [···] Menu: Download to device, AI actions (contextual), Open in chat, Delete
|
|
|
|
**AI Actions (contextual):**
|
|
- Images: OCR (extract text)
|
|
- Text files: TTS (read aloud)
|
|
- Audio/Video: Transcribe
|
|
- Video: Extract audio
|
|
|
|
---
|
|
|
|
### 6.7 Activity
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ Activity │
|
|
├─────────────────────────┤
|
|
│ [Tasks] [Logs] [Procs] │
|
|
├─────────────────────────┤
|
|
│ │
|
|
│ Native Tasks │
|
|
│ ┌──────────────────┐ │
|
|
│ │ 📋 Format Code │ │
|
|
│ │ Formats source │ │
|
|
│ │ files with... │ │
|
|
│ │ [Run ▶] │ │
|
|
│ └──────────────────┘ │
|
|
│ ┌──────────────────┐ │
|
|
│ │ 📋 Compress Imgs │ │
|
|
│ │ Optimizes image │ │
|
|
│ │ files in... │ │
|
|
│ │ [Run ▶] │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ User Tasks │
|
|
│ ┌──────────────────┐ │
|
|
│ │ 📋 Backup DB │ │
|
|
│ │ Runs pg_dump... │ │
|
|
│ │ [Run ▶] │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
**Tabs:**
|
|
1. **Tasks**: List of available tasks (native/global/user). Each shows name, description, trigger types. "Run" button opens a sheet to select target file/directory.
|
|
2. **Logs**: Recent task execution logs. Each shows task name, status (success/failure), timestamp, duration. Tap to see full log output.
|
|
3. **Processes**: Currently running background processes. Show name, status, uptime. No start/stop controls (too dangerous on mobile - manage via web).
|
|
|
|
**Task Runner:**
|
|
- When "Run" is tapped, show a bottom sheet:
|
|
- Select target file/directory (mini file browser)
|
|
- Confirm and run
|
|
- Show real-time output in a scrollable log view
|
|
- Toast on completion
|
|
|
|
---
|
|
|
|
### 6.8 Settings
|
|
|
|
```
|
|
┌─────────────────────────┐
|
|
│ Settings │
|
|
├─────────────────────────┤
|
|
│ │
|
|
│ ┌──────────────────┐ │
|
|
│ │ 👤 John Doe │ │
|
|
│ │ john@example.com │ │
|
|
│ │ Super Admin │ │
|
|
│ └──────────────────┘ │
|
|
│ │
|
|
│ Profile │
|
|
│ Name, Username, Avatar │
|
|
│ Change Password → │
|
|
│ Passkeys → │
|
|
│ │
|
|
│ ───────────────────── │
|
|
│ │
|
|
│ App │
|
|
│ Theme System │
|
|
│ Notifications → │
|
|
│ Default Model Claude │
|
|
│ Biometric Lock [🔘] │
|
|
│ │
|
|
│ ───────────────────── │
|
|
│ │
|
|
│ Connection │
|
|
│ Server 192.168.1.5 → │
|
|
│ Status Connected 🟢│
|
|
│ │
|
|
│ ───────────────────── │
|
|
│ │
|
|
│ [ Sign Out ] │
|
|
│ │
|
|
│ Officer v1.0.0 │
|
|
│ │
|
|
└─────────────────────────┘
|
|
```
|
|
|
|
**Sections:**
|
|
1. **Profile card**: Avatar, name, email, role. Tap to edit.
|
|
2. **Profile**: Name/username editing, avatar upload (camera or gallery), password change, passkey management.
|
|
3. **App**: Theme (light/dark/system), notification preferences, default chat model/provider, biometric lock toggle.
|
|
4. **Connection**: Current server URL (editable), connection status indicator, last sync time.
|
|
5. **Sign Out**: Clears JWT, biometric keys, navigates to login.
|
|
|
|
---
|
|
|
|
## 7. Tablet Adaptations
|
|
|
|
Tablets (iPad, Android tablets/foldables) get an enhanced layout while using the same app binary. Detection via screen width:
|
|
- **Phone**: < 600dp (Android) / Compact (iOS)
|
|
- **Tablet**: >= 600dp (Android) / Regular (iOS)
|
|
|
|
### 7.1 Navigation
|
|
|
|
**Phone**: Bottom tab bar (5 tabs)
|
|
**Tablet**: Side navigation rail (Android) / Sidebar (iOS)
|
|
|
|
```
|
|
Tablet Layout:
|
|
┌────┬──────────────────────────────┐
|
|
│ │ │
|
|
│ 🏠 │ │
|
|
│ 💬 │ [Main Content] │
|
|
│ 📁 │ │
|
|
│ ⚡ │ │
|
|
│ │ │
|
|
│ │ │
|
|
│ ⚙️ │ │
|
|
└────┴──────────────────────────────┘
|
|
```
|
|
|
|
### 7.2 Chat on Tablet
|
|
|
|
**Split view**: Session list on left (1/3), conversation on right (2/3).
|
|
|
|
```
|
|
┌────┬──────────┬───────────────────┐
|
|
│ │Sessions │ Debug login │
|
|
│ 🏠 │──────────│ Claude 3.5 Sonnet │
|
|
│ 💬 │ Debug.. │───────────────────│
|
|
│ 📁 │ Refact.. │ │
|
|
│ ⚡ │ Recipe.. │ [Messages...] │
|
|
│ │ │ │
|
|
│ │ │ │
|
|
│ │ ├───────────────────│
|
|
│ ⚙️ │ │ [Input] [Send] │
|
|
└────┴──────────┴───────────────────┘
|
|
```
|
|
|
|
### 7.3 Files on Tablet
|
|
|
|
**Split view**: Directory browser on left, file viewer on right. Tap a file to preview it in the right pane without navigating away.
|
|
|
|
```
|
|
┌────┬──────────┬───────────────────┐
|
|
│ │ / > Docs │ notes.md │
|
|
│ 🏠 │──────────│───────────────────│
|
|
│ 💬 │ 📁 Proj │ # Meeting Notes │
|
|
│ 📁 │ 📁 Reps │ │
|
|
│ ⚡ │ 📄 note │ ## Action Items │
|
|
│ │ 🖼️ phot │ - Fix login bug │
|
|
│ │ 🎵 song │ - Update docs │
|
|
│ │ │ │
|
|
│ ⚙️ │ │ │
|
|
└────┴──────────┴───────────────────┘
|
|
```
|
|
|
|
### 7.4 Dashboard on Tablet
|
|
|
|
Widgets in a 2-3 column grid instead of single column. More widgets visible at once.
|
|
|
|
```
|
|
┌────┬──────────────────────────────┐
|
|
│ │ ┌──────┐ ┌──────┐ ┌──────┐ │
|
|
│ 🏠 │ │Clock │ │Weath.│ │Pomo │ │
|
|
│ 💬 │ └──────┘ └──────┘ └──────┘ │
|
|
│ 📁 │ ┌─────────────┐ ┌──────────┐│
|
|
│ ⚡ │ │ Daily Goals │ │ Quick ││
|
|
│ │ │ ☑ Exercise │ │ Notes ││
|
|
│ │ │ ☐ Read 30min │ │ ... ││
|
|
│ │ └─────────────┘ └──────────┘│
|
|
│ ⚙️ │ Workspaces │
|
|
│ │ ┌────┐ ┌────┐ ┌────┐ ┌────┐ │
|
|
└────┴──────────────────────────────┘
|
|
```
|
|
|
|
### 7.5 Tablet-Only Features
|
|
|
|
| Feature | Description |
|
|
|---------|-------------|
|
|
| **Drag & Drop** | Drag files between panes, drag text into chat |
|
|
| **Keyboard Shortcuts** | External keyboard support with shortcuts (Cmd+N for new chat, etc.) |
|
|
| **Picture-in-Picture** | Video playback continues in PiP when navigating away |
|
|
| **Multi-Window** | Android: Split screen support. iPad: Stage Manager support. |
|
|
|
|
---
|
|
|
|
## 8. Technical Challenges
|
|
|
|
### 8.1 WebSocket Lifecycle on Mobile
|
|
|
|
**Problem**: Mobile OSes aggressively kill background connections. WebSocket to chat/terminal will disconnect when app is backgrounded.
|
|
|
|
**Solution**:
|
|
- Detect app going to background → gracefully close WebSocket
|
|
- On foreground → reconnect WebSocket, resume session
|
|
- Use `{ type: 'resume', sessionId }` to restore chat state
|
|
- Buffer any pending assistant output server-side (the server already does this - messages are saved to JSON files)
|
|
- Show "Reconnecting..." indicator when connection drops
|
|
- Implement exponential backoff for reconnection attempts
|
|
|
|
**Android specifics:**
|
|
- Use `ProcessLifecycleOwner` to detect app lifecycle
|
|
- Consider foreground service for active chat sessions (shows notification "Chat in progress")
|
|
|
|
**iOS specifics:**
|
|
- Use `UIApplication.willResignActiveNotification` / `didBecomeActiveNotification`
|
|
- Background URL session for critical requests
|
|
- Use `BGAppRefreshTask` for periodic sync
|
|
|
|
### 8.2 Push Notifications
|
|
|
|
**Problem**: The Officer API server currently has no push notification infrastructure.
|
|
|
|
**Required Backend Changes:**
|
|
1. New endpoint: `POST /api/devices/register` — stores FCM/APNs token per user
|
|
2. New database table: `DeviceTokens` (userId, token, platform, createdAt)
|
|
3. Server-side notification triggers:
|
|
- Chat: When assistant finishes responding and user's WebSocket is disconnected
|
|
- Tasks: When a long-running task completes
|
|
- Downloads: When yt-dlp or git clone finishes
|
|
4. Firebase Admin SDK (Android) + APNs library (iOS) on the server
|
|
|
|
**Notification Types:**
|
|
| Type | Title | Body | Action |
|
|
|------|-------|------|--------|
|
|
| Chat complete | "Claude responded" | First 100 chars of response | Open chat session |
|
|
| Task complete | "Task finished" | "Format Code completed successfully" | Open task log |
|
|
| Download complete | "Download ready" | "video.mp4 downloaded" | Open file browser |
|
|
| Error | "Task failed" | "Backup DB failed: connection refused" | Open task log |
|
|
|
|
### 8.3 File Upload from Mobile
|
|
|
|
**Problem**: Mobile file uploads have different sources than web (camera, photo library, share sheet).
|
|
|
|
**Solution**:
|
|
- Use platform file pickers that support multiple sources
|
|
- Camera capture: Take photo → upload via `POST /api/file-browser/upload`
|
|
- Compress images before upload (configurable quality setting)
|
|
- Show upload progress (the existing XHR-based upload in `useFiles.ts` supports progress)
|
|
- Background upload for large files using WorkManager (Android) / BGTaskScheduler (iOS)
|
|
- Resume interrupted uploads if possible
|
|
|
|
### 8.4 Offline Support
|
|
|
|
**Problem**: Users may open the app with no connectivity.
|
|
|
|
**Strategy**: Read-heavy offline, write-queued.
|
|
|
|
| Feature | Offline Behavior |
|
|
|---------|-----------------|
|
|
| Dashboard | Cached widget data, local timer for pomodoro |
|
|
| Chat history | Cached session list and recent messages |
|
|
| New chat | Queue message, show "Will send when online" |
|
|
| File browser | Show last cached directory listing |
|
|
| File viewer | Show cached/downloaded files only |
|
|
| Settings | Show cached profile, edits queued |
|
|
|
|
**Local Database Schema (Room / SwiftData):**
|
|
```
|
|
ChatSessions: id, name, provider, lastMessagePreview, updatedAt, isArchived
|
|
ChatMessages: id, sessionId, role, content, timestamp (cache recent N per session)
|
|
DirectoryCache: path, root, entriesJson, cachedAt
|
|
FileCache: path, root, localPath, size, cachedAt (for downloaded/viewed files)
|
|
PendingActions: id, type, payload, createdAt, status
|
|
```
|
|
|
|
### 8.5 Deep Linking & Share Sheet
|
|
|
|
**URL Scheme**: `officer://`
|
|
**Universal Links**: `https://officer.dev/app/...` (if applicable)
|
|
|
|
**Deep Link Routes:**
|
|
- `officer://chat/{sessionId}` — open specific chat
|
|
- `officer://files?path=/Documents` — open file browser at path
|
|
- `officer://chat/new?prompt=...` — start new chat with prefilled text
|
|
|
|
**Share Sheet Integration (receiving):**
|
|
- Register app as share target for: text, URLs, images, files
|
|
- When content is shared to Officer:
|
|
1. Show a sheet: "Send to Chat" or "Upload to Files"
|
|
2. If Chat: open new chat with shared content as first message
|
|
3. If Files: open file browser with upload in current directory
|
|
|
|
### 8.6 Image Handling in Chat
|
|
|
|
**Problem**: The web app sends images as base64 in the WebSocket message. Large images on mobile could cause memory issues.
|
|
|
|
**Solution**:
|
|
1. Before sending, resize images to max 2048px on longest edge
|
|
2. Compress to JPEG at 85% quality (unless PNG is needed for screenshots)
|
|
3. Convert to base64 and include in chat message
|
|
4. Show thumbnail in the message thread, tap to view full-size
|
|
5. Consider using the upload endpoint first, then sending attachment IDs instead of base64 (check if API supports this path — it does via `attachmentIds` in the chat message)
|
|
|
|
### 8.7 Markdown & Code Rendering
|
|
|
|
**Problem**: Rendering rich markdown with code blocks, tables, and images in native views.
|
|
|
|
**Android Solution:**
|
|
- Use a Markdown library (Markwon or compose-markdown)
|
|
- Code blocks: custom span with monospace font + syntax highlighting
|
|
- Consider a lightweight WebView-based renderer for complex markdown as fallback
|
|
|
|
**iOS Solution:**
|
|
- Use `AttributedString` with custom markdown parsing
|
|
- Or use a SwiftUI markdown library (MarkdownUI)
|
|
- Code blocks: custom view with monospace font + syntax highlighting
|
|
|
|
**Code Syntax Highlighting:**
|
|
- Android: Highlight.js via WebView, or a native library like CodeView
|
|
- iOS: Splash or Highlightr library
|
|
|
|
### 8.8 Streaming Text Display
|
|
|
|
**Problem**: Chat responses stream token-by-token via WebSocket. Need smooth text rendering without layout thrashing.
|
|
|
|
**Solution**:
|
|
- Buffer incoming tokens (e.g., collect for 50ms before rendering)
|
|
- Use a single mutable text state that appends
|
|
- Avoid re-rendering the entire message list on each token
|
|
- Only the last (active) message bubble needs updating
|
|
- Auto-scroll to bottom during streaming, unless user has scrolled up
|
|
|
|
### 8.9 Security
|
|
|
|
**Concerns:**
|
|
- JWT stored on device must be encrypted
|
|
- Biometric key binding for stored credentials
|
|
- Certificate pinning for API connections (optional but recommended for self-hosted)
|
|
- No sensitive data in logs
|
|
- Clear all data on sign out
|
|
- Jailbreak/root detection (optional)
|
|
|
|
**Android:**
|
|
- EncryptedSharedPreferences for tokens
|
|
- Android Keystore for biometric-bound keys
|
|
- ProGuard/R8 for release builds
|
|
|
|
**iOS:**
|
|
- Keychain with `kSecAttrAccessibleWhenUnlockedThisDeviceOnly`
|
|
- Secure Enclave for biometric keys
|
|
- App Transport Security (ATS) enforced
|
|
|
|
### 8.10 Self-Hosted Server Discovery
|
|
|
|
**Problem**: Officer is self-hosted. Users need to point the app at their server. The server might be on a local network, behind a VPN, or on the public internet.
|
|
|
|
**Solution**:
|
|
- First-launch: manual URL entry with validation
|
|
- Local network: mDNS/Bonjour discovery (optional nice-to-have)
|
|
- QR code scanning from web app settings page (generate QR with server URL + optional auth token)
|
|
- Store multiple server profiles (for users who have dev/prod setups)
|
|
- Connection status indicator in settings
|
|
|
|
---
|
|
|
|
## 9. API Integration Notes
|
|
|
|
### Endpoints the Mobile App Will Use
|
|
|
|
**Auth:**
|
|
- `POST /api/auth/signin` — login
|
|
- `POST /api/auth/signup` — register (if enabled)
|
|
- `POST /api/auth/signout` — logout
|
|
- `GET /api/auth/me` — validate token, get user info
|
|
- `POST /api/auth/change-password` — password change
|
|
- `*/passkeys/*` — passkey registration/auth
|
|
|
|
**Chat:**
|
|
- `GET /api/sessions?provider={provider}` — list sessions
|
|
- `DELETE /api/sessions/{provider}/{id}` — delete session
|
|
- `POST /api/sessions/{provider}/{id}/rename` — rename session
|
|
- `POST /api/sessions/{provider}/{id}/archive` — archive session
|
|
- WebSocket `/api/harness/{provider}/ws` — chat communication
|
|
|
|
**Files:**
|
|
- `GET /api/file-browser/ls?path={path}&root={root}` — list directory
|
|
- `GET /api/file-browser/read?path={path}` — read text file
|
|
- `GET /api/file-browser/raw?path={path}` — raw file (binary, supports Range)
|
|
- `GET /api/file-browser/search?q={query}` — search files
|
|
- `POST /api/file-browser/mkdir` — create directory
|
|
- `POST /api/file-browser/write` — write file
|
|
- `POST /api/file-browser/upload` — upload files (multipart)
|
|
- `POST /api/file-browser/rename` — rename
|
|
- `POST /api/file-browser/copy` — copy
|
|
- `POST /api/file-browser/move` — move
|
|
- `DELETE /api/file-browser/rm` — delete
|
|
- `GET /api/file-browser/download?path={path}` — download file
|
|
- `POST /api/file-browser/download` — download multiple as zip
|
|
- `GET /api/file-browser/transcode?path={path}` — transcode video
|
|
- `POST /api/file-browser/tts` — text-to-speech
|
|
- `POST /api/file-browser/ocr` — OCR image
|
|
- `POST /api/file-browser/transcribe` — transcribe audio
|
|
- `POST /api/file-browser/extract-audio` — extract audio from video
|
|
- `POST /api/file-browser/extract` — extract archive
|
|
- `POST /api/file-browser/git-clone` — clone git repo
|
|
- `POST /api/file-browser/download-video` — yt-dlp download
|
|
|
|
**Tasks:**
|
|
- `GET /api/tasks` — list tasks
|
|
- `GET /api/tasks/{name}` — task details
|
|
- `POST /api/tasks/{name}/run` — run task
|
|
|
|
**Task Logs:**
|
|
- `GET /api/task-logs` — list logs
|
|
- `GET /api/task-logs/{id}` — log details
|
|
|
|
**Processes:**
|
|
- `GET /api/processes` — list processes
|
|
|
|
**Settings:**
|
|
- `GET /api/user/settings` — user settings
|
|
- `PUT /api/user/settings` — update settings
|
|
- `PUT /api/auth/users` — update profile
|
|
|
|
**Upload:**
|
|
- `POST /api/upload` — upload attachments (for chat)
|
|
|
|
**Scrape:**
|
|
- `POST /api/scrape` — scrape URL (for sharing URLs to chat)
|
|
|
|
### New Endpoints Needed
|
|
|
|
| Endpoint | Purpose |
|
|
|----------|---------|
|
|
| `POST /api/devices/register` | Register push notification token |
|
|
| `DELETE /api/devices/{token}` | Unregister device on logout |
|
|
| `GET /api/sessions/{provider}/{id}/messages` | Get messages for a session (currently read from files, need API endpoint) |
|
|
|
|
### Authentication Header
|
|
|
|
All requests include: `Authorization: Bearer {jwt_token}`
|
|
|
|
WebSocket connections pass token as query parameter: `?token={jwt_token}`
|
|
|
|
---
|
|
|
|
## 10. Shared Design Language
|
|
|
|
While each platform follows its native design guidelines (Material 3 for Android, Human Interface Guidelines for iOS), some design elements should be consistent:
|
|
|
|
### Color Palette
|
|
|
|
From the web app's Tailwind theme:
|
|
- **Primary (Teal)**: `duck-teal` — used for primary actions, active states
|
|
- **Accent (Yellow)**: `duck-yellow` — used for text on primary, highlights
|
|
- **Dark**: `duck-dark` — text, icons
|
|
- **Background**: Light/dark mode adaptive
|
|
|
|
Both apps should use these brand colors while adapting to platform conventions:
|
|
- Android: Material 3 dynamic color with Officer palette as seed
|
|
- iOS: Officer palette with system-adaptive backgrounds
|
|
|
|
### Typography
|
|
|
|
- **Android**: Roboto (system default), with monospace for code
|
|
- **iOS**: SF Pro (system default), with SF Mono for code
|
|
- Both: Similar size scale, consistent heading hierarchy
|
|
|
|
### Iconography
|
|
|
|
- **Android**: Material Symbols (outlined)
|
|
- **iOS**: SF Symbols
|
|
- Map lucide-react icons used in web to platform equivalents
|
|
|
|
### Animations
|
|
|
|
- **Android**: Material Motion (shared element transitions, container transforms)
|
|
- **iOS**: Spring animations, matched geometry effects
|
|
- Both: Smooth transitions between screens, subtle feedback on interactions
|
|
|
|
---
|
|
|
|
## 11. Implementation Phases
|
|
|
|
### Phase 1: Foundation (MVP)
|
|
|
|
**Goal**: Core functionality that makes the app useful enough to ship.
|
|
|
|
| Feature | Details |
|
|
|---------|---------|
|
|
| Auth | Email/password login, token storage, biometric unlock |
|
|
| Server setup | URL configuration, validation |
|
|
| Chat | New chat, session list, real-time conversation with streaming |
|
|
| Chat basics | Text messages, model selection, session management |
|
|
| Settings | Profile view, theme, logout |
|
|
|
|
**Estimated screens**: Login, Home (simple), Chat List, Chat Conversation, Settings
|
|
|
|
### Phase 2: Files & Polish
|
|
|
|
**Goal**: File management and mobile-specific enhancements.
|
|
|
|
| Feature | Details |
|
|
|---------|---------|
|
|
| File browser | Directory listing, navigation, search |
|
|
| File viewer | Text, images, PDF, audio, video |
|
|
| File actions | Upload (camera, files), download, rename, delete |
|
|
| Chat attachments | Image attachment, voice input |
|
|
| Push notifications | Basic chat completion notifications |
|
|
|
|
**Estimated screens**: File Browser, File Viewer, enhanced Chat
|
|
|
|
### Phase 3: Productivity
|
|
|
|
**Goal**: Dashboard, tasks, and deeper integration.
|
|
|
|
| Feature | Details |
|
|
|---------|---------|
|
|
| Dashboard | Widget system (clock, weather, pomodoro, goals, notes) |
|
|
| Tasks | Task list, run tasks, view logs |
|
|
| Workspace quick access | Open workspaces (single-panel) |
|
|
| Share sheet | Receive content from other apps |
|
|
| Offline support | Cached data, queued actions |
|
|
|
|
**Estimated screens**: Dashboard, Task List, Task Logs, enhanced Home
|
|
|
|
### Phase 4: Tablet & Advanced
|
|
|
|
**Goal**: Tablet optimization and advanced features.
|
|
|
|
| Feature | Details |
|
|
|---------|---------|
|
|
| Tablet layouts | Split views for chat, files |
|
|
| Tablet navigation | Side rail / sidebar |
|
|
| Home screen widgets | Android widgets, iOS widgets |
|
|
| AI file actions | OCR, TTS, Transcribe from mobile |
|
|
| Multi-select | Batch file operations |
|
|
| Deep linking | URL scheme, universal links |
|
|
|
|
### Phase 5: Nice-to-Haves
|
|
|
|
| Feature | Details |
|
|
|---------|---------|
|
|
| QR code server setup | Scan from web to configure |
|
|
| Local network discovery | mDNS/Bonjour |
|
|
| Picture-in-Picture | Video PiP on tablet |
|
|
| Passkey auth | Full WebAuthn support |
|
|
| Keyboard shortcuts | External keyboard on tablet |
|
|
| Skills browser | View and search skills library |
|
|
| Background downloads | yt-dlp and git clone in background |
|
|
|
|
---
|
|
|
|
## Appendix: Decision Log
|
|
|
|
| Decision | Rationale |
|
|
|----------|-----------|
|
|
| Native (Kotlin + Swift) over cross-platform | User preference. Also gives best performance, platform-native UX, and full access to platform APIs (biometrics, push, background tasks, share sheet). |
|
|
| Exclude terminal | Phone keyboards make shell commands impractical. The sandboxed Docker setup is inherently desktop-oriented. |
|
|
| Exclude code editor (write) | Writing code on a phone is poor UX. Read-only with syntax highlighting is sufficient for review. |
|
|
| Exclude automation builder | Complex form-heavy UI that doesn't translate to small screens. |
|
|
| Include chat as primary feature | AI assistant in your pocket is the strongest mobile use case. Voice input makes it even more compelling on mobile. |
|
|
| Bottom tabs (phone) / Side rail (tablet) | Standard navigation patterns for each form factor. 5 top-level destinations fits within guidelines. |
|
|
| Offline-first for reads, queue for writes | Mobile connectivity is unreliable. Showing cached data is better than empty screens. |
|
|
| Phase 1 = Auth + Chat | Chat is the killer feature. Get it right first, everything else is complementary. |
|