officerdev/platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

vnc: back to mirroring the physical screen, at 1920x1080

Browse Source 
Reverts the Xvnc virtual-desktop work (83bf746, f22c667). The separate desktop was the right answer to
"4K on the TV and 1080p remote", but it brought a chain of its own problems — a dock left stranded
below the bottom of the screen after every resize, because xfce4-panel does not follow a RandR change
reliably — and the owner would rather have one session that works than two that need supervision.

So: one GNOME session, mirrored, with the TV set to 1920x1080. That is under SCALE_ABOVE_WIDTH, so
x11vnc serves it 1:1 with no scaling, and both ends see the same 1920x1080 desktop.

resizeSession is now off on the client — a mirror reflects a physical screen and cannot be resized.
scaleViewport stays ON and is load-bearing: noVNC maps a click as
(clientX - canvasRect.left) / display._scale, and autoscale() is the only code that sets the canvas's
displayed size and _scale in the same call. Disabling it pins _scale at 1 while the canvas is displayed
at some other size, which doubled every pointer coordinate. That was my change and my bug.

Kept from the Xvnc detour, because all of it applies to the mirror too: the display is discovered by
socket ownership rather than assuming :0 (GDM gives :0 to its greeter), -noxdamage is gone, the clip to
the primary output stays, noVNC loads as one bundle, and showDotCursor covers the invisible pointer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
pastilhas
2026-08-01 23:35:22 +00:00
co-authored by Claude Opus 5
parent 5ba89ae74d
commit 25da4dc31d
2 changed files with 171 additions and 157 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/servers/sidecar/vnc/vnc-manager.ts
+165 -146
View File
@@ -1,45 +1,71 @@
import { existsSync, mkdirSync } from 'node:fs';
import { randomBytes } from 'node:crypto';
import { existsSync, mkdirSync, readdirSync, statSync } from 'node:fs';
import { join } from 'node:path';
import type { VncStartParams, VncSessionInfo } from '../protocol';
import { getOwnerHomeDir } from '@@/data-path';
// A dedicated virtual desktop for remote access, NOT a mirror of the physical screen.
//
// This replaces mirroring, and the reason is a hard constraint rather than a preference: the TV runs at
// 4K so it can play 4K video, while a usable remote desktop wants ~1080p. One framebuffer cannot be both,
// and mirroring meant everything remote was a scaled-down 4K desktop — dense, and expensive to encode.
//
// Xvnc (TigerVNC) is the X server AND the VNC server in one process. Nothing polls or scales: the X
// server knows exactly which rectangles changed and encodes them directly, where x11vnc had to diff a
// framebuffer it did not own. It also implements RandR SetDesktopSize, so the noVNC client's
// `resizeSession` makes the desktop resize itself to the browser panel — no scaling on either side, at
// any panel size.
//
// XFCE, not GNOME, and this is NOT about it being lighter. Ubuntu's GNOME is managed by per-USER systemd
// units — org.gnome.Shell@x11.service, gnome-session-manager@ubuntu.service and the whole
// org.gnome.SettingsDaemon.* set all live under user@<uid>.service, and gnome-session@.target is marked
// RefuseManualStart. A second GNOME session for the SAME user collides with every one of those. That is
// almost certainly what b2f0d13 recorded as "a ghost logind session that broke lightdm login". XFCE has
// no such per-user units, so it coexists with the GNOME session on the TV — same user, same home, same
// files, just a different shell.
const GEOMETRY = '1920x1080';
const DEPTH = 24;
const VNC_PORT = 5900;
const READY_TIMEOUT_MS = 15_000;
const FIRST_DISPLAY = 2;
const MAX_DISPLAY = 32;
// Mirrors the physical display instead of spawning a virtual desktop per user, so the
// browser shows the same session as the screen. There is one seat, hence one shared
// mirror — the first caller's VNC password is the one it answers with.
const MIRROR_PORT = 5900;
const READY_TIMEOUT_MS = 5000;
// Above this framebuffer width the stream is halved; at or below it, pixels are served 1:1.
const SCALE_ABOVE_WIDTH = 2560;
type VirtualDesktop = {
// The display number is NOT fixed. Under LightDM the user session got :0, so this was hardcoded to it.
// GDM gives :0 to its own greeter and starts the user's Xorg with -displayfd, letting the kernel pick —
// in practice :1. Hardcoding :0 made the mirror fail with "Can't open display" on every GNOME machine.
//
// Find it by socket ownership instead: /tmp/.X11-unix/X<n> is owned by whoever runs that X server, so
// the one owned by us is the owner's session and any other is the greeter's. Fall back to the lowest
// socket (covers a root-run Xorg, as LightDM did) and finally to :0 so behaviour never gets worse than
// the constant it replaced.
function resolveDisplayNum(): number {
const uid = typeof process.getuid === 'function' ? process.getuid() : null;
let sockets: number[] = [];
try {
sockets = readdirSync('/tmp/.X11-unix')
.filter((n) => /^X\d+$/.test(n))
.map((n) => Number(n.slice(1)))
.sort((a, b) => a - b);
} catch {
return 0;
}
if (uid != null) {
const own = sockets.find((n) => {
try {
return statSync(`/tmp/.X11-unix/X${n}`).uid === uid;
} catch {
return false;
}
});
if (own !== undefined) return own;
}
return sockets[0] ?? 0;
}
// x11vnc reads the display's cookie from the logged-in user's X authority, so no root is needed. Where
// that cookie lives depends on the display manager: GDM (Ubuntu GNOME on Xorg) keeps it in the
// per-session dir /run/user/<uid>/gdm/Xauthority; LightDM (or a manual startx) uses the classic
// ~/.Xauthority. Try the GDM path first, then fall back. While the greeter owns the seat the user's
// cookie doesn't exist yet and mirroring fails until someone is logged in.
function resolveXauthority(): string | null {
const uid = typeof process.getuid === 'function' ? process.getuid() : null;
const candidates = [
uid != null ? `/run/user/${uid}/gdm/Xauthority` : null,
join(process.env.HOME ?? '', '.Xauthority'),
].filter((p): p is string => Boolean(p));
return candidates.find((p) => existsSync(p)) ?? null;
}
type MirrorSession = {
email: string;
username: string;
display: number;
port: number;
pid: number;
sessionPid: number;
};
let desktop: VirtualDesktop | null = null;
let mirror: MirrorSession | null = null;
function isProcessAlive(pid: number): boolean {
try {
@@ -50,23 +76,14 @@ function isProcessAlive(pid: number): boolean {
}
}
// :0 and :1 belong to GDM and the TV session. Skip anything with a socket or a lock file — a stale lock
// with no server still makes an X server refuse to start on that number.
function findFreeDisplay(): number {
for (let n = FIRST_DISPLAY; n < MAX_DISPLAY; n++) {
if (!existsSync(`/tmp/.X11-unix/X${n}`) && !existsSync(`/tmp/.X${n}-lock`)) return n;
}
throw new Error(`No free X display between :${FIRST_DISPLAY} and :${MAX_DISPLAY}`);
}
// Provision the pair of files a VNC session needs: `password` in plaintext, which the API hands to
// the browser, and `passwd` in rfbauth format, which the VNC server authenticates against. Idempotent —
// the browser, and `passwd` in rfbauth format, which x11vnc authenticates against. Idempotent —
// returns the existing password when both are already present.
//
// Written with `x11vnc -storepasswd`. That looks odd now that Xvnc serves the desktop, but the rfbauth
// format is the same for both and this path is known-good: it replaced a call to tigervnc's `vncpasswd`,
// which is not shipped in tigervnc-common, so the file silently never appeared and only a zero exit code
// triggered the write, swallowing the failure.
// x11vnc writes the rfbauth file itself via -storepasswd. This used to shell out to tigervnc's
// `vncpasswd -f`, which is not installed by default (and is not in tigervnc-common, despite what
// setup-desktop.sh claimed), so the rfbauth file silently never appeared — the failure was swallowed
// because only a zero exit code triggered the write.
export async function ensureVncPassword(homeDir: string): Promise<{ password: string; passwdFile: string }> {
const vncDir = join(homeDir, '.vnc');
const passwdFile = join(vncDir, 'passwd');
@@ -103,23 +120,39 @@ export async function ensureVncPassword(homeDir: string): Promise<{ password: st
return { password, passwdFile };
}
// The X cookie for the virtual display. Its own file, not ~/.Xauthority — that one belongs to the TV
// session and writing to it risks disturbing a session we do not own.
function ensureXauthority(vncDir: string, display: number): string {
const authFile = join(vncDir, 'Xauthority');
const cookie = randomBytes(16).toString('hex');
type Geometry = { w: number; h: number; x: number; y: number };
type DisplayGeometry = { framebufferWidth: number | null; primary: Geometry | null; connected: number };
// X composes every attached output into ONE framebuffer, so with two monitors side by side the
// framebuffer is their combined width and mirroring it whole sends both screens squashed together.
// Read the layout so the caller can clip to a single output.
//
// Returns nulls when xrandr is missing or X is not up; callers treat that as "don't scale, don't clip",
// because serving too many pixels beats serving nothing.
function readDisplayGeometry(xauthority: string, display: string): DisplayGeometry {
const proc = Bun.spawnSync({
cmd: ['xauth', '-f', authFile, 'add', `:${display}`, 'MIT-MAGIC-COOKIE-1', cookie],
stdout: 'ignore',
stderr: 'pipe',
cmd: ['xrandr', '--current'],
env: { ...process.env, DISPLAY: display, XAUTHORITY: xauthority },
stdout: 'pipe',
stderr: 'ignore',
});
if (proc.exitCode !== 0) {
throw new Error(`xauth failed for :${display}: ${proc.stderr.toString().trim()}`);
}
if (proc.exitCode !== 0) return { framebufferWidth: null, primary: null, connected: 0 };
Bun.spawnSync({ cmd: ['chmod', '600', authFile], stdout: 'ignore', stderr: 'ignore' });
return authFile;
const out = proc.stdout.toString();
const fb = out.match(/current\s+(\d+)\s*x\s*(\d+)/);
// Count ENABLED outputs, not merely connected ones: a screen that is plugged in but switched off
// still reports "connected" and contributes nothing to the framebuffer. Only an enabled output
// carries a WxH+X+Y geometry, so requiring that is what distinguishes the two.
const connected = (out.match(/^\S+ connected(?: primary)? \d+x\d+\+\d+\+\d+/gm) ?? []).length;
// "HDMI-A-0 connected primary 3840x2160+0+0 (normal left ..." — the geometry only appears on an
// output that is actually enabled, so a connected-but-off output correctly yields no match.
const p = out.match(/^\S+ connected primary (\d+)x(\d+)\+(\d+)\+(\d+)/m);
const primary = p
? { w: Number(p[1]), h: Number(p[2]), x: Number(p[3]), y: Number(p[4]) }
: null;
return { framebufferWidth: fb ? Number(fb[1]) : null, primary, connected };
}
async function isPortOpen(port: number): Promise<boolean> {
@@ -136,32 +169,10 @@ async function isPortOpen(port: number): Promise<boolean> {
}
}
// A listener on the port is NOT proof our server started. The running desktop is tracked in module
// state, so a sidecar restart forgets it while the server keeps running orphaned — then the next start
// spawns a second one, which cannot bind the port, while this check sees the ORPHAN listening and
// reports success. The platform then believes it started a desktop the browser is not looking at.
// So: reclaim the port before spawning, and treat our own process dying as failure.
async function reclaimPort(port: number): Promise<void> {
if (!(await isPortOpen(port))) return;
console.log(`[vnc] port ${port} already held — killing the orphan before starting`);
Bun.spawnSync({ cmd: ['fuser', '-k', '-TERM', `${port}/tcp`], stdout: 'ignore', stderr: 'ignore' });
for (let i = 0; i < 20; i++) {
await Bun.sleep(100);
if (!(await isPortOpen(port))) return;
}
Bun.spawnSync({ cmd: ['fuser', '-k', '-KILL', `${port}/tcp`], stdout: 'ignore', stderr: 'ignore' });
await Bun.sleep(300);
}
// Xvnc stays in the foreground, so readiness is the listening port rather than exit code — but only
// once we know the process we spawned is the one still alive to own it.
async function waitForPort(port: number, proc: { exitCode: number | null }): Promise<boolean> {
// x11vnc stays in the foreground, so readiness is the listening port rather than exit code.
async function waitForPort(port: number): Promise<boolean> {
const deadline = Date.now() + READY_TIMEOUT_MS;
while (Date.now() < deadline) {
if (proc.exitCode !== null) return false;
if (await isPortOpen(port)) return true;
await Bun.sleep(100);
}
@@ -169,51 +180,76 @@ async function waitForPort(port: number, proc: { exitCode: number | null }): Pro
}
export async function startSession(params: VncStartParams): Promise<{ port: number; display: number }> {
if (desktop && isProcessAlive(desktop.pid)) {
return { port: desktop.port, display: desktop.display };
if (mirror && isProcessAlive(mirror.pid)) {
return { port: mirror.port, display: mirror.display };
}
desktop = null;
mirror = null;
const homeDir = getOwnerHomeDir(params.email);
const { passwdFile } = await ensureVncPassword(homeDir);
// Before choosing a display: an orphan from a previous sidecar life would still hold the port, and
// findFreeDisplay would then hand us a number whose server can never bind it.
await reclaimPort(VNC_PORT);
const displayNum = resolveDisplayNum();
const display = `:${displayNum}`;
const display = findFreeDisplay();
const authFile = ensureXauthority(join(homeDir, '.vnc'), display);
const xauthority = resolveXauthority();
if (!xauthority) {
throw new Error(`No X authority found (GDM or ~/.Xauthority) — nobody is logged in on ${display}`);
}
// With more than one output attached the framebuffer spans them all, and mirroring it whole shows
// every monitor side by side, squashed. Clip to the primary so the remote view is one screen at the
// right proportions. Only when a primary is actually marked and there is more than one output —
// otherwise the framebuffer already IS the single screen and clipping would just add a failure mode.
const geo = readDisplayGeometry(xauthority, display);
const clipTo = geo.connected > 1 && geo.primary ? geo.primary : null;
const clip = clipTo ? ['-clip', `${clipTo.w}x${clipTo.h}+${clipTo.x}+${clipTo.y}`] : [];
if (clipTo) {
console.log(`[vnc] ${geo.connected} outputs attached — clipping to the primary ${clip[1]}`);
}
// Halving a 4K screen keeps the stream sane over the tailnet, but the same 0.5 applied to a small
// screen is just lost detail — and with no monitor plugged in, X falls back to something tiny
// (800x480 here), which halves to an unreadable 400x240. Scale only when there is genuinely too much
// to send, and judge that on what is actually being served: the clip if there is one, else the
// whole framebuffer.
const width = clipTo ? clipTo.w : geo.framebufferWidth;
const scale = width !== null && width > SCALE_ABOVE_WIDTH ? ['-scale', '0.5'] : [];
if (width !== null) {
console.log(`[vnc] serving ${width}px wide — ${scale.length ? 'scaling to 50%' : '1:1'}`);
}
const proc = Bun.spawn({
cmd: [
'Xvnc',
`:${display}`,
'-geometry',
GEOMETRY,
'-depth',
String(DEPTH),
'x11vnc',
'-display',
display,
'-auth',
xauthority,
'-rfbport',
String(VNC_PORT),
String(MIRROR_PORT),
'-rfbauth',
passwdFile,
'-SecurityTypes',
'VncAuth',
// Loopback only: the browser reaches this through the platform's websocket bridge, never directly.
'-localhost',
// Without this a second browser tab silently disconnects the first.
'-AlwaysShared',
'-auth',
authFile,
'-desktop',
'officer',
'-dpi',
'96',
'-forever',
'-shared',
...clip,
...scale,
// No -noxdamage. It was set when this was written, with no recorded reason, and it is expensive:
// without the DAMAGE extension x11vnc cannot be told which rectangles changed, so it polls the
// WHOLE framebuffer continuously to find out. That makes cost scale with screen area rather than
// with what actually moved — on a 4K mirror it is the single biggest source of latency, and it is
// why a smaller machine felt instant by comparison. This X server reports DAMAGE, MIT-SHM and
// XFIXES, so the optimisation is genuinely available.
//
// If stale patches ever appear on screen (some drivers under-report damage), putting -noxdamage
// back is the fix — at the cost of the polling above.
'-quiet',
],
stdout: 'ignore',
stderr: 'pipe',
});
// Drain stderr so the pipe cannot fill and stall the server; keep the tail for errors.
// Drain stderr so the pipe cannot fill and stall x11vnc; keep the tail for errors
let stderrTail = '';
(async () => {
const reader = (proc.stderr as ReadableStream<Uint8Array>).getReader();
@@ -229,71 +265,54 @@ export async function startSession(params: VncStartParams): Promise<{ port: numb
}
})();
if (!(await waitForPort(VNC_PORT, proc))) {
if (!(await waitForPort(MIRROR_PORT))) {
try {
proc.kill();
} catch {
// already dead
}
throw new Error(`Xvnc failed to listen on ${VNC_PORT}: ${stderrTail.trim() || 'exited or timed out'}`);
throw new Error(`x11vnc failed to listen on ${MIRROR_PORT}: ${stderrTail.trim() || 'timed out'}`);
}
// The desktop itself, once the server is accepting. dbus-run-session gives this session its own bus:
// inheriting the GNOME session's bus is how two desktops start fighting over the same daemons.
const env: Record<string, string> = { ...(process.env as Record<string, string>) };
delete env.SESSION_MANAGER;
delete env.DBUS_SESSION_BUS_ADDRESS;
const session = Bun.spawn({
cmd: ['dbus-run-session', '--', 'startxfce4'],
env: { ...env, DISPLAY: `:${display}`, XAUTHORITY: authFile, XDG_SESSION_TYPE: 'x11' },
stdout: 'ignore',
stderr: 'ignore',
});
desktop = {
mirror = {
email: params.email,
username: params.username ?? '',
display,
port: VNC_PORT,
display: displayNum,
port: MIRROR_PORT,
pid: proc.pid,
sessionPid: session.pid,
};
console.log(`[vnc] virtual desktop on :${display} (${GEOMETRY}) port ${VNC_PORT} — Xvnc ${proc.pid}, xfce ${session.pid}`);
console.log(`[vnc] mirroring ${display} on port ${MIRROR_PORT} (pid ${proc.pid})`);
return { port: VNC_PORT, display };
return { port: MIRROR_PORT, display: displayNum };
}
export function stopSession(_email: string): void {
if (!desktop) return;
if (!mirror) return;
// The session first: killing Xvnc out from under XFCE leaves its children reparented and running.
for (const pid of [desktop.sessionPid, desktop.pid]) {
try {
if (pid > 0) process.kill(pid);
} catch {
// already dead
}
try {
process.kill(mirror.pid);
} catch {
// already dead
}
console.log(`[vnc] stopped the virtual desktop on :${desktop.display}`);
desktop = null;
console.log(`[vnc] stopped mirror of :${mirror.display} (pid ${mirror.pid})`);
mirror = null;
}
export function getSession(_email: string): VncSessionInfo | null {
if (!desktop) return null;
if (!mirror) return null;
const alive = desktop.pid > 0 && isProcessAlive(desktop.pid);
const alive = mirror.pid > 0 && isProcessAlive(mirror.pid);
if (!alive) {
desktop = null;
mirror = null;
return null;
}
return {
email: desktop.email,
display: desktop.display,
port: desktop.port,
pid: desktop.pid,
email: mirror.email,
display: mirror.display,
port: mirror.port,
pid: mirror.pid,
alive,
};
}
src/workspaces/officerdev/src/apps/Desktop/DesktopView.tsx
+6 -11
View File
@@ -140,17 +140,12 @@ export const DesktopView = ({ className, style }: DesktopViewProps) => {
credentials: { password },
});
// BOTH. They are not alternatives — scaling is the fallback that covers the gap between resize
// events, and more importantly it is what keeps the coordinate maths self-consistent.
//
// noVNC maps a click as (clientX - canvasRect.left) / display._scale. Only scaleViewport's
// autoscale() sets the canvas's displayed size and _scale TOGETHER, so the two cannot disagree.
// Turning it off pins _scale at 1 while the canvas keeps being displayed at some other size, and
// nothing reconciles them: measured here, pointing a quarter of the way across the desktop landed
// at 51%, and pointing at the middle saturated at the right-hand edge. A clean factor of two.
// Clicks near the right and bottom edges still appeared to work, because doubling an already-large
// coordinate clamps back onto the edge it was aimed at.
rfb.resizeSession = true;
// The mirror cannot resize — it reflects the physical screen — so resizeSession is off.
// scaleViewport STAYS ON, and it is load-bearing: noVNC maps a click as
// (clientX - canvasRect.left) / display._scale, and autoscale() is the only thing that sets the
// canvas's displayed size and _scale in the same call. Turn it off and _scale is pinned at 1 while
// the canvas is displayed at some other size, which doubled every coordinate on 2026-08-01.
rfb.resizeSession = false;
rfb.scaleViewport = true;
rfb.focusOnClick = true;
// noVNC hides the browser's own cursor over the canvas and draws the REMOTE cursor in its place.