Reported as "no way to send a left click". `xinput --query-state` on the display showed `Shift_L` down on the XTEST device with nothing holding it, so every click reached the game as shift-click, which Minesweeper reads as a right click. The board flagged instead of opening and nothing about the panel looked wrong. Mine, from the keyboard work two commits ago, and two separate mistakes. ── modifiers should never have been sent ── Press shift, release it while the canvas no longer has focus, and the keyup never arrives — the key is then down on the server forever. They are also POINTLESS: `ev.key` is the character the layout already produced, so a shifted key arrives as `A` and x11vnc presses shift itself to type it. Sending our own on top was redundant on the good path and load-bearing on the bad one. `keysymFor` now returns null for them, which also hands them back to the browser — shift-tab should still move focus out of a game of Minesweeper. ── and nothing released what was held ── Any key can strand the same way, so the panel now tracks what it has pressed and releases it on blur and on unmount. Cheap, and the failure it prevents is invisible from the panel: the server has no way to know a key was released somewhere it could not see. The live display is unstuck too — `xdotool keyup` on the modifiers, since the fix stops it happening again but does not undo the one already stranded. The general lesson, which is the same one the difficulty command taught: input synthesised into another program's session is state you now own. Press without a guaranteed release is a leak, and it does not look like one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Officer-Session: de3ed4e6-f88a-429c-94e6-a760336ba83b
368 lines
17 KiB
TypeScript
368 lines
17 KiB
TypeScript
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
|
|
import { Bomb, Loader2, Play, RefreshCw } from 'lucide-react';
|
|
import { useClient } from 'hooks/useClient';
|
|
import type { RfbError } from './rfb';
|
|
import { RfbClient, keysymFor } from './rfb';
|
|
|
|
type SessionStatus = {
|
|
installed: boolean;
|
|
running: boolean;
|
|
};
|
|
|
|
// The whole framebuffer is the picture, and the sidecar keeps the game centred in it. So there is no
|
|
// crop here, no board geometry to fetch, and nothing that has to be re-read when the difficulty changes
|
|
// — the canvas is whatever size the server said at handshake and stays that size for the session.
|
|
//
|
|
// It used to crop to the window's rectangle at the origin, which needed the geometry polled every four
|
|
// seconds and was wrong twice over: the panel resized under the player mid-game, and wine puts the
|
|
// window at 3,29 rather than 0,0, so a crop from the origin showed a black strip on top and cut a row
|
|
// of cells off the bottom. That was the "expert gets cut off at the bottom" report.
|
|
//
|
|
// Only a placeholder until `onReady`, which sizes the canvas from the real framebuffer. Drift with the
|
|
// sidecar's stage costs one frame of the wrong aspect ratio and nothing else.
|
|
const PLACEHOLDER = { width: 512, height: 384 };
|
|
|
|
const BUTTON_MASK: Record<number, number> = { 0: 1, 1: 2, 2: 4 };
|
|
|
|
/** Breathing room around the stage, in device pixels. Small — it is a frame, not a margin. */
|
|
const MARGIN_DEVICE_PX = 24;
|
|
|
|
/**
|
|
* The ceiling on the size at 100%, whatever room the panel has.
|
|
*
|
|
* A taste decision, not a limit of the method: fitting the panel is not the same as looking right, and
|
|
* past 2x this stops reading as a window on a desktop and starts reading as a wall.
|
|
*
|
|
* It caps the BASELINE, not the result. The panel's zoom multiplies what comes out of here, so zooming
|
|
* in still goes past 2x — the first version of this cap did not make that distinction and flattened the
|
|
* zoom control into doing nothing, because the fit it capped is the same number at every zoom level.
|
|
*/
|
|
const MAX_SCALE = 2;
|
|
|
|
// Plugin panels have no localisation mechanism yet — `officerdev`'s `useTranslation`/`panelCatalogs`
|
|
// (merged by `I18nBridge`) is platform-internal and not exported from the barrel, and
|
|
// `scripts/i18n/scan-hardcoded.ts` explicitly does not scan `plugins/`. So these stay plain English
|
|
// sentences rather than catalog keys, same as every other plugin's user-facing strings.
|
|
const describeRfbError = (error: RfbError): string => {
|
|
switch (error.code) {
|
|
case 'connectionFailed':
|
|
return 'The connection to the Minesweeper session failed.';
|
|
case 'refused':
|
|
return 'The VNC server refused the connection.';
|
|
case 'authUnsupported':
|
|
return 'The VNC server wants authentication, which this client does not implement.';
|
|
case 'handshakeRejected':
|
|
return 'The VNC server rejected the handshake.';
|
|
case 'unknownMessage':
|
|
return `The VNC server sent an unknown message type (${error.type}).`;
|
|
case 'unknownEncoding':
|
|
return `The VNC server used encoding ${error.encoding}, which this client did not ask for.`;
|
|
}
|
|
};
|
|
|
|
export const MinesweeperPanel = () => {
|
|
// useClient() rebuilds its verbs on EVERY render, so putting one in a dependency list gives that
|
|
// list a new identity every render. Held in a ref instead — the same fix the Jellyfin player needed
|
|
// after its playback reports silently stopped for days. Getting this wrong here meant `refresh` was
|
|
// unstable, its mount effect re-ran every render, and each run set state and caused another render:
|
|
// an endless fetch loop that starved the socket and left the panel saying "connecting…" forever.
|
|
const client = useClient();
|
|
const clientRef = useRef(client);
|
|
clientRef.current = client;
|
|
const canvasRef = useRef<HTMLCanvasElement | null>(null);
|
|
const rfbRef = useRef<RfbClient | null>(null);
|
|
const buttonsRef = useRef(0);
|
|
// Every keysym we have sent a press for and not yet a release. A key held when focus leaves never gets
|
|
// its keyup, and a key stuck down on an X server stays stuck until something releases it.
|
|
const heldKeysRef = useRef(new Set<number>());
|
|
|
|
const boxRef = useRef<HTMLDivElement | null>(null);
|
|
|
|
const [frame, setFrame] = useState<{ width: number; height: number } | null>(null);
|
|
const [box, setBox] = useState<{ width: number; height: number; dpr: number; zoom: number } | null>(null);
|
|
const [status, setStatus] = useState<SessionStatus | null>(null);
|
|
const [error, setError] = useState<string | null>(null);
|
|
const [connected, setConnected] = useState(false);
|
|
const [busy, setBusy] = useState(false);
|
|
|
|
// Measured rather than assumed, because the scale below has to be exact and a panel is resizable.
|
|
// Re-attached when the session starts: until then this branch of the tree is not rendered and the ref
|
|
// is null, so a mount-only observer would watch nothing for the rest of the session.
|
|
useEffect(() => {
|
|
const node = boxRef.current;
|
|
if (!node) return;
|
|
const observer = new ResizeObserver(() => {
|
|
// `offsetWidth` is in the element's OWN pixels; `getBoundingClientRect` is in visual ones, so their
|
|
// ratio is the panel's CSS `zoom`. That matters because the panel header has a zoom control and
|
|
// `devicePixelRatio` knows nothing about it — at 75% the scale below would be off by a quarter and
|
|
// put the cell grid back on fractional boundaries, which is the exact thing it exists to prevent.
|
|
const rect = node.getBoundingClientRect();
|
|
const zoom = node.offsetWidth > 0 ? rect.width / node.offsetWidth : 1;
|
|
setBox({
|
|
width: node.offsetWidth,
|
|
height: node.offsetHeight,
|
|
dpr: (window.devicePixelRatio || 1) * zoom,
|
|
zoom,
|
|
});
|
|
});
|
|
observer.observe(node);
|
|
return () => observer.disconnect();
|
|
}, [status?.running]);
|
|
|
|
/**
|
|
* How big to draw the stage. Computed in DEVICE pixels, so a 2x display gets twice the detail rather
|
|
* than the same picture twice the size — and allowed to scale UP past the framebuffer's own size,
|
|
* without which a wide panel leaves the canvas sitting at a fraction of the pixels available to it.
|
|
*
|
|
* ── Why the scale is snapped, and to what ──
|
|
*
|
|
* This is a 16px-per-cell bitmap from 1995. At an arbitrary scale a cell is 26.4 screen pixels, so the
|
|
* grid lines themselves land unevenly and the digits wobble: sharp everywhere and wrong overall, which
|
|
* is what "not quite right" looks like on pixel art. `image-rendering: pixelated` does not save you —
|
|
* it removes the blur, not the unevenness. A fractional devicePixelRatio produces this at what looks
|
|
* like 1:1, which is the case that is easiest to miss.
|
|
*
|
|
* QUARTER steps, and that is not a compromise on the above. The unit that has to stay regular is the
|
|
* CELL, not the pixel: the board is a 16px grid, so every quarter step gives a whole cell — 20px at
|
|
* 1.25x, 36px at 2.25x — and only sub-cell pixels alternate. Below 1x there is nothing to floor to, so
|
|
* it takes a unit fraction and the rows that survive are at least evenly spaced.
|
|
*
|
|
* Quarters all the way down rather than whole numbers under 2x, because the zoom control has to have
|
|
* somewhere to go: with whole numbers below 2x, zooming out from a 2x baseline fell straight to 1x and
|
|
* halved the board in one press.
|
|
*
|
|
* ── zoom multiplies, it does not compete ──
|
|
*
|
|
* `fit` is the same number at every zoom level — the panel's CSS `zoom` shrinks `offsetWidth` by
|
|
* exactly what it adds to the pixel ratio — so zoom has to be applied deliberately or it does nothing
|
|
* at all. The baseline is what fits (capped), and zoom moves the picture up and down from there.
|
|
*/
|
|
const stage = useMemo(() => {
|
|
if (!frame || !box || box.width <= 0 || box.height <= 0) return null;
|
|
// Air around the stage. Filling the panel edge to edge is most of what "too large" looks like, and it
|
|
// leaves the game with no frame at all.
|
|
const usableW = Math.max(1, box.width * box.dpr - MARGIN_DEVICE_PX * 2);
|
|
const usableH = Math.max(1, box.height * box.dpr - MARGIN_DEVICE_PX * 2);
|
|
const fit = Math.min(usableW / frame.width, usableH / frame.height);
|
|
const wanted = Math.min(fit, MAX_SCALE) * box.zoom;
|
|
const scale = wanted < 1 ? 1 / Math.ceil(1 / wanted) : Math.floor(wanted * 4) / 4;
|
|
return { width: (frame.width * scale) / box.dpr, height: (frame.height * scale) / box.dpr };
|
|
}, [frame, box]);
|
|
|
|
const refresh = useCallback(async () => {
|
|
try {
|
|
setStatus(await clientRef.current.get<SessionStatus>('/minesweeper/status'));
|
|
} catch {
|
|
setError('Could not reach the server.');
|
|
}
|
|
}, []);
|
|
|
|
useEffect(() => {
|
|
void refresh();
|
|
}, [refresh]);
|
|
|
|
const start = async () => {
|
|
setBusy(true);
|
|
setError(null);
|
|
try {
|
|
setStatus(await clientRef.current.post<SessionStatus>('/minesweeper/start', {}));
|
|
} catch {
|
|
setError('The session would not start. Is wine-minesweeper installed on the server?');
|
|
} finally {
|
|
setBusy(false);
|
|
}
|
|
};
|
|
|
|
useEffect(() => {
|
|
if (!status?.running) return;
|
|
|
|
const scheme = window.location.protocol === 'https:' ? 'wss' : 'ws';
|
|
// 'BEARER_TOKEN' is the key the rest of the app uses (Terminal.tsx, useAudioRecording). Reading
|
|
// 'token' here sent an empty string, and upgradeWs answered 401 before any RFB byte was exchanged.
|
|
const token = localStorage.getItem('BEARER_TOKEN') ?? sessionStorage.getItem('BEARER_TOKEN') ?? '';
|
|
// Plugin relay, not a platform websocket provider: `/api/plugin-ws/minesweeper` rather than the old
|
|
// `/api/minesweeper/ws`. Same byte relay onto x11vnc, mounted by the plugin instead of the shell.
|
|
const url = `${scheme}://${window.location.host}/api/plugin-ws/minesweeper?token=${encodeURIComponent(token)}`;
|
|
|
|
const rfb = new RfbClient(url, {
|
|
// No offscreen buffer: with the crop gone the visible canvas IS the framebuffer, so a second
|
|
// canvas would only add a full blit per frame to arrive at the same pixels.
|
|
onReady: (width, height) => {
|
|
const canvas = canvasRef.current;
|
|
if (canvas) {
|
|
canvas.width = width;
|
|
canvas.height = height;
|
|
}
|
|
setFrame({ width, height });
|
|
setConnected(true);
|
|
setError(null);
|
|
},
|
|
onFrame: (image, x, y) => {
|
|
canvasRef.current?.getContext('2d')?.putImageData(image, x, y);
|
|
},
|
|
onCopyRect: (from, to) => {
|
|
// Same canvas as source and destination, which is what CopyRect is: the 2D context takes a
|
|
// snapshot of the bitmap first, so an overlapping copy is well defined rather than smeared.
|
|
const canvas = canvasRef.current;
|
|
const context = canvas?.getContext('2d');
|
|
if (!canvas || !context) return;
|
|
context.drawImage(canvas, from.x, from.y, to.w, to.h, to.x, to.y, to.w, to.h);
|
|
},
|
|
onResize: (width, height) => {
|
|
const canvas = canvasRef.current;
|
|
if (canvas) {
|
|
canvas.width = width;
|
|
canvas.height = height;
|
|
}
|
|
setFrame({ width, height });
|
|
rfb.requestUpdate(false);
|
|
},
|
|
onError: (rfbError) => setError(describeRfbError(rfbError)),
|
|
onClose: () => setConnected(false),
|
|
});
|
|
|
|
rfbRef.current = rfb;
|
|
return () => {
|
|
releaseHeldKeys();
|
|
rfb.close();
|
|
rfbRef.current = null;
|
|
setConnected(false);
|
|
};
|
|
}, [status?.running]);
|
|
|
|
const toFramebuffer = (ev: React.MouseEvent<HTMLCanvasElement>) => {
|
|
const rect = ev.currentTarget.getBoundingClientRect();
|
|
const scaleX = ev.currentTarget.width / rect.width;
|
|
const scaleY = ev.currentTarget.height / rect.height;
|
|
return { x: (ev.clientX - rect.left) * scaleX, y: (ev.clientY - rect.top) * scaleY };
|
|
};
|
|
|
|
/**
|
|
* Keys, forwarded only while the canvas has focus.
|
|
*
|
|
* The canvas is focusable and takes focus on mousedown, which is the gesture you use to play anyway.
|
|
* Scoping it to focus is what keeps the panel from eating keys meant for the rest of the app — this is
|
|
* one panel among several on a dashboard, and a window-level listener would be a global keyboard trap.
|
|
*
|
|
* A key with a modifier is left alone: cmd-W and ctrl-R belong to the browser and the game wants none
|
|
* of them. So is anything `keysymFor` has no mapping for, which keeps Tab moving focus rather than
|
|
* disappearing into a game of Minesweeper. `preventDefault` happens only for keys actually sent.
|
|
*/
|
|
const onKey = (ev: React.KeyboardEvent<HTMLCanvasElement>, down: boolean) => {
|
|
const rfb = rfbRef.current;
|
|
if (!rfb || ev.metaKey || ev.ctrlKey || ev.altKey) return;
|
|
const keysym = keysymFor(ev.key);
|
|
if (keysym === null) return;
|
|
ev.preventDefault();
|
|
if (down) heldKeysRef.current.add(keysym);
|
|
else heldKeysRef.current.delete(keysym);
|
|
rfb.key(keysym, down);
|
|
};
|
|
|
|
/**
|
|
* Release everything on the way out.
|
|
*
|
|
* A key held when the canvas loses focus never sends its keyup, and the server has no way to know: it
|
|
* stays down until something says otherwise. That is not a cosmetic leak — a stranded modifier turned
|
|
* every left click into a shift-click, which Minesweeper reads as a right click, and the board flagged
|
|
* instead of opening.
|
|
*
|
|
* Modifiers are no longer sent at all (see `keysymFor`), so this now covers the ordinary keys. Cheap,
|
|
* and the failure it prevents is invisible from the panel.
|
|
*/
|
|
const releaseHeldKeys = () => {
|
|
const rfb = rfbRef.current;
|
|
for (const keysym of heldKeysRef.current) rfb?.key(keysym, false);
|
|
heldKeysRef.current.clear();
|
|
};
|
|
|
|
const onPointer = (ev: React.MouseEvent<HTMLCanvasElement>, kind: 'down' | 'up' | 'move') => {
|
|
const rfb = rfbRef.current;
|
|
if (!rfb) return;
|
|
ev.preventDefault();
|
|
if (kind === 'down') ev.currentTarget.focus();
|
|
|
|
if (kind !== 'move') {
|
|
const bit = BUTTON_MASK[ev.button] ?? 0;
|
|
buttonsRef.current = kind === 'down' ? buttonsRef.current | bit : buttonsRef.current & ~bit;
|
|
}
|
|
const { x, y } = toFramebuffer(ev);
|
|
rfb.pointer(x, y, buttonsRef.current);
|
|
};
|
|
|
|
if (status && !status.installed) {
|
|
return (
|
|
<Centred>
|
|
<Bomb className="size-8 text-muted-foreground" />
|
|
<p className="text-sm text-muted-foreground">Minesweeper is not installed on this server.</p>
|
|
{/* The package name sits at the end so the sentence is one key rather than two around an
|
|
inline element — see docs/localization.md on the fragmentation that produces. */}
|
|
<p className="max-w-sm text-center text-xs text-muted-foreground">
|
|
It needs wine, a 32-bit prefix, and this package beside the platform: <code>wine-minesweeper</code>
|
|
</p>
|
|
</Centred>
|
|
);
|
|
}
|
|
|
|
if (status && !status.running) {
|
|
return (
|
|
<Centred>
|
|
<Bomb className="size-8 text-muted-foreground" />
|
|
<p className="text-sm text-muted-foreground">The Minesweeper session is not running.</p>
|
|
{error ? <p className="text-xs text-destructive">{error}</p> : null}
|
|
<button
|
|
type="button"
|
|
onClick={start}
|
|
disabled={busy}
|
|
className="flex items-center gap-2 rounded-md bg-primary px-3 py-1.5 text-sm text-primary-foreground disabled:opacity-60"
|
|
>
|
|
{busy ? <Loader2 className="size-4 animate-spin" /> : <Play className="size-4" />}
|
|
{busy ? 'Starting' : 'Start it'}
|
|
</button>
|
|
</Centred>
|
|
);
|
|
}
|
|
|
|
return (
|
|
<div className="flex size-full flex-col items-center justify-center gap-3 bg-muted/30 p-4">
|
|
{/* min-h-0 on the row, so the canvas gives way to the status line rather than overflowing it. */}
|
|
<div ref={boxRef} className="flex min-h-0 w-full flex-1 items-center justify-center overflow-hidden">
|
|
<canvas
|
|
ref={canvasRef}
|
|
width={PLACEHOLDER.width}
|
|
height={PLACEHOLDER.height}
|
|
tabIndex={0}
|
|
onMouseDown={(ev) => onPointer(ev, 'down')}
|
|
onMouseUp={(ev) => onPointer(ev, 'up')}
|
|
onMouseMove={(ev) => onPointer(ev, 'move')}
|
|
onKeyDown={(ev) => onKey(ev, true)}
|
|
onKeyUp={(ev) => onKey(ev, false)}
|
|
onBlur={releaseHeldKeys}
|
|
onContextMenu={(ev) => ev.preventDefault()}
|
|
// Nearest-neighbour: smoothing a 1995 bitmap looks like a photograph of a screen rather than
|
|
// the screen. It is half of what keeps this sharp; `stage` above is the other half.
|
|
//
|
|
// The max-* pair is only the fallback for the frame before `stage` is known — once it is, the
|
|
// explicit size governs and cannot exceed the box, so capping one axis would only squash the
|
|
// other.
|
|
className={`rounded-sm shadow-lg outline-none ring-primary/50 [image-rendering:pixelated] focus:ring-2 ${stage ? '' : 'max-h-full max-w-full'}`}
|
|
style={{ ...(stage ?? {}), cursor: connected ? 'default' : 'progress' }}
|
|
/>
|
|
</div>
|
|
<div className="flex items-center gap-3 text-xs text-muted-foreground">
|
|
<span>{connected ? 'connected' : 'connecting…'}</span>
|
|
{/* Focus decides where keys go, and a canvas gives no hint that it can hold any. */}
|
|
<span>click the board to send it keys</span>
|
|
<button type="button" onClick={() => void refresh()} className="flex items-center gap-1 hover:text-foreground">
|
|
<RefreshCw className="size-3" /> refresh
|
|
</button>
|
|
</div>
|
|
{error ? <p className="text-xs text-destructive">{error}</p> : null}
|
|
</div>
|
|
);
|
|
};
|
|
|
|
const Centred = ({ children }: { children: React.ReactNode }) => (
|
|
<div className="flex size-full flex-col items-center justify-center gap-3 p-6">{children}</div>
|
|
);
|