qwen-local: the first provider that runs on our own GPU
sd-server (stable-diffusion.cpp, Vulkan) serving Qwen-Image 2.1 on this machine. It already speaks OpenAI's image API, which is the vocabulary `ImageRequest` is written in, so there is no translation layer — only the parts that are true of a local GPU and not of a hosted one. ── keyless is a category, not an exception ── `credentials.ts` decided a provider was usable by asking whether a key existed, so a service that takes none read as unconfigured forever. The fix is an adapter declaring `keyless`, not a dummy secret: a dummy would record a credential that does not exist and make the secret store lie about what is configured, and that store is the one place meant to be the truth about them. A keyless adapter never reaches `keyFor` at all. That function's own comment says why it uses `getSecret` rather than `getKey` — so a missing FOREIGN credential is never minted — and a provider with no credential by design has no business in that path. whisper, OCR and TTS already run on this machine, so this will not be the last. ── measured, not assumed ── The integration doc said "tens of seconds, not two". The server's own log says 512x512 completes in ~185s and 1024x1024 samples at ~41.4s/step over 20 steps, about 14 minutes. And it serialises strictly: a second caller waits for the first to finish entirely, so a queued 512 can wait 17 minutes before a byte comes back. So: 512 is the default and the substitute for `auto`, 1024 is offered with the cost in its label, and `timeout: false` is a requirement rather than caution — every default anyone would reach for is far under 17 minutes. The caller's `signal` still aborts, because a user cancelling is not a client giving up. ── the model id moved, because only it could ── `GET /v1/models` returns `sd-cpp-local`, hardcoded in sd-server with no override. The catalogue now agrees with the server and carries "Qwen-Image 2.1 (local)" in the label, so requests name a model the server knows while the UI still says something meaningful. ── not shipped: edit() ── Same GPU, same queue. It doubles the ways generation can be blocked while the generate path is unproven in anger, and `ImageAdapter` makes absence the way to say so — the route answers 400 instead of the queue finding out late. Verified: /v1/models answers, provider statuses report qwen-local as configured/keyless/usable, and the shipped entry survives the merge with the stored catalogue.json. NOT verified end to end — a generation is occupying the GPU and I left it alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Officer-Session: de3ed4e6-f88a-429c-94e6-a760336ba83b
This commit is contained in:
@@ -83,5 +83,49 @@ export const SHIPPED_CATALOGUE: Catalogue = {
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
/**
|
||||
* The first LOCAL provider: sd-server on this machine's GPU, not a vendor.
|
||||
*
|
||||
* No `envVar`, because there is no key and never will be — see `keyless` on the adapter. It is
|
||||
* `configured` by existing.
|
||||
*
|
||||
* The id is `sd-cpp-local` and not `qwen-image-2.1` because that is what `GET /v1/models`
|
||||
* returns, hardcoded in sd-server with no override. The catalogue is the only side that can
|
||||
* move, so it moves: the id agrees with the server, and the LABEL carries what the thing
|
||||
* actually is.
|
||||
*/
|
||||
name: 'qwen-local',
|
||||
label: 'Qwen-Image (local)',
|
||||
defaultModel: 'sd-cpp-local',
|
||||
models: [
|
||||
{
|
||||
id: 'sd-cpp-local',
|
||||
/**
|
||||
* The timings are in the label because there is nowhere else for them, and a user choosing
|
||||
* 1024 should know before they wait rather than after. Measured from the server's own log on
|
||||
* this machine: 512 completes in ~185s; 1024 samples at ~41.4s/step over 20 steps.
|
||||
*
|
||||
* A `note` field on CatalogueModel would be the cleaner home for this. Not added on a guess —
|
||||
* it needs the UI to render it, and a field nothing displays is worse than a long label.
|
||||
*/
|
||||
label: 'Qwen-Image 2.1 (local) — 512 takes ~3 min, 1024 ~14 min',
|
||||
kind: 'image',
|
||||
/**
|
||||
* 512 first, and the adapter substitutes it for `auto` or for nothing. 1024 is offered
|
||||
* because it is genuinely better and sometimes worth the wait — but a default nobody chose
|
||||
* should not spend a quarter of an hour of the only GPU there is.
|
||||
*/
|
||||
sizes: ['512x512', '768x768', '1024x1024'],
|
||||
transparency: false,
|
||||
/** No `edit()` in this version — same GPU, same queue, and the generate path is unproven. */
|
||||
edit: false,
|
||||
/** Zero is the true answer, not a missing one: it runs on hardware we already own. */
|
||||
pricing: { unit: 'image', perImage: 0 },
|
||||
status: 'current',
|
||||
source: 'shipped',
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
@@ -71,8 +71,14 @@ export type CatalogueProvider = {
|
||||
/** Matches an adapter's `name` when code for it exists. A provider with no adapter is a listing, not an offer. */
|
||||
name: string;
|
||||
label: string;
|
||||
/** The bootstrap credential's variable, for a connection with no stored key. */
|
||||
envVar: string;
|
||||
/**
|
||||
* The bootstrap credential's variable, for a connection with no stored key.
|
||||
*
|
||||
* Optional since the first LOCAL provider: a service of ours, on this machine, authenticates with
|
||||
* nothing, and naming an environment variable for a key that does not exist would describe a way to
|
||||
* configure it that does not work. See `keyless` on `ImageAdapter`.
|
||||
*/
|
||||
envVar?: string;
|
||||
defaultModel?: string;
|
||||
models: CatalogueModel[];
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user