officerdev/platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

the marketplace publishes releases, not working trees

Browse Source 
The sync read `platform/plugins/` — the directories this machine has installed — and imported each
manifest off the working tree. So publishing meant "get officer-dev's checkout to the release, then
run the script", and nothing showed when that had not happened. Live proof at the time of writing:
av-club 2026.08.29.01, kanban 2026.08.29.02, openseo 0.1.0 and fofos-admin 2026.08.29.26 were all in
the store, and none of the four has ever been tagged.

It is also why the job could not go on a timer. A cron over it published whatever was in a working
tree, on a schedule.

Now every registered plugin is mirrored bare into ~/.cache/officer-marketplace/mirrors and read AT ITS
NEWEST TAG — manifest, README and file list all out of `git cat-file`. No working tree in the path, so
publishing something uncommitted stops being a mistake anyone can make.

docs/marketplace.md proposed reading manifests over Gitea's API, which needed a sandbox or a committed
manifest.json. Git gives the same guarantee with neither, and the trust model is unchanged: we import
our own repositories' code. Third-party submissions still need one of those two.

The list of plugins now comes from the `plugins` registry table, which the docs always called the
authoritative half and which was in fact a by-product of what happened to be installed. That is what
lets `finances` be published without being installed here, and it spans orgs — fofos-admin lives under
fofos/, which an org listing would have missed. Adding one is `--add <clone url>`.

A plugin with no tag is left alone and named in the report, not deleted. It has made no release, so
there is nothing true to say about it; dropping its row would be a decision about somebody's plugin
dressed up as a refresh.

Also: `release_tag` on plugin_metadata, because the tag is the fact when a manifest version disagrees
with it; the sync warns on a mismatch. And icon URLs pin to the tag — `/raw/HEAD/` served whatever was
on the default branch, via a 303, so a plugin could show one icon and ship another.

Verified: 21/25 published from a release, 4 named as untagged, all 14 tag-pinned icons answer 200.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
pastilhas
2026-09-03 13:05:50 +01:00
co-authored by Claude Opus 5
parent 8d39586de5
commit 78e89ebcec
4 changed files with 385 additions and 164 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/marketplace-sync.ts
+58 -162
View File
@@ -1,43 +1,30 @@
#!/usr/bin/env bun
import { existsSync, readFileSync } from 'node:fs';
import { basename, join, resolve } from 'node:path';
import { upsertMetadata, upsertPlugin } from '../src/databases/marketplace_db/src/plugins/queries';
import { discoverPlugins } from '../src/servers/plugins/discover';
import { upsertPlugin } from '../src/databases/marketplace_db/src/plugins/queries';
import { syncAll, syncPlugin } from '../src/servers/marketplace/sync';
// Fill the marketplace database from the plugin repositories.
//
// ── What this reads, and why it is not the marketplace's own opinion ──
//
// Every field except "where to clone from" comes out of the plugin's `manifest.ts` and its README. The
// marketplace stores a COPY so the website can render without cloning three repositories per page load;
// the manifest stays the source of truth, and the platform re-reads it after cloning. A row here that
// disagrees with the repository is a stale cache, not a second opinion — which is why re-running this is
// always safe and never merges: it overwrites.
//
// ── Reading the manifest by importing it ──
//
// A manifest is TypeScript, not JSON, so it is imported and evaluated rather than parsed. That is code
// execution, and it is fine for OUR repositories and not fine as a general mechanism.
//
// `[blocker for third-party submissions]` A stranger's plugin cannot be imported by the machine that
// serves the catalogue. Whoever builds submissions has to either evaluate manifests in a sandbox or make
// authors publish a `manifest.json` alongside. Recorded rather than solved: nobody can submit yet.
//
// ── Local clones, for now ──
//
// It syncs from `plugins/` on this machine rather than by cloning each repo into a temp directory. That
// keeps tonight's scope to "get real data into the database" — and it means the machine running the sync
// must have the plugins checked out. See `docs/marketplace.md` → "Sync" for the version that clones.
/**
* Fill the marketplace catalogue from the plugins' releases.
*
* bun --env-file=.env scripts/marketplace-sync.ts every registered plugin
* bun --env-file=.env scripts/marketplace-sync.ts photos just one
* bun --env-file=.env scripts/marketplace-sync.ts --add <url> register a new one, then sync it
*
* All the reasoning is in `src/servers/marketplace/sync.ts` — this file is the command line around it.
* It is a thin wrapper on purpose: the same functions are what the Gitea webhook calls, so a plugin
* published by hand and one published by pushing a tag go through exactly the same code.
*
* Re-running is always safe. It overwrites and never merges.
*/
type Sync = { appName: string; developer: string; repoUrl: string };
const args = process.argv.slice(2);
/**
* A clone URL the whole internet can use.
*
* `plugins/example` and `plugins/offscale` have `git@gitea.officer.dev:plugins/<name>.git` remotes,
* because whoever set them up had a key on that host. Publishing that to a catalogue would hand every
* installer a URL that only works on the machine it was published from — the install fails at
* authentication, which reads as "the plugin is broken" rather than "the catalogue is wrong".
* Several plugin repos have `git@gitea.officer.dev:plugins/<name>.git` remotes, because whoever set them
* up had a key on that host. Registering one of those would hand every installer a URL that only works
* on the machine it was published from — the install fails at authentication, which reads as "the plugin
* is broken" rather than "the catalogue is wrong".
*/
export function toHttpsRemote(remote: string): string | null {
const trimmed = remote.trim();
@@ -60,142 +47,51 @@ export function toHttpsRemote(remote: string): string | null {
return null;
}
const git = (dir: string, args: string[]): string | null => {
const result = Bun.spawnSync(['git', '-C', dir, ...args], { stdout: 'pipe', stderr: 'ignore' });
if (result.exitCode !== 0) return null;
return new TextDecoder().decode(result.stdout).trim() || null;
};
/*
Which plugins directory to publish FROM.
There is one: `platform/plugins/`, holding what this machine has INSTALLED — updated by
`plugins:update`, which is a git pull, so it sits at whatever version was last installed.
Publishing from it means the catalogue advertises the version this machine happens to be running
rather than the version that was released. That is exactly what happened on 2026-08-27: 21 plugins
were tagged and the store went on showing 1.0.0.
A second directory — `officer-dev/plugins/`, every repo in the org cloned side by side — was created
to fix that, and never did: its entries were SYMLINKS into this one, so both paths read the same
working copy and published the same version. It was removed on 2026-08-28.
**The problem is still open, and a second checkout was never the answer.** Each directory here is a
real git repo with its own tags, so the released manifest can be read out of the tag itself —
`git show <newest-tag>:manifest.ts` — without caring what is checked out. That is the fix; it is not
written yet, and until it is, this script advertises what is installed.
*/
const PLUGINS_DIR = process.env.PLUGINS_DIR ?? resolve(import.meta.dir, '..', 'plugins');
async function syncOne(dir: string): Promise<Sync | null> {
const appName = basename(dir);
const manifestPath = join(dir, 'manifest.ts');
if (!existsSync(manifestPath)) {
console.warn(`[marketplace-sync] ${appName}: no manifest.ts — skipped`);
return null;
if (args[0] === '--add') {
const raw = args[1];
if (!raw) {
console.error('usage: marketplace-sync.ts --add <clone url>');
process.exit(1);
}
const remote = git(dir, ['remote', 'get-url', 'origin']);
const repoUrl = remote ? toHttpsRemote(remote) : null;
const repoUrl = toHttpsRemote(raw);
if (!repoUrl) {
console.warn(`[marketplace-sync] ${appName}: no usable https remote (${remote ?? 'none'}) — skipped`);
return null;
console.error(`[marketplace-sync] ${raw} is not a clone URL this can turn into https`);
process.exit(1);
}
const appName = repoUrl.replace(/\.git$/, '').split('/').pop() ?? '';
if (!/^[a-z0-9][a-z0-9-]{0,63}$/.test(appName)) {
console.error(`[marketplace-sync] "${appName}" is not a usable app name`);
process.exit(1);
}
const { manifest } = (await import(manifestPath)) as { manifest: Record<string, unknown> };
if (!manifest) {
console.warn(`[marketplace-sync] ${appName}: manifest.ts exports no \`manifest\` — skipped`);
return null;
// Registered with a placeholder developer, then immediately synced — which reads the real publisher
// out of the released manifest and overwrites it. A repository with no tag stays registered and
// unpublished, which is the correct state for one that has not made a release yet.
await upsertPlugin({ appName, developer: 'unknown', repoUrl });
console.log(`[marketplace-sync] registered ${appName} → ${repoUrl}`);
const one = await syncPlugin(appName);
report([one]);
process.exit(one.status === 'failed' ? 1 : 0);
}
// What the plugin SHIPS, read from the directory rather than declared — the same rule discovery
// follows, so a catalogue entry cannot claim a sidecar the repository does not have.
const has = {
api: existsSync(join(dir, 'api', 'router.ts')),
schema: existsSync(join(dir, 'db', 'schema.ts')),
sidecar: existsSync(join(dir, 'sidecar', 'index.ts')) || existsSync(join(dir, 'sidecar', 'index.mjs')),
web: existsSync(join(dir, 'web')),
};
const results = args[0] ? [await syncPlugin(args[0])] : await syncAll();
report(results);
process.exit(results.some((r) => r.status === 'failed') ? 1 : 0);
const readmePath = ['README.md', 'readme.md'].map((n) => join(dir, n)).find((p) => existsSync(p));
const readme = readmePath ? readFileSync(readmePath, 'utf-8') : null;
const developer = String(manifest.publisher ?? 'unknown');
const pluginId = await upsertPlugin({ appName, developer, repoUrl });
await upsertMetadata(pluginId, {
version: String(manifest.version ?? '0.0.0'),
platform: String(manifest.platform ?? '>=1.0.0'),
label: String(manifest.label ?? appName),
summary: String(manifest.summary ?? ''),
// Taken verbatim, NOT validated here. `manifestProblems` already refuses a tag outside the
// vocabulary at discovery, so anything reaching this point is either valid or belongs to a plugin
// the platform would not load anyway — and a sync that silently dropped a tag would be a catalogue
// quietly disagreeing with the repository it claims to mirror.
tags: Array.isArray(manifest.tags) ? manifest.tags : [],
color: (manifest.color as string) ?? null,
tile: (manifest.tile as string) ?? null,
// Artwork lives in the repo, so the URL is built from the remote rather than stored by hand. Only
// claimed when the file is actually there — a broken image is worse than a glyph.
iconUrl: existsSync(join(dir, 'assets', 'icon.png')) ? rawUrl(repoUrl, 'assets/icon.png') : null,
icon: (manifest.icon as string) ?? null,
permissions: manifest.permissions ?? [],
osDependencies: manifest.osDependencies ?? null,
has,
readme,
sourceCommit: git(dir, ['rev-parse', 'HEAD']),
});
console.log(
`[marketplace-sync] ${appName} ← ${repoUrl}${readme ? '' : ' (no README)'}${has.web ? '' : ' (no web/)'}`,
);
return { appName, developer, repoUrl };
function report(results: Awaited<ReturnType<typeof syncAll>>): void {
for (const r of results) {
if (r.status === 'synced') console.log(`[marketplace-sync] ${r.appName} ← ${r.tag}`);
else if (r.status === 'unreleased') console.warn(`[marketplace-sync] ${r.appName}: no tag — nothing published`);
else console.error(`[marketplace-sync] ${r.appName} failed: ${r.error}`);
}
/**
* A raw-file URL for a Gitea repository.
*
* `HEAD`, not a branch name. This said `branch/main` and the comment here warned that "a repo whose
* default branch is `master` gets a 404" — which then happened: four plugin repositories were created
* on `master` and every one of them served a broken image in the marketplace on any server that reads
* the remote catalogue. A warning in a comment is not a guard.
*
* Gitea resolves `HEAD` to whatever the default branch is, so the question stops being asked. Verified
* against both spellings.
*
* `[open]` Still Gitea-shaped. A GitHub-hosted plugin needs the other form, and that wants the forge to
* be part of the registry rather than inferred from the URL.
*/
function rawUrl(repoUrl: string, path: string): string {
return `${repoUrl.replace(/\.git$/, '')}/raw/HEAD/${path}`;
const synced = results.filter((r) => r.status === 'synced').length;
const unreleased = results.filter((r) => r.status === 'unreleased').map((r) => r.appName);
const failed = results.filter((r) => r.status === 'failed').map((r) => r.appName);
console.log(`[marketplace-sync] ${synced}/${results.length} published from a release`);
// Named rather than counted. "3 unreleased" is a number nobody acts on; the names are a to-do list,
// and every one of them is a plugin whose store entry is frozen until somebody tags it.
if (unreleased.length) console.warn(`[marketplace-sync] never tagged, so not updated: ${unreleased.join(', ')}`);
if (failed.length) console.error(`[marketplace-sync] failed: ${failed.join(', ')}`);
}
const dirs = existsSync(PLUGINS_DIR)
? [...new Bun.Glob('*/').scanSync({ cwd: PLUGINS_DIR, onlyFiles: false })]
// Not a plugin: the shared install target for plugin npm packages, which lives here because bun
// resolves node_modules up the tree. `discoverPlugins` skips it for the same reason.
.filter((d) => d.replace(/\/$/, '') !== 'node_modules')
.map((d) => join(PLUGINS_DIR, d))
: [];
if (dirs.length === 0) {
console.log(`[marketplace-sync] no plugin directories under ${PLUGINS_DIR} — nothing to sync`);
process.exit(0);
}
// Discovery is not used to FIND them — a plugin that fails discovery is exactly the kind of thing a
// catalogue should still be able to describe — but its broken list is worth surfacing here.
const { broken } = await discoverPlugins().catch(() => ({ broken: [] as { appName: string; error: string }[] }));
for (const { appName, error } of broken) console.warn(`[marketplace-sync] ${appName} is broken on disk: ${error}`);
const synced: Sync[] = [];
for (const dir of dirs) {
const result = await syncOne(dir).catch((err) => {
console.error(`[marketplace-sync] ${basename(dir)} failed: ${err instanceof Error ? err.message : err}`);
return null;
});
if (result) synced.push(result);
}
console.log(`[marketplace-sync] ${synced.length}/${dirs.length} plugin(s) in the catalogue`);
process.exit(0);
src/databases/marketplace_db/src/plugins/queries.ts
+25
View File
@@ -14,6 +14,14 @@ export type CatalogueEntry = {
appName: string;
publisher: string;
version: string;
/**
* The git tag this entry was published from.
*
* Optional on the wire, not in the database, and the two mean different things. Here it is absent for
* a row that predates release-based publishing OR for a catalogue too old to send the field at all —
* so a consumer must treat "no tag" as "cannot tell" and fall back to whatever it knew before.
*/
releaseTag?: string;
platform: string;
label: string;
summary: string;
@@ -52,6 +60,7 @@ function toCatalogueEntry(row: Row): CatalogueEntry | null {
appName: row.appName,
publisher: row.developer,
version: m.version,
...(m.releaseTag ? { releaseTag: m.releaseTag } : {}),
platform: m.platform,
label: m.label,
summary: m.summary,
@@ -106,6 +115,22 @@ export async function catalogueEntry(appName: string): Promise<CatalogueEntry |
return { ...entry, ...(row.metadata.readme ? { readme: row.metadata.readme } : {}) };
}
/**
* Every registered plugin — identity and clone URL, no metadata.
*
* This is what the sync iterates, and it is the reason the registry is called authoritative: the
* catalogue lists what somebody deliberately registered, not what happens to be checked out on the
* machine that runs the job. Unlisted rows are included; hiding a plugin from the public API is not the
* same as no longer tracking it, and one taken down for a bad release has to be able to come back
* current.
*/
export async function listRegistry(): Promise<{ id: number; appName: string; repoUrl: string }[]> {
return marketplaceDb
.select({ id: plugins.id, appName: plugins.appName, repoUrl: plugins.repoUrl })
.from(plugins)
.orderBy(plugins.appName);
}
// ── Writes. Used by the sync script, never by the public API ──
type RegisterParams = { appName: string; developer: string; repoUrl: string };
src/databases/marketplace_db/src/plugins/schema.ts
+13
View File
@@ -73,6 +73,19 @@ export const pluginMetadata = pgTable(
.references(() => plugins.id, { onDelete: 'cascade' }),
version: text('version').notNull(),
/**
* The git tag this row was read from — what "published" actually means.
*
* `version` above is the manifest's own claim, and the two are the same string every time somebody
* tags correctly. They are kept apart because when they DISAGREE the tag is the fact: it is what was
* pushed, and a manifest whose version nobody bumped would otherwise make a real release invisible
* to anything comparing versions.
*
* Nullable because rows existed before releases were the unit of publishing. A null one is a
* snapshot taken from a working tree by the old sync, and every consumer should read it as "unknown"
* rather than "no release".
*/
releaseTag: text('release_tag'),
/** The platform range it declares support for, e.g. `>=1.0.0`. */
platform: text('platform').notNull(),
label: text('label').notNull(),
src/servers/marketplace/sync.ts
+287
View File
@@ -0,0 +1,287 @@
import { existsSync, mkdirSync } from 'node:fs';
import { homedir, tmpdir } from 'node:os';
import { join } from 'node:path';
import {
listRegistry,
upsertMetadata,
upsertPlugin,
} from '../../databases/marketplace_db/src/plugins/queries';
/**
* Fill the catalogue from what each plugin has RELEASED.
*
* ── The bug this file exists to remove ──
*
* Until 2026-09-03 the sync read `platform/plugins/` — the directories this machine has INSTALLED — and
* imported each `manifest.ts` off the working tree. So "publish" meant "make sure officer-dev's checkout
* happens to be at the release, then run the script", and nobody could see when it was not. On the day
* this was rewritten the live catalogue was advertising four plugins from untagged working copies:
* av-club 2026.08.29.01, kanban 2026.08.29.02, openseo 0.1.0 and fofos-admin 2026.08.29.26, none of
* which had ever been tagged. The store was describing somebody's afternoon.
*
* That is also why the old script could not be put on a timer: a cron over it would have published
* whatever was sitting in a working tree, on a schedule.
*
* ── What replaces it: a bare mirror, read at a tag ──
*
* Every plugin in the registry is mirrored (`git clone --mirror`) into a cache directory, refreshed, and
* read AT ITS NEWEST TAG — the manifest, the README, the file list, all out of `git cat-file`. There is
* no working tree anywhere in this path, so publishing something uncommitted is not a mistake that can
* be made. A mirror is cheap: the whole set clones in a couple of seconds and refreshes in well under
* one.
*
* `docs/marketplace.md` proposed instead reading manifests over Gitea's API, which needed either a
* sandbox to evaluate a fetched string or a committed `manifest.json` for every plugin. Git gives the
* same guarantee — content at a named release, independent of any checkout — with neither, and the trust
* model is unchanged from today: we import our own repositories' code.
*
* `[still a blocker for third-party submissions]` It is unchanged BECAUSE it is unchanged. Importing a
* stranger's manifest is code execution no matter which ref it came from. Submissions still need a
* sandbox or a published `manifest.json`.
*
* ── Which plugins, and where from: the registry, and only the registry ──
*
* The list comes from the `plugins` table, which `docs/marketplace.md` has always called the
* authoritative, hand-curated half. It was not actually being used that way — the old sync discovered
* plugins by listing a directory, so the registry was a by-product of whatever officer-dev had
* installed. Now it decides, which means:
*
* · a plugin nobody has installed here can still be published (`finances` is in exactly that state),
* · a plugin lives wherever its registry row says, across orgs — `fofos-admin` is under `fofos/`,
* not `plugins/`, so an org listing would have missed it,
* · and adding one is a deliberate act: `marketplace-sync.ts --add <https clone url>`.
*
* ── A plugin with no tag is left ALONE, not removed ──
*
* It has never made a release, so this file has nothing true to say about it and says nothing: the
* existing metadata row stays, and the run reports it. Deleting the row would drop it out of the store
* on the day the rule changed, which is a decision about somebody's plugin dressed up as a refresh.
* Tagging it is the fix, and the report is there to ask for one.
*/
export type SyncOutcome =
| { appName: string; status: 'synced'; tag: string; version: string }
| { appName: string; status: 'unreleased' }
| { appName: string; status: 'failed'; error: string };
/*
Where the mirrors live.
A fixed path under the user's cache directory, NOT one derived from the environment. This runs three
ways — by hand with `--env-file=.env`, from the webhook inside `officer-marketplace`, and from cron —
and those three do not have the same environment. A path that moved with `DATA_PATH` would have each
of them re-cloning twenty-five repositories into its own copy, and the "is it already mirrored" check
would answer differently depending on who asked.
Not in the source tree, because it is a cache of remote repositories and a `.gitignore` entry is a
worse way to keep several hundred megabytes out of a repo than not putting them there.
*/
const MIRROR_DIR =
process.env.MARKETPLACE_MIRROR_DIR ?? join(homedir(), '.cache', 'officer-marketplace', 'mirrors');
const GIT_TIMEOUT_MS = 60_000;
type Git = { ok: true; out: string } | { ok: false; err: string };
async function git(args: string[], cwd?: string): Promise<Git> {
// A missing `cwd` makes posix_spawn fail with ENOENT, which reads as "git is not installed" and sent
// a first run of this looking for git for ten minutes. It means the mirror is not there yet.
if (cwd && !existsSync(cwd)) return { ok: false, err: `${cwd} does not exist` };
const proc = Bun.spawn(['git', ...args], {
cwd,
stdout: 'pipe',
stderr: 'pipe',
// Never inherited: a mirror fetch that hits a host asking for a password would otherwise sit on a
// terminal prompt forever, and in the webhook server there is no terminal to sit on.
env: { ...process.env, GIT_TERMINAL_PROMPT: '0', GIT_ASKPASS: '/bin/true' },
signal: AbortSignal.timeout(GIT_TIMEOUT_MS),
});
const [out, err] = await Promise.all([new Response(proc.stdout).text(), new Response(proc.stderr).text()]);
return (await proc.exited) === 0 ? { ok: true, out: out.trim() } : { ok: false, err: err.trim() || 'git failed' };
}
/** The mirror for a plugin, cloned if this is the first time and refreshed if it is not. */
async function mirror(appName: string, repoUrl: string): Promise<string> {
mkdirSync(MIRROR_DIR, { recursive: true });
const dir = join(MIRROR_DIR, `${appName}.git`);
// Re-cloned rather than repaired when it is not a healthy bare repo: an interrupted clone leaves a
// directory that every later command fails against, and there is nothing in it worth keeping.
const existing = await git(['rev-parse', '--is-bare-repository'], dir);
if (!existing.ok || existing.out !== 'true') {
if (existsSync(dir)) await Bun.$`rm -rf ${dir}`.quiet().nothrow();
const cloned = await git(['clone', '--mirror', '--quiet', repoUrl, dir]);
if (!cloned.ok) throw new Error(`clone failed: ${cloned.err}`);
return dir;
}
// The registry is allowed to move a plugin. Setting the URL every time is one command and removes the
// failure where a repo is relocated and the mirror keeps refreshing from where it used to be.
await git(['remote', 'set-url', 'origin', repoUrl], dir);
// `--prune --prune-tags`: a deleted or MOVED tag has to disappear here too. Without it, a tag that was
// pushed by mistake and deleted would stay in the mirror forever and could go on being published as
// the newest release — a store advertising a version its repository no longer has.
const fetched = await git(['fetch', '--quiet', '--prune', '--prune-tags', '--tags', 'origin'], dir);
if (!fetched.ok) throw new Error(`fetch failed: ${fetched.err}`);
return dir;
}
/**
* The newest release, or null for a plugin that has never made one.
*
* Asked of git rather than by sorting version strings, which is the same rule `servers/plugins/release.ts`
* follows and for the same reason: parsing `2026.08.27.01` works right up until somebody tags `v2`, and
* then fails silently. `--abbrev=0` gives the closest tag reachable from HEAD, and a mirror's HEAD is the
* remote's default branch, so this is "the newest release on the branch the repository points at".
*/
async function newestTag(dir: string): Promise<string | null> {
const described = await git(['describe', '--tags', '--abbrev=0', 'HEAD'], dir);
return described.ok ? described.out : null;
}
/** A file at a ref, or null when it is not in that tree. */
async function fileAt(dir: string, ref: string, path: string): Promise<string | null> {
const shown = await git(['cat-file', '-p', `${ref}:${path}`], dir);
return shown.ok ? shown.out : null;
}
/**
* The manifest at a release, as an object.
*
* Written to a temp file and imported, because a manifest is TypeScript. Keyed by the COMMIT, so the
* module cache is correct rather than merely convenient: the same commit is the same file, and a tag
* moved to a different commit gets a different path and a fresh import. That matters in the webhook
* server, which is long-lived and would otherwise serve the first version of a manifest it ever read.
*
* The `import type { PluginManifest }` at the top of every manifest is erased by the transpiler, so the
* file has no runtime dependency on the platform's path aliases and imports fine from anywhere.
*/
async function manifestAt(dir: string, ref: string, appName: string, commit: string): Promise<Record<string, unknown>> {
const source = await fileAt(dir, ref, 'manifest.ts');
if (source === null) throw new Error(`no manifest.ts at ${ref}`);
const scratch = join(tmpdir(), 'officer-marketplace-manifests');
mkdirSync(scratch, { recursive: true });
const path = join(scratch, `${appName}-${commit}.ts`);
await Bun.write(path, source);
const mod = (await import(path)) as { manifest?: Record<string, unknown> };
if (!mod.manifest) throw new Error(`manifest.ts at ${ref} exports no \`manifest\``);
return mod.manifest;
}
/**
* A raw-file URL for a Gitea repository, PINNED TO THE RELEASE.
*
* It used to be `/raw/HEAD/…`, which works only because Gitea 303s it to `/raw/branch/HEAD/…`; the
* artwork it served was whatever was on the default branch, so a plugin could show one icon in the store
* and ship another. `/raw/tag/<tag>/…` answers 200 directly and shows the icon that belongs to the
* version being advertised.
*
* `[open]` Still Gitea-shaped. A GitHub-hosted plugin needs the other form, and that wants the forge to
* be part of the registry rather than inferred from the URL.
*/
function rawUrl(repoUrl: string, tag: string, path: string): string {
return `${repoUrl.replace(/\.git$/, '')}/raw/tag/${encodeURIComponent(tag)}/${path}`;
}
async function syncRegistered(row: { id: number; appName: string; repoUrl: string }): Promise<SyncOutcome> {
const { appName, repoUrl } = row;
const dir = await mirror(appName, repoUrl);
const tag = await newestTag(dir);
if (!tag) return { appName, status: 'unreleased' };
const commitAt = await git(['rev-list', '-n', '1', tag], dir);
if (!commitAt.ok) throw new Error(`cannot resolve ${tag}`);
const commit = commitAt.out;
const manifest = await manifestAt(dir, tag, appName, commit);
// What the plugin SHIPS, read from the tree at the release rather than from a declaration — the same
// rule discovery follows on disk, so a catalogue entry cannot claim a sidecar the release does not
// have.
const listed = await git(['ls-tree', '-r', '--name-only', tag], dir);
const files = new Set(listed.ok ? listed.out.split('\n') : []);
const has = {
api: files.has('api/router.ts'),
schema: files.has('db/schema.ts'),
sidecar: files.has('sidecar/index.ts') || files.has('sidecar/index.mjs'),
web: [...files].some((f) => f.startsWith('web/')),
};
const readme = (await fileAt(dir, tag, 'README.md')) ?? (await fileAt(dir, tag, 'readme.md'));
const version = String(manifest.version ?? tag);
if (version !== tag) {
// Not an error and not corrected here. The tag is what was released and the manifest is what the
// plugin will report about itself once installed; when they disagree, an update badge that compares
// versions goes quiet. Saying so is the only way anyone finds out.
console.warn(`[marketplace-sync] ${appName}: tag ${tag} but manifest says ${version} — they should match`);
}
const developer = String(manifest.publisher ?? 'unknown');
const pluginId = await upsertPlugin({ appName, developer, repoUrl });
await upsertMetadata(pluginId, {
version,
releaseTag: tag,
platform: String(manifest.platform ?? '>=1.0.0'),
label: String(manifest.label ?? appName),
summary: String(manifest.summary ?? ''),
// Taken verbatim, NOT validated here. `manifestProblems` already refuses a tag outside the
// vocabulary at discovery, so anything reaching this point is either valid or belongs to a plugin
// the platform would not load anyway — and a sync that silently dropped a tag would be a catalogue
// quietly disagreeing with the repository it claims to mirror.
tags: Array.isArray(manifest.tags) ? manifest.tags : [],
color: (manifest.color as string) ?? null,
tile: (manifest.tile as string) ?? null,
iconUrl: files.has('assets/icon.png') ? rawUrl(repoUrl, tag, 'assets/icon.png') : null,
icon: (manifest.icon as string) ?? null,
permissions: manifest.permissions ?? [],
osDependencies: manifest.osDependencies ?? null,
has,
readme,
sourceCommit: commit,
});
return { appName, status: 'synced', tag, version };
}
/** One plugin by app name. Unknown to the registry is a failure, not a silent no-op. */
export async function syncPlugin(appName: string): Promise<SyncOutcome> {
const row = (await listRegistry()).find((r) => r.appName === appName);
if (!row) return { appName, status: 'failed', error: 'not in the registry' };
return syncRegistered(row).catch((err) => ({
appName,
status: 'failed' as const,
error: err instanceof Error ? err.message : String(err),
}));
}
/**
* Every registered plugin, one at a time.
*
* Sequential on purpose. Twenty-four concurrent fetches against one Gitea is a thundering herd for a job
* nothing is waiting on, and the whole set finishes in about a second anyway.
*
* Unlisted rows are included: `listed` hides a plugin from the public API, usually after a bad release,
* and it has to be able to come back with current metadata rather than with whatever was true when it
* was pulled.
*/
export async function syncAll(): Promise<SyncOutcome[]> {
const rows = await listRegistry();
const out: SyncOutcome[] = [];
for (const row of rows) {
out.push(
await syncRegistered(row).catch((err) => ({
appName: row.appName,
status: 'failed' as const,
error: err instanceof Error ? err.message : String(err),
})),
);
}
return out;
}