officerdev/platform
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

publishing a plugin is pushing a tag

Browse Source 
Three mechanisms where there should be one. The catalogue was filled by hand, and each host separately
polled twenty-one repositories with git to work out whether a release existed — a second source of
truth for a fact one place already knew. Shipping photos 2026.09.03.02 correctly and watching alpha
not notice for ten minutes is what started this.

The timer. scripts/marketplace-cron.sh, every minute, on the machine that publishes. Safe now only
because the previous commit made the sync read tags instead of working trees; over the old one this
would have published somebody's afternoon on a schedule. Nothing is written unless the release commit
differs from the one already published, so a tick with nothing to do is one query, one fetch per repo
and no writes — about 2.7s. One timestamped summary line per tick regardless, because a fresh timestamp
is the only thing that separates "nothing to publish" from "the timer died"; a named line only for a
plugin that moved.

The badge. `updateable` now compares the tag at HEAD with the tag the catalogue says is newest —
difference, not ordering, since parsing 2026.08.27.01 works right up until somebody tags v2 and then
fails silently. refreshRemote() and its two-minute window are gone with the polling they existed to
paper over. The git rule stays as the fallback for a plugin the store cannot speak for: an unregistered
checkout, or an unreachable store. That case now reads whatever refs the last pull left, which is the
accepted cost of not making every machine poll every repository forever.

fetchCatalogue()'s cache went from five minutes to one. It only drew tiles before; now the wait for a
release to appear would have been the sync's minute plus its five.

Also: the Update button names its destination whenever a release is known, not only on `new-release`.
An untagged checkout reports `untagged` and used to lose the tag with it, so the button said "Update"
and the owner had to guess where it landed — the marketplace knows, even for a checkout that has never
fetched it.

Verified: forced a stale row, the timer republished it inside 60s and logged one line. Badge decisions
checked against the live catalogue for seven plugins, including the two shapes that matter — at a tag
the store agrees with, and untagged with a release known only to the store. 9 release tests pass, 100%
of release.ts covered, four of them new and all against real repositories.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
pastilhas
2026-09-03 13:13:43 +01:00
co-authored by Claude Opus 5
parent 78e89ebcec
commit 19bbbab907
14 changed files with 360 additions and 119 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/marketplace-todo.md
+13 -9
View File
@@ -29,17 +29,21 @@ Ranked by what actually blocks something.
## Staleness
- [ ] **Nothing refreshes the catalogue automatically.** Publishing a plugin update changes nothing until
somebody runs `bun --env-file=.env scripts/marketplace-sync.ts`. A Gitea webhook or a cron is the
obvious next step.
- [x] **Nothing refreshes the catalogue automatically.** Done 2026-09-03: `scripts/marketplace-cron.sh`
every minute. A Gitea webhook would make it a second instead, at the price of a public write route
on the one process here with no login — not worth it for a minute.
- [ ] **The sync reads local clones, not the remotes.** The machine running it must have every plugin
checked out, so the catalogue describes that machine's copies. The version that clones each repo
into a temp directory is a small change.
- [x] **The sync reads local clones, not the remotes.** Done 2026-09-03. Bare mirrors, read at the newest
tag, driven by the registry rather than by a directory listing. The catalogue no longer describes
any machine's working copies.
- [ ] **`rawUrl()` assumes Gitea and a `main` branch.** A GitHub-hosted plugin, or one whose default branch
is `master`, gets a 404 for its icon. The forge belongs in the registry rather than inferred from a
URL shape.
- [ ] **`rawUrl()` assumes Gitea.** A GitHub-hosted plugin gets a 404 for its icon. The forge belongs in
the registry rather than inferred from a URL shape. (The `main`-vs-`master` half of this is gone —
URLs pin to the release tag now.)
- [ ] **Four plugins have never been tagged**, so the store still shows rows the old sync took from a
working tree: av-club, kanban, openseo, fofos-admin. They are frozen until somebody tags them, and
the sync names them on every run somebody asks for.
## Scale
docs/marketplace.md
+82 -17
View File
@@ -10,8 +10,9 @@ from. **The website installs nothing** — installing means cloning a repository
starting a process on someone else's machine, so it happens entirely inside the owner's own Officer.
```
gitea.officer.dev/plugins/* the repositories (source of truth)
│ scripts/marketplace-sync.ts (manual, reads manifest.ts + README.md)
gitea.officer.dev/** the repositories (source of truth)
│ bare mirrors, read AT THE NEWEST TAG — src/servers/marketplace/sync.ts
│ every minute, scripts/marketplace-cron.sh
▼
officer_marketplace (postgres) plugins + plugin_metadata
│ src/servers/marketplace/server.tsx :9100
@@ -20,6 +21,7 @@ marketplace.officer.dev GET /api/plugins, /api/plugins/:appName
│ fetchCatalogue() live → cached → bundled
▼
an Officer install /marketplace → clone → push schema → mount
and the update badge — see "Publishing" below
```
## The parts
@@ -78,19 +80,87 @@ disappearing from the store.
the clone, the plugin's own manifest is what the platform reads. A metadata row that disagrees with the
repository is a marketplace bug, never a second source of truth.
## Sync
## Publishing
**Push a tag. That is the whole procedure.** A timer on officer-dev picks it up within a minute and every
Officer install sees it on the next page load.
```bash
bun --env-file=.env scripts/marketplace-sync.ts
bun --env-file=.env scripts/marketplace-sync.ts # all of them, now
bun --env-file=.env scripts/marketplace-sync.ts photos # one
bun --env-file=.env scripts/marketplace-sync.ts --add https://gitea.officer.dev/plugins/finances.git
bun --env-file=.env scripts/marketplace-sync.ts --force # re-publish rows this file's own logic changed
```
Reads every directory under `plugins/`, imports its `manifest.ts`, derives `has` from the directory
rather than from a declaration, reads `README.md`, and upserts. Re-running is always safe: it overwrites,
never merges.
### It reads releases, never a working tree
It rewrites SSH remotes to HTTPS. `example` and `offscale` had `git@gitea…` origins, which would have
handed every installer a URL that only works on the machine that published it — the install fails at
authentication, which reads as "the plugin is broken" rather than "the catalogue is wrong".
Every registered plugin is mirrored bare into `~/.cache/officer-marketplace/mirrors` and read at its
newest tag — manifest, README and file list all out of `git cat-file`. There is no working tree in the
path, so publishing something uncommitted is not a mistake that can be made.
That is a change, and it was load-bearing. Until 2026-09-03 the sync imported `manifest.ts` off
`platform/plugins/` — the directories THIS machine has installed — so publishing meant "get officer-dev's
checkout to the release, then run the script", and nothing showed when that had not happened. On the day
it was rewritten the live store was advertising four plugins from untagged working copies: av-club
2026.08.29.01, kanban 2026.08.29.02, openseo 0.1.0 and fofos-admin 2026.08.29.26, none of which had ever
been tagged. It is also why the job could not go on a timer before: a cron over it published whatever was
in a working tree, on a schedule.
### The registry decides which plugins, and where from
The list comes from the `plugins` table — the half this document has always called authoritative, and
which was in fact a by-product of what happened to be checked out. Now it decides. A plugin nobody has
installed here can still be published, and a plugin lives wherever its row says: `fofos-admin` is under
`fofos/`, not `plugins/`, which an org listing would have missed. Registering one is `--add`.
### A plugin with no tag is left alone
It has never made a release, so the sync has nothing true to say about it: the existing row stays and the
run names it. Deleting it would drop somebody's plugin out of the store on the day the rule changed.
Tagging it is the fix, and the report is the reminder.
### `release_tag`, and why it is not `version`
`version` is the manifest's own claim; `release_tag` is what was pushed. They match every time somebody
tags correctly, and when they do not, the tag is the fact — a manifest whose version nobody bumped would
otherwise make a real release invisible to anything comparing versions. The sync warns on a mismatch.
Icon URLs pin to the tag too. `/raw/HEAD/…` served whatever was on the default branch, via a 303, so a
plugin could show one icon in the store and ship another.
### The timer
`scripts/marketplace-cron.sh`, `* * * * *`, logging to `officer-dev/marketplace-sync.log`. One
timestamped summary line per tick — always, including the ticks that do nothing, because a fresh
timestamp is the only thing that distinguishes "nothing to publish" from "the timer died". A plugin-level
line appears only when one actually moved.
A tick costs about 2.7s with nothing to do: one query, one fetch per repository, no writes. Nothing is
written unless the release commit differs from the one already published.
A Gitea webhook would cut the delay from a minute to a second, at the price of a public write route and a
shared secret. A minute was never the problem anybody had — the complaint that started this was a release
that never appeared at all.
### Remotes
`--add` rewrites SSH to HTTPS. A `git@gitea…` origin in the catalogue hands every installer a URL that
only works on the machine that published it, and the install fails at authentication — which reads as
"the plugin is broken" rather than "the catalogue is wrong".
## The update badge reads the catalogue
`updateable` on `/api/plugins` compares the tag at the checkout's HEAD with the tag the catalogue says is
newest. Difference, not ordering: parsing `2026.08.27.01` works right up until somebody tags `v2`, and
then it fails silently.
Before this, each host ran a rate-limited background `git fetch` against all twenty-one repositories to
re-derive a fact one place already knew. That is gone. `releaseState()` still holds the git rule as a
fallback for a plugin the store cannot speak for — an unregistered checkout, or a store that could not be
reached — and in that case its refs are whatever the last `pullPlugin` left, which is the accepted cost.
`fetchCatalogue()`'s cache dropped from five minutes to one when the badge started depending on it:
otherwise the wait for a release to appear was the sync's minute plus the cache's five.
## The platform degrades, it does not break
@@ -111,13 +181,8 @@ The full list, ranked, is `docs/marketplace-todo.md`. The ones that shape the de
- **`[blocker]` Third-party submissions cannot use this sync.** It reads a manifest by importing it, which
is code execution. Fine for our own repositories; not a mechanism a stranger's plugin can go through.
Submissions need either a sandbox or a published `manifest.json` alongside the TypeScript one.
- **`[open]` Sync reads local clones, not the remotes.** The machine running it must have the plugins
checked out. The version that clones each repo into a temp directory is a small change and was left out
of the first pass.
- **`[open]` `rawUrl()` assumes Gitea and a `main` branch.** A GitHub-hosted plugin or one defaulting to
`master` gets a 404 for its icon. The forge should become part of the registry rather than inferred.
- **`[open]` `rawUrl()` assumes Gitea.** A GitHub-hosted plugin needs the other URL form. The forge should
become part of the registry rather than inferred from the URL.
- **`[open]` No versions table.** `plugin_metadata` is one row per plugin — a snapshot, not a history. A
store that can offer "install 1.2 instead of 1.3" needs a different shape.
- **`[open]` Nothing refreshes automatically.** Publishing a plugin update does not change the catalogue
until somebody runs the sync. A cron or a Gitea webhook is the obvious next step.
- **`[open]` No search, no categories, no counts.** Three plugins do not need them.