Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.
Python agent-orchestrationai-agentai-coding-assistantclaude-codecoding-agentcomputer-usedesktop-applocal-firstmcppersonal-jarvisvoice-assistantvoice-coding
docs(marketplace): the publish-half design records, with their status brought up to date
rubenluetke10-beepbef19638914d
diff --git a/docs/marketplace/github-login-analysis.md b/docs/marketplace/github-login-analysis.mdnew file mode 100644index 0000000000..8a99034038--- /dev/null+++ b/docs/marketplace/github-login-analysis.md@@ -0,0 +1,299 @@+# Sign in with GitHub — design analysis for marketplace uploads++**Status:** ANALYSIS — no code written yet ·+**Written:** 2026-08-14 ·+**Depends on:** [community-registry.md](community-registry.md) (shipped registry),+[public-marketplace-analysis.md](public-marketplace-analysis.md) (D1–D9 decisions)++The goal under analysis: **a "Sign in with GitHub" gate on the Jarvis+marketplace — GitHub is the only identity provider, and only a signed-in+account can upload a skill or a plugin.** No email/password, no second+account system.++Everything in §1 was measured against the live repos and the live site on+2026-08-14, not inferred from the docs.++---++## 1. Measured starting position++| Piece | State on 2026-08-14 |+|---|---|+| Registry repo `PersonalJarvis/marketplace` | Live. `submissions/`, `plugins/`, `skills/`, `registry.json`, `schemas/submission.schema.json`, four CI scripts (`validate.py`, `automerge_gate.py`, `expand.py`, `build_index.py`), three workflows (`validate`, `automerge`, `publish`). |+| Compiled feed | Live — `https://personaljarvis.github.io/marketplace/index.json` answers 200; Pages serves `main` at `/`. |+| Ownership ledger | `registry.json` holds two entries (`sentry`, `three-bullet-brief`), each with `publisher` = a GitHub **login string** and a SemVer version. |+| Auto-merge gate | Merges only when the PR touches exactly one `submissions/<name>.json`, validation passes, **`publisher` equals the PR author's login**, and (on updates) the owner is unchanged and the version strictly increases. |+| Submission format | One JSON file. A skill embeds its complete `SKILL.md` as the `skill_md` string (≤ 64 KB); the whole file is capped at 128 KB; descriptions at 500 chars. |+| Publish path used so far | **Zero pull requests have ever been opened** against the registry — both live entries were committed directly. The contributor path is therefore untested in practice. |+| Storefront | Private Astro 6 repo, **static build, no adapter, no server routes**. `src/pages/` has the landing page, docs, `de/`, `es/`, legal pages — **no `/marketplace` and no `/marketplace/submit`**. |+| Live site | `https://personaljarvis.ai` answers 200 **through Cloudflare**. `/marketplace` and `/marketplace/submit` both answer **404** — the paths the registry README already advertises do not exist yet. |+| App-side | Browse + install + consent dialog are shipped (`marketplace_routes.py`, `PluginsCommunity.tsx`). There is **no publish path in the app at all**. |+| GitHub auth substrate | `jarvis/marketplace/auth/oauth_device.py` is a complete, working **OAuth Device Flow** handler (start, poll, refresh, all four GitHub error codes). It is currently unused for GitHub: the shipped `github` plugin uses `pat_paste`. **No OAuth App or GitHub App is registered for the project.** |++Two consequences fall straight out of this table:++* **The identity gate already exists in the registry** — it is the+ `publisher == PR author` rule. A login button does not introduce+ GitHub identity; it replaces *"prove it by opening a PR yourself"* with+ *"prove it by clicking a button"*.+* **The upload surface is what is missing**, not the trust model. Both the+ storefront submit page and any in-app publish button are unwritten.++---++## 2. What the login must buy, and what it must not pretend to buy++**Buys:**++1. **Upload without git.** Today publishing requires forking a repo, writing+ a JSON file by hand, and opening a PR. That excludes most skill authors.+2. **Un-forgeable publisher identity.** The `publisher` field stops being+ self-declared text and becomes a value the server derived from a GitHub+ token.+3. **A rate-limiting subject.** "Five submissions per account per day" is+ only expressible once submissions carry a verified account.+4. **A "my submissions" view** — without any database, because+ `registry.json` already *is* the ownership table.++**Does not buy:**++* **Trust in the content.** A GitHub account costs nothing and takes+ seconds. The login proves *who* published, never *whether it is safe*.+ The "Community · not reviewed" badge, the consent dialog, and+ `validate.py` stay exactly as load-bearing as they are today.+* **Spam immunity.** Throwaway accounts exist. Account age checks, per-account+ quotas, and a Cloudflare Turnstile challenge on the submit endpoint are the+ actual anti-spam layer (§7).++---++## 3. The hard constraint: a browser cannot finish a GitHub login++GitHub's token endpoint (`https://github.com/login/oauth/access_token`)+serves **no CORS headers and no OPTIONS**, deliberately and unchanged for+years. A purely static page therefore cannot exchange the `code` for a+token — every browser-based GitHub login on earth runs the exchange on a+server. Two documented facts shape every option below:++* **Web flow requires the client secret** at the token exchange. A secret+ cannot live in a static bundle, so the exchange must be server-side.+* **Device flow requires no client secret** (only `client_id`, `device_code`,+ `grant_type`) — which is precisely why it is the right flow for the+ desktop app, and why `oauth_device.py` was written that way. It must be+ explicitly enabled in the app's settings.++So: the storefront login needs exactly one server endpoint. The desktop app+needs none. That asymmetry drives the recommendation.++---++## 4. Candidate designs++### L0 — No login: pre-filled PR link (today's documented plan)++The submit form builds the JSON client-side and hands the user a GitHub+"create new file" URL with the content pre-filled. GitHub itself demands the+login, auto-forks, and opens the PR.++* **Pros:** zero infrastructure, zero secrets, zero new attack surface; the+ existing gate works untouched; degrades to the current manual path.+* **Cons:** the user still lands in GitHub's PR UI (three unfamiliar screens);+ the site never knows who is signed in, so no quota, no "my submissions",+ no per-account abuse handling; URL length limits make a 64 KB `skill_md`+ fragile; **fails the stated requirement** ("only signed-in users can+ upload") in spirit, since the site itself has no notion of signed-in.++### L1 — OAuth App + user's own token opens the PR++Classic web flow with scope `public_repo`. A server endpoint exchanges the+code; then the *user's* token forks the registry, commits the file, and opens+the PR as the user.++* **Pros:** the existing auto-merge gate needs **no change at all** (the PR+ author really is the publisher); provenance is visible in git history.+* **Cons:** `public_repo` grants **write access to every public repo the user+ owns** — the coarsest scope GitHub offers, and a hard sell to exactly the+ developer audience this targets; leaves a fork in every publisher's account;+ the server must hold a live user token for the duration of the write.++### L2 — GitHub App + bot opens the PR *(recommended)*++A GitHub App registered under the `PersonalJarvis` org, installed **only on+the registry repo**. Login uses the App's web flow purely to answer *"who are+you"* — the consent screen asks for **no repository access on the user's+side**. The server then writes `submissions/<name>.json` to a branch **in the+registry repo** with its own installation token and opens the PR as the App.++* **Pros:** the user grants nothing but identity; no fork litter; the server+ never needs to retain a user token past the identity check; the App's+ installation token is scoped to one repo with two permissions; the same App+ serves the desktop app via device flow, so there is one identity for both+ surfaces.+* **Cons:** the auto-merge gate needs a second, equally strict trust rule+ (§6); every PR shows the same bot as author (the publisher lives in the+ file and in the PR body).++### L3 — Hosted registry with its own accounts and database++Rejected in the earlier analysis (Model D) and still rejected: it trades the+project's "no maintainer infrastructure in the critical path" doctrine for+UX that L2 already delivers.++### Trade-off summary++| | L0 pre-filled PR | L1 OAuth App + fork | L2 GitHub App + bot | L3 hosted service |+|---|---|---|---|---|+| Meets "only signed-in can upload" | ✗ (GitHub gates, site doesn't) | ✓ | ✓ | ✓ |+| What the user grants | nothing | write to **all** public repos | identity only | account + password |+| New infrastructure | none | 1 endpoint + secret | 1 endpoint + secret + key | server + DB + ops |+| Change to the merge gate | none | none | one added rule | replaced |+| Per-account quotas | impossible | possible | possible | possible |+| Failure blast radius | — | publishing only | publishing only | browse + install too |++---++## 5. Recommendation++**L2 — one GitHub App, login for identity only, the App commits the+submission and opens the pull request.**++Why over the runner-up (L1): both give a real login and identical+publishing UX, but L1 makes every skill author hand a marketing website+write access to *all of their public repositories* just to upload a+Markdown file — for an audience of developers that is the difference+between "sure" and "no thanks". L2's consent screen asks for nothing on the+user's account, and the cost is a single extra branch in the gate.++The endpoint belongs on **Cloudflare** (Pages Functions or a Worker on+`personaljarvis.ai/api/*`): the live site already serves through Cloudflare,+so this adds no new vendor, no new bill, and no new domain — and the Astro+site can stay a static build with the function beside it.++Doctrine check (contract §3, "no maintainer infrastructure in the critical+path"): if that endpoint is down, **browsing, installing, the feed, and the+manual PR route all keep working** — only the convenience upload pauses. That+is an acceptable, honestly-degrading dependency, and it must be documented as+such on the submit page.++---++## 6. What we need from GitHub — the concrete checklist++One GitHub App, created in the `PersonalJarvis` organisation. Cost: **€0**.++| Setting | Value | Why |+|---|---|---|+| App name | `Personal Jarvis Marketplace` | Shown on the consent screen. |+| Homepage URL | `https://personaljarvis.ai` | — |+| Callback URL | `https://personaljarvis.ai/api/auth/github/callback` | Exact match required; add a `localhost` second callback for development. |+| Request user authorization (OAuth) during installation | **off** | Users never install the App; they only log in. |+| Enable Device Flow | **on** | Needed for the desktop-app publish path (§9). |+| Webhook | **off** | Nothing listens. |+| Where can this App be installed | **Only on this account** | It is installed on exactly one repo. |+| Repository permissions | `Contents: Read & write`, `Pull requests: Read & write` | Enough to push a branch and open the PR. Nothing else. |+| Account permissions | **none** | The public profile (`GET /user`) needs no permission — this is what keeps the consent screen empty. |+| Installed on | `PersonalJarvis/marketplace` only | Blast radius = one repo. |++Credentials it produces, all of which live **only** in the Cloudflare+project's secret store — never in a repo, never in `jarvis.toml`:++* `GITHUB_APP_ID`, `GITHUB_APP_CLIENT_ID` (not secret, but keep them together)+* `GITHUB_APP_CLIENT_SECRET` — the web-flow exchange+* `GITHUB_APP_PRIVATE_KEY` (PEM) — mints installation tokens+* `SESSION_SIGNING_KEY` — signs the login cookie++Rate limits are a non-issue at this scale: 5.000 requests/hour both for+user-to-server and installation tokens.++---++## 7. Security analysis++| Risk | Why it matters here | Mitigation |+|---|---|---|+| **Login renaming / publisher hijack** | `registry.json` stores the publisher as a **login string**. GitHub logins can be renamed, and the freed name can be claimed by anyone — who would then inherit ownership of every skill under it. This is the classic repo-jacking shape, and it is live today. | Store the **numeric account id** alongside the login (`publisher_id`) and make the ownership check compare ids, with the login kept for display only. Worth doing regardless of the login work. |+| Confused deputy (token from a foreign app) | The desktop app sends a token to our endpoint; a token minted by *someone else's* OAuth app would otherwise be accepted as proof of identity. | Verify every incoming token with `POST /applications/{client_id}/token` under the App's own basic auth — it answers 200 only for tokens belonging to *our* client id. |+| Forged `publisher` in a bot PR | Bot PRs are trusted by design, so the branch must be unforgeable. | Only the App installation can push to the registry repo; the gate accepts the trusted path only when `head.repo.full_name == PersonalJarvis/marketplace` **and** the PR author is the App's bot login. Fork PRs keep the old `publisher == author` rule. |+| CSRF / code interception on login | Standard web-flow exposure. | `state` nonce bound to a short-lived signed cookie; strict `redirect_uri` matching; PKCE where GitHub supports it. |+| Token leakage into the browser | A leaked user token is a live GitHub credential. | The user token never leaves the server and is discarded right after the identity check — nothing to leak. The browser gets only an httpOnly, `SameSite=Lax`, signed session cookie carrying login + id. |+| Spam submissions | A login is free; a bot farm can hold many. | Cloudflare Turnstile on the submit endpoint, a per-account daily quota, and a minimum account age; every submission still runs the full `validate.py`. |+| Malicious content | Unchanged by the login. | `validate.py` in CI, the same rules re-enforced client-side at install (`agent_plugins_loader.py`), the unreviewed badge, and the consent dialog. |+| GDPR / data held | An identity provider tempts you into a user table. | Store **nothing**. The session cookie is stateless; ownership lives in `registry.json`, which is public data (login + numeric id) already. |++**One CI trap worth naming:** pull requests opened with a GitHub App+installation token *do* trigger workflows (unlike ones opened with Actions'+own `GITHUB_TOKEN`, which deliberately does not). The auto-merge gate will+therefore fire normally on bot PRs — but this must be verified on the first+real run, because it is the single point where the whole flow silently+stalls if it is wrong.++---++## 8. What actually changes, per repo++**`PersonalJarvis/marketplace`** (small, surgical)++1. `automerge_gate.py`: add the trusted-branch rule described above, keeping+ the fork rule untouched.+2. `validate.py` + `expand.py` + `schemas/submission.schema.json`: add the+ optional `publisher_id`; make ownership compare ids when present.+3. `README.md`: describe the two publishing routes honestly.++**Storefront repo** (the bulk of the work — all of it new)++4. `/marketplace` — browse, client-side fetch of the compiled `index.json`.+5. `/marketplace/submit` — the gated upload form: drag a `SKILL.md` in (or+ paste a `plugin.json`), validate the same rules the CI enforces *before*+ submitting, show a preview of the exact file that will be committed.+6. `/api/auth/github/{login,callback,logout}` and `/api/marketplace/submit`+ as Cloudflare functions.+7. A deliberate switch from a purely static build to static + functions.++**`PersonalJarvis/PersonalJarvis`** (optional, later — see §9)++8. A "Publish to marketplace" action in the Skills view.++---++## 9. The in-app publish path (why it is nearly free)++The desktop app should not use the website's login. It has a better one+already written: `oauth_device.py`. With device flow enabled on the same+GitHub App, publishing from inside Jarvis is: show the user code → user+approves in a browser → the app holds a user token in the existing keyring+`TokenStore` → the app POSTs the submission plus that token to the same+`/api/marketplace/submit` endpoint, which verifies the token belongs to our+App and takes it from there.++That keeps **one** publishing implementation and one identity, works+headless (device flow is designed for exactly that), and needs no client+secret in a binary that anyone can download — the reason the handler was+written this way in the first place.++---++## 10. Open decisions++| # | Decision | Options (lean in bold) |+|---|---|---|+| D11 | Identity model | **L2 GitHub App, identity-only consent**, L1 OAuth App + fork, L0 no login |+| D12 | Endpoint host | **Cloudflare (same vendor as the live site)**, Vercel, own VPS |+| D13 | Ownership key | **Numeric account id + login for display**, login only (status quo) |+| D14 | Anti-spam bar | **Turnstile + daily quota + minimum account age**, quota only, none |+| D15 | Storefront browse page | Ship with submit, or ship browse first and submit after |+| D16 | In-app publish | **Wave 2, after the web path proves out**, same wave, never |+| D17 | Domain of record | `personaljarvis.ai` is live; the Astro config still says `personaljarvis.dev` — fix before any callback URL is registered |++## 11. Suggested build order++1. **W1 — Registry hardening (no login involved).** `publisher_id`, the+ trusted-branch gate rule, README honesty. Independently valuable, and it+ closes the rename-hijack hole that exists today.+2. **W2 — Login.** GitHub App registered, Cloudflare functions for+ login/callback/logout, session cookie, `/marketplace/submit` behind the+ gate showing only "Sign in with GitHub" when signed out.+3. **W3 — Upload.** Client-side validation mirroring `validate.py`, the file+ preview, the submit endpoint, the bot PR, and the first end-to-end run+ that proves auto-merge fires on a bot PR.+4. **W4 — Browse page** on the storefront (pure static, reads the feed).+5. **W5 — In-app publish** via device flow against the same endpoint.diff --git a/docs/marketplace/github-signin-implementation.md b/docs/marketplace/github-signin-implementation.mdnew file mode 100644index 0000000000..1b59ce94e3--- /dev/null+++ b/docs/marketplace/github-signin-implementation.md@@ -0,0 +1,308 @@+# Sign in with GitHub — implementation spec++**Status:** BUILT — storefront half live on personaljarvis.ai, desktop half (§7)+shipped in the app on 2026-08-22 (`components/marketplace/PublishIdentity.tsx`) ·+**Written:** 2026-08-14 ·+**Decision:** [github-login-analysis.md](github-login-analysis.md) (L2 chosen) ·+**Registry:** [community-registry.md](community-registry.md)++This is the build sheet for the storefront's GitHub-only sign-in and the+gated upload behind it. It covers what to register at GitHub, how the+session survives a page reload without a consent banner, the endpoint+contracts, and the verification chain that makes a submission provably+belong to a GitHub account.++Scope note: while the registry repo is private, nobody outside the+organisation can open a submission pull request at all. The login is what+lets it be reopened safely.++---++## 1. What to register at GitHub++One **GitHub App**, owned by the `PersonalJarvis` organisation. Cost: €0.+Settings → Developer settings → GitHub Apps → New GitHub App.++| Field | Value | Note |+|---|---|---|+| GitHub App name | `Personal Jarvis Marketplace` | Appears on the consent screen — this is the "verify against our organisation" surface. |+| Homepage URL | `https://personaljarvis.ai` | |+| Callback URL #1 | `https://personaljarvis.ai/api/auth/github/callback` | Exact string match, no trailing slash. |+| Callback URL #2 | `http://localhost:4321/api/auth/github/callback` | Astro's dev port; GitHub Apps allow up to 10 callbacks, so no second app is needed for development. |+| Request user authorization (OAuth) during installation | **unchecked** | Users log in; they never install anything. |+| Enable Device Flow | **checked** | Required for the desktop-app publish path (§7). |+| Webhook → Active | **unchecked** | Nothing listens. |+| Where can this GitHub App be installed | **Only on this account** | |+| Repository permissions → Contents | **Read and write** | Push the submission branch. |+| Repository permissions → Pull requests | **Read and write** | Open the PR. |+| Account permissions | **all "No access"** | Public profile needs no permission — this is what keeps the consent screen empty. |++After creation:++1. **Generate a client secret** → `GITHUB_APP_CLIENT_SECRET`.+2. **Generate a private key** (PEM download) → `GITHUB_APP_PRIVATE_KEY`.+3. Note the **App ID** and **Client ID**.+4. **Install the App** on `PersonalJarvis/marketplace` only. Note the+ resulting **installation id** (visible in the installation URL) —+ caching it avoids one API call per submission.++Secrets live **only** in the Cloudflare project's environment (encrypted).+Never in a repo, never in `jarvis.toml`, never in the client bundle. The+private key is multi-line: store it base64-encoded to survive env-var+handling.++Environment variables the backend expects:++```+GITHUB_APP_ID+GITHUB_APP_CLIENT_ID+GITHUB_APP_CLIENT_SECRET+GITHUB_APP_PRIVATE_KEY_B64+GITHUB_APP_INSTALLATION_ID+REGISTRY_REPO=PersonalJarvis/marketplace+SESSION_SIGNING_KEY # 32 random bytes, base64+TURNSTILE_SECRET_KEY # anti-spam, §6+```++---++## 2. Where the code runs++The storefront is a static Astro build served through Cloudflare. Add+**Cloudflare Pages Functions** beside it — `functions/api/**` — so the+endpoints answer on the *same origin* as the site.++Same origin matters for three reasons: the session cookie is first-party,+there is no CORS preflight, and no third-party-cookie policy in any browser+can touch it. Do not put the endpoints on a separate `api.` subdomain.++**Cloudflare trap:** every response that sets or reads the session cookie+must carry `Cache-Control: private, no-store`. A cached `Set-Cookie` at the+edge would hand one visitor's session to the next.++---++## 3. The session: no banner, survives reload++### 3.1 The decision++Use a **signed, stateless, `httpOnly` session cookie**. No database, no+server-side session table, no consent banner.++Two facts settle this, and they run against the common assumption that+"cookie = banner, therefore use localStorage":++* **A login session is consent-free.** § 25 (2) TDDDG (and ePrivacy Art.+ 5(3)) exempts storage that is *strictly necessary for a service the user+ explicitly requested*. A session that exists only because the user clicked+ "Sign in" is the textbook case, next to shopping carts and language+ choice. Analytics and marketing are what need the banner — we ship+ neither on this page. *(Engineering summary, not legal advice.)*+* **localStorage is governed by the same rule.** § 25 covers *any* storing+ of or access to information on the user's device — the statute never says+ "cookie". Moving the token to localStorage changes nothing legally while+ making it readable by any injected script. An `httpOnly` cookie is+ invisible to JavaScript, so an XSS bug cannot steal the session.++So the cookie is both the safer and the cheaper option; nothing is gained by+avoiding it.++### 3.2 Cookie shape++```+Name: pj_session+Value: base64url(payload) "." base64url(HMAC-SHA256(payload, SESSION_SIGNING_KEY))+Payload: {"uid":<numeric GitHub id>,"login":"<login>","av":"<avatar url>",+ "iat":<unix>,"exp":<unix>}+Attributes: HttpOnly; Secure; SameSite=Lax; Path=/; Max-Age=2592000 (30 days)+```++* **`uid` is the numeric GitHub account id and is the only identity key.**+ Logins can be renamed and the freed name re-registered by someone else;+ the numeric id never changes. `login` rides along for display only.+* The cookie carries no token and no email — nothing worth stealing, and+ nothing that needs a privacy notice beyond the existing one.+* `SameSite=Lax` is required, not cosmetic: `Strict` would withhold the+ cookie on the top-level redirect *back from github.com*, and the login+ would appear to fail on the last step.++### 3.3 What happens on reload (the actual answer to "stay signed in")++1. Browser re-requests the page and automatically attaches `pj_session`.+2. The submit page calls `GET /api/auth/session`.+3. The function verifies the HMAC and `exp`, and answers+ `{"login": "...", "id": 123, "avatar": "..."}` or `401`.+4. On `401` the page renders the "Sign in with GitHub" button; otherwise it+ renders the upload form with the user's handle.++That is the whole persistence mechanism: no refresh-token dance, because+after the identity check the GitHub token is discarded — there is nothing to+refresh. When the 30 days elapse, the user signs in again, which is one+click and no re-consent (GitHub remembers the authorisation).++**Renewal:** if a valid cookie is older than 7 days, re-issue it with a+fresh `exp` on the same response. Active users are never logged out;+abandoned sessions still die on schedule.++---++## 4. Endpoint contracts++### `GET /api/auth/github/login`+Generates a 32-byte random `state`, stores it in a short-lived cookie+(`pj_oauth_state`, `HttpOnly; Secure; SameSite=Lax; Max-Age=600`), and+redirects (302) to:++```+https://github.com/login/oauth/authorize+ ?client_id=<GITHUB_APP_CLIENT_ID>+ &redirect_uri=<exact callback URL>+ &state=<nonce>+```++Accepts an optional `?next=/marketplace/submit` (validated as a same-site+relative path) and carries it inside the state cookie.++### `GET /api/auth/github/callback`+1. Compare `?state` with the state cookie; mismatch or missing → `400`,+ clear the cookie. (This is the CSRF gate — never skip it.)+2. `POST https://github.com/login/oauth/access_token` with `client_id`,+ `client_secret`, `code`, `redirect_uri`, `Accept: application/json`.+ **This is the step a browser cannot do** (GitHub sends no CORS headers),+ and the reason this function exists at all.+3. `GET https://api.github.com/user` with the token → `id`, `login`,+ `avatar_url`, `created_at`.+4. Reject accounts younger than the configured minimum age (§6).+5. **Discard the GitHub token.** Do not store it, do not send it to the+ browser.+6. Set `pj_session`, clear `pj_oauth_state`, redirect to `next`.++### `GET /api/auth/session`+Returns the identity from the cookie, or `401`. Never touches GitHub.++### `POST /api/auth/logout`+Clears the cookie (`Max-Age=0`), returns `204`.++### `POST /api/marketplace/submit`+The gated upload. Requires a valid `pj_session` — no session, no upload,+which is the whole point of the feature.++```+Request: { kind: "skill" | "plugin", name, version, title?, description?,+ categories?, skill_md? | plugin_json?, mcp_json?, usage_card?,+ turnstileToken }+Response: 201 { prUrl, submissionPath } | 4xx { error, field? }+```++Server-side sequence:++1. Verify the session cookie; `401` if absent or expired.+2. Verify the Turnstile token; `403` on failure.+3. Enforce the per-account quota (§6); `429` when exceeded.+4. **Re-validate everything** — the same rules `scripts/validate.py`+ enforces in CI: name pattern, reserved names, https-only, launcher+ allowlist with pinned versions, secret scan, 64 KB `skill_md`,+ 128 KB file, 500-char description. Client-side checks are UX only; this+ is the boundary that counts.+5. Build the submission object and **set `publisher` and `publisher_id`+ from the session, never from the request body.** A client-supplied+ publisher is ignored outright.+6. Check ownership against `registry.json` on `main`: if the name exists+ and its `publisher_id` differs → `409` with a plain-language message. If+ it matches, require a strictly higher version.+7. Mint an installation token (JWT signed with the App private key →+ `POST /app/installations/{id}/access_tokens`), then, in the registry+ repo: create branch `submit/<name>-<version>` off `main`, PUT+ `submissions/<name>.json`, open the PR with the publisher recorded in+ the body.+8. Return the PR URL. The existing CI takes over from there.++---++## 5. The verification chain++What makes a submission provably belong to a GitHub account, end to end:++1. GitHub authenticated the human — we never see a password.+2. The token was exchanged with **our** client secret, so the identity came+ back from GitHub to us and to nobody else.+3. `publisher_id` is copied from the session on the server; the browser has+ no way to influence it.+4. The branch was pushed with **our App's installation token** — the only+ credential on earth with write access to that private repo besides the+ maintainer.+5. The merge gate therefore trusts the file's publisher **only** when the PR+ comes from a branch inside the registry repo and was opened by our App's+ bot. Fork PRs keep the existing rule (`publisher` == PR author).++Two extra defences worth building in from the start:++* **Token provenance check** (needed for the desktop path in §7): when a+ token arrives from a client rather than from our own callback, verify it+ with `POST /applications/{client_id}/token` under the App's basic auth.+ That endpoint answers 200 only for tokens minted for *our* client id — it+ is what stops someone presenting a token from an unrelated OAuth app.+* **Gate-fires check:** pull requests opened with an App installation token+ *do* trigger Actions workflows, unlike ones opened with Actions' own+ `GITHUB_TOKEN`. Verify on the first live run — a silent non-trigger here+ would stall every submission at "open, never merged".++### Optional later: organisation-verified publishers++Publishing under an *organisation's* name (rather than a personal login)+needs proof of membership, which the GitHub App cannot see: `GET /user/orgs`+with a GitHub App user token lists only organisations where the App is+installed. If org namespaces are ever wanted, the choice is a separate OAuth+App with `read:org`, or a manually curated allowlist of verified publishers.+Not needed for v1, where every publisher is a personal account.++---++## 6. Anti-abuse++A GitHub account is free, so the login is an identity, not a filter. Three+cheap layers do the actual work:++| Layer | Setting | Rationale |+|---|---|---|+| Account age | Reject accounts younger than **7 days** (`created_at` from `GET /user`) | Kills same-day throwaway accounts at near-zero false-positive cost. |+| Quota | Max **5 submissions per account per 24 h** | Counted from open PRs by that publisher; no storage needed. |+| Challenge | **Cloudflare Turnstile** on the submit endpoint | Free, same vendor, no puzzle for real users. |++Everything above is *in addition to* `validate.py`, the unreviewed badge,+and the install-time consent dialog — none of which the login replaces.++---++## 7. Desktop-app publishing (later wave)++The app must not reuse the website's cookie. It uses the **device flow** on+the same GitHub App — the handler already exists at+`jarvis/marketplace/auth/oauth_device.py` (start, poll, refresh, all four+GitHub error codes), which is why enabling device flow in §1 matters.++Flow: app shows the user code → user approves in a browser → token lands in+the existing keyring `TokenStore` → the app POSTs the submission plus that+token to `/api/marketplace/submit` (as `Authorization: Bearer`, no cookie)+→ the endpoint runs the token-provenance check from §5 and proceeds+identically.++One publishing implementation, one identity, and no client secret inside a+binary anyone can download.++---++## 8. Build order and proof++1. **Registry:** add `publisher_id` to the schema, `validate.py`,+ `expand.py`, and the ownership comparison; add the trusted-branch rule to+ `automerge_gate.py`. *Proof: a test submission with a mismatched id is+ rejected.*+2. **App registration** per §1. *Proof: the consent screen lists no+ permissions.*+3. **Auth endpoints** (login, callback, session, logout). *Proof: sign in,+ hard-reload the page, still signed in; delete the cookie, signed out.*+4. **Submit endpoint** + form. *Proof: a skill uploaded from the browser+ appears as a bot PR, auto-merges, and shows up in `index.json`.*+5. **Reopen the registry repo** to the public once 1–4 are green.+6. **Browse page**, then the in-app publish path (§7).diff --git a/docs/marketplace/install-standard.md b/docs/marketplace/install-standard.mdnew file mode 100644index 0000000000..0414d6956a--- /dev/null+++ b/docs/marketplace/install-standard.md@@ -0,0 +1,60 @@+# The marketplace install standard++Status: shipped 2026-08-14. Consumer half of the store; the publishing half+lives in [publishing-plan.md](publishing-plan.md).++Every installable community entry — plugin or skill — is offered the same+three ways on its detail page, the pattern comparable stores (ClawHub,+Smithery, LobeHub) converged on, adapted from the npm world to a Python app:++| Surface | Line | Who types it |+| -------- | ------------------------------------------------------------- | ------------ |+| `cli` | `jarvis marketplace install <name>` | a terminal with the CLI installed |+| `runner` | `uvx --from personal-jarvis jarvis marketplace install <name>` | a terminal WITHOUT the CLI — `uvx` resolves the published package into a throwaway environment first |+| `prompt` | `Install the "<name>" plugin from the community marketplace.` | pasted at the assistant, which runs the curated CLI command through its CLI tool |++## The one-source rule++The three strings are computed in **one** place —+`jarvis/marketplace/install_standard.py` — and flow outward verbatim:++1. `/api/marketplace/community` embeds an `install` block per entry+ (`marketplace_routes.py::_community_payload`).+2. The store's detail sheets render that block verbatim+ (`PluginsCommunity.tsx::InstallStandard`) — the frontend never derives+ command strings itself.+3. The curated command (`jarvis/cli_ctl/commands/marketplace.py`) is the only+ consumer-side implementation. Renaming or reshaping it requires the same+ change in `install_standard.py`, or the store advertises a command that+ does not exist — the one bug a store is judged on.++Entries that cannot install get **no block** instead of a failing command:+skills without a `raw_url` (manual installs) and plugins whose name collides+with a built-in id.++## Resolution semantics of `jarvis marketplace install <name>`++One name, no flags, kind inferred — the CLI resolves against the community+index: plugin first, then skill (a future registry enforces one namespace, so+the tiebreak is transitional). `--refresh` re-fetches the index first.+Before the confirm gate the command prints the SAME facts the store's consent+dialog shows — publisher, hosted URL or local argv — because community+content is unreviewed; `--yes` is the CLI's version of clicking Install.+`jarvis marketplace uninstall <name>` mirrors the store's Remove buttons.++## Why `uvx` is the npx equivalent++`npx` runs an npm package's binary without installing it; `uvx` does exactly+that for a PyPI package's entry point. The runner line therefore works on any+OS with `uv` present and no prior install — it fetches the published+`personal-jarvis` package and executes the same curated command against the+locally running app. It deliberately does NOT bypass the app: installs always+go through the running instance's REST routes, so consent, conflict checks+and rollback behave identically on every surface.++## Why the prompt works end to end++The assistant has no dedicated install tool, on purpose: its CLI tool runs+`jarvis marketplace install <name>`, which carries the confirm gate and the+danger metadata every curated mutation has. One command, one safety model,+three surfaces.diff --git a/docs/marketplace/package-layout.md b/docs/marketplace/package-layout.mdnew file mode 100644index 0000000000..b559766e05--- /dev/null+++ b/docs/marketplace/package-layout.md@@ -0,0 +1,178 @@+# What you upload — the two shapes++**For authors.** This is the reference the publish form, the CLI scaffold,+and the registry README all point at. The packaging policy is+[agent-plugins-standard.md](agent-plugins-standard.md); how a submission+travels is [community-registry.md](community-registry.md).++You publish one of two things. Pick by what you have, not by what sounds+bigger:++| | A **skill** | A **plugin** |+|---|---|---|+| What it is | Instructions for the model | A connection to a service, optionally with instructions |+| What it is made of | One Markdown file | A directory: manifest, MCP server, skills |+| Needs an account or token? | No | Usually yes |+| Runs code? | No | The MCP server does, on its own host or as a pinned launcher |++---++## A skill — one file++```+todo-triage/+└── SKILL.md+```++That is the whole thing. `SKILL.md` is Markdown with a YAML frontmatter+header on top:++```markdown+---+schema_version: "1"+name: todo-triage+description: Sort an inbox of tasks into today, this week, and later.+when_to_use: Use when the user asks what to work on next, or to tidy a task list.+category: productivity+---++Group open tasks by due date before answering. Say the count for each+group, then name at most three tasks in "today"...+```++`name` must match the directory name, and it is the name the store lists.+`description` and `when_to_use` are what the model reads to decide whether+the skill applies — write them for a reader who has never seen your work.++---++## A plugin — a directory++```+todo-fox/+├── plugin.json ← required: identity + the Jarvis block+├── mcp.json ← the server your plugin talks to+├── skills/ ← optional: instructions that ship with it+│ ├── todo-triage/+│ │ └── SKILL.md+│ └── todo-weekly-review/+│ └── SKILL.md+├── io.github.personaljarvis/ ← everything specific to this client+│ └── usage-card.md+└── LICENSE+```++Only `plugin.json` is required by the standard. In practice a marketplace+plugin has `mcp.json` too, because that is what gives it tools; `skills/`+is what makes it *good* — the tools plus the instructions for using them+well, in one install.++**`plugin.json`** — the identity. Everything Jarvis-specific sits inside the+`io.github.personaljarvis` namespace, which every other client ignores:++```json+{+ "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",+ "name": "todo-fox",+ "description": "Tasks and reminders from TodoFox",+ "version": "1.2.0",+ "license": "MIT",+ "extensions": {+ "io.github.personaljarvis": {+ "display_name": "TodoFox",+ "category": "Lists & Tasks",+ "logo_slug": "todofox",+ "auth": {+ "mode": "hosted_mcp_oauth_dcr",+ "discovery_url": "https://todofox.example/.well-known/oauth-authorization-server",+ "mcp_url": "https://mcp.todofox.example/mcp",+ "refresh_supported": true+ }+ }+ }+}+```++**`mcp.json`** — exactly one server, either hosted over HTTPS:++```json+{+ "$schema": "https://agent-plugins.org/schemas/1.0.0/mcp.schema.json",+ "mcpServers": {+ "todo-fox": {+ "type": "streamable-http",+ "url": "https://mcp.todofox.example/mcp"+ }+ }+}+```++…or launched locally, from `npx` / `uvx` / `docker` only, at a pinned+version, with tokens as placeholders and never as literals:++```json+{+ "mcpServers": {+ "todo-fox": {+ "type": "stdio",+ "command": "npx",+ "args": ["-y", "@todofox/[email protected]"],+ "env": { "TODOFOX_TOKEN": "$plugin_todo-fox_access_token" }+ }+ }+}+```++**`io.github.personaljarvis/usage-card.md`** — the words that decide *when*+your plugin is offered. Without it, Jarvis only reaches for the plugin when+a turn names it outright:++```markdown+---+keywords: todo, task, reminder, due date, checklist, todofox+---+Use the TodoFox tools when the user asks what is due, wants to add or+complete a task, or asks to plan a day. Prefer reading over writing unless+the user clearly asks to change something.+```++---++## Naming, versioning, ownership++- **Names** are 1–64 characters, lowercase `a-z 0-9 - .` only, starting and+ ending alphanumeric, no `--` or `..`. **No underscores** — `todo_fox` is+ invalid, `todo-fox` is right.+- **Versions** are SemVer and must increase on every update.+- The first accepted submission of a name **claims it**. Updates are+ accepted only from the same GitHub account.++## What is not accepted++| | Why |+|---|---|+| `scripts/` inside a skill | Executable code from an unreviewed author. Instructions and reference text only. |+| A skill that sets `risk_policy` | It decides which tools run without asking you. The built-in default applies instead. |+| `plugin.json` binding a `native_tool` | Native tools are Python inside the app; that tier is contributed to the main repo. |+| Plain `http://` anywhere | Your users' tokens travel over these URLs. |+| A stdio launcher other than `npx`/`uvx`/`docker`, or an unpinned version | Anything else is an arbitrary command line on a stranger's machine. |+| A token written into `headers` or `env` | Those files are public. Use a `$plugin_…` placeholder; Jarvis fills it at connect time. |+| A package with no components at all | It would be a card that collects a token and does nothing. |++Limits: 10 skills per package, 64 KB per `SKILL.md`.++## What happens after you upload++```+your directory ──► one submission file ──► automated checks ──► auto-merge+ │+ plugins/<name>/… (the real Agent Plugins package)+ │+ index.json ──► the store, in the app and on the web+```++Nothing is reviewed by a person. The checks stop credential smuggling,+plaintext endpoints, unpinned code and name hijacking — they cannot judge+whether your service is trustworthy. Every community entry is therefore+badged as unreviewed, and users see your exact MCP URL or launch command+before they install.diff --git a/docs/marketplace/publishing-plan.md b/docs/marketplace/publishing-plan.mdnew file mode 100644index 0000000000..894483d933--- /dev/null+++ b/docs/marketplace/publishing-plan.md@@ -0,0 +1,271 @@+# Publishing — how someone gets a plugin INTO the marketplace++**Status:** SHIPPED in-app on 2026-08-22 — the Marketplace section's Publish Studio+(`components/marketplace/PublishStudio.tsx`, routes in `marketplace_publish_routes.py`);+this document remains the design record · **Written:** 2026-08-14 ·+**Consumes:** [community-registry.md](community-registry.md) (the shipped+consumer half) · [agent-plugins-standard.md](agent-plugins-standard.md)+(the packaging policy) · [public-marketplace-analysis.md](public-marketplace-analysis.md)+(the decisions that led there)++The registry is live and the app installs from it. What is missing is the+door: **there is no way to publish that a normal person can walk through.**+This document decides the package standard, the upload paths, and the order+we build them.++---++## 1. Where we actually stand++| Half | State |+|---|---|+| Consume — fetch index, browse, consent, install, connect | **Shipped** 2026-08-12 |+| Registry — CI validation, auto-merge, ownership ledger, Pages index | **Shipped** 2026-08-12 |+| Publish — any interface a human uses to submit | **Missing entirely** |++Three concrete gaps, in order of how much they hurt:++1. **A broken public promise.** `PersonalJarvis/marketplace/README.md` tells+ every visitor to "use the form at <https://personaljarvis.ai/marketplace/submit>".+ That storefront repo does not exist. Today the only real path is: hand-write+ `submissions/<name>.json`, fork, open a PR. Nobody outside this project+ will do that.+2. **One file per submission.** `scripts/automerge_gate.py` matches+ `^submissions/[a-z0-9][a-z0-9.-]*\.json$` and requires the PR to change+ *exactly one* file. Bundled skills fit inside that file, so this is no+ longer a blocker for the standard — but a skill with `references/`, or+ any package that genuinely needs several files, still cannot auto-merge.+3. **Plugin and skill were split.** Our submission schema has+ `kind: "plugin" | "skill"` as mutually exclusive, and Agent Plugins+ v1.0.0 says a plugin *contains* skills and MCP servers. **Closed+ 2026-08-14** on the client side: the two upload doors stay, but a plugin+ package may now bundle skills (§3).++---++## 2. What ClawHub does, and what we take from it++ClawHub's "Add a skill or plugin" screen makes four choices visible:++| ClawHub | Take it? |+|---|---|+| Publisher context first (personal account or org), changeable later | **Yes** — but personal-only in v1 (see §7, D3) |+| A `Skill` / `Plugin` toggle before anything else | **No** — see §3, this is the split we are removing |+| Source type: **Code plugin** (`package.json` + manifest, executable) | **No** — executable community code stays out of scope |+| Source type: **Bundle plugin** (`.zip` / `.tgz` archive upload) | **Partly** — accept a folder or zip as *input*, but the bytes land in git, not in blob storage |++The important divergence: ClawHub asks the author to classify their work+**up front**, then validates. We invert it — **the folder is the+classification**. The author submits an Agent Plugins directory; what is+inside it decides what kind of card it becomes. Fewer wrong turns, and it+matches the spec, where components are optional and additive.++The second divergence: ClawHub hosts archives. We keep Model A1 — *the+reviewed bytes are the shipped bytes*. A submission's files live in the+registry repo, so there is no archive to fetch at install time, no hash to+verify, no second host to trust, and delisting is a `git revert`.++---++## 3. The standard: two upload shapes, one package format++**Decision (maintainer, 2026-08-14): the two upload shapes stay.** You+publish *either* a skill *or* a plugin, because a skill genuinely is just a+Markdown file and asking its author to wrap it in a manifest is friction for+nothing. What changes is the plugin side: a plugin package may now **bundle+skills**, which is the combination Agent Plugins v1.0.0 exists for. The+author-facing reference is [package-layout.md](package-layout.md).++The submission file keeps `kind: "plugin" | "skill"` and stays ONE file per+PR — a bundled skill travels inside the plugin submission as+`skills: [{name, skill_md}]`, so the auto-merge gate needs no change for+this. Multi-file packages (a skill with `references/`) remain blocked by+the gate and are deliberately out of scope until someone needs them.++**Shipped 2026-08-14** (`convert_package` + `bundled_skills.py`): the loader+validates and installs bundled skills, ownership markers keep a plugin from+overwriting or deleting a skill it does not own, `$schema` is enforced, and+a package with no working component is rejected.++Below is the package layout underneath both shapes.++```+submissions/<name>/+├── plugin.json # REQUIRED — $schema, name, description, version, license+├── mcp.json # optional — exactly one server+├── skills/<skill-name>/SKILL.md # optional, repeatable — instructions+│ └── references/*.md # optional — text the skill may read+└── io.github.personaljarvis/+ ├── usage-card.md # optional — relevance keywords + guidance+ └── logo.svg # optional — square mark+```++Within the plugin shape, what the author puts in decides the rest — the+store labels it, the author never sub-classifies it (this is where ClawHub+asks up front and we do not):++| Shape | Contains | Carries code? | Community |+|---|---|---|---|+| **Connector** | `plugin.json` + `mcp.json` + auth extension | No (endpoint or pinned launcher) | Allowed — this is what ships today |+| **Bundle** | connector + `skills/` | No | Allowed — the shape the spec exists for |+| **Skill** | one `SKILL.md`, uploaded as a skill | No (Markdown only) | Allowed — needs no manifest, no auth, no connect flow |+| **Native** | `extensions[…].native_tool`, Python entry point | **Yes** | **Blocked** — repo-contributed only, already rejected by `agent_plugins_loader.py` |++Four rules that keep the "no foreign code, no self-granted privilege" line+intact — all four enforced in `agent_plugins_loader.py` as of 2026-08-14:++1. **No `scripts/` in community skills.** The spec allows a skill to ship+ executable scripts; the index embeds only `SKILL.md`, and the registry+ rejects the directory on submission. Repo-contributed skills keep+ `scripts/` because they pass human review.+2. **No self-declared `risk_policy`.** `skills/runner.py` evaluates a+ skill's tools against the SKILL'S declared tier rather than the tool's+ own — an auto-merged author could otherwise mark a tool `safe` and skip+ its confirmation.+3. **stdio stays allowlisted and pinned** — `npx` / `uvx` / `docker`, exact+ versions, no `@latest`. Unchanged from today, enforced in both places.+4. **Every package carries a working component.** No MCP server, no hosted+ auth URL and no skills means a card that collects a token and does+ nothing.++**Why the bundle shape is worth having.** A bundle written for us runs+unchanged in ChatGPT, Cursor, Copilot, Kiro, and VS Code, and theirs run+here — they read `skills/` and `mcp.json` and ignore our extension+namespace, exactly as the spec intends. ClawHub's hard split gives that up+on the plugin side; we keep the friendly two-door upload AND the portable+package underneath.++**Back-compat is free.** `CommunityIndex` is already `extra="allow"`, so the+new `skills` block on a plugin entry is simply ignored by an older app,+which keeps installing the connector alone. The compiled `index.json` goes+on emitting both `plugins[]` and `skills[]`.++---++## 4. The upload paths, in build order++### Path 1 — `jarvis plugin …` (the CLI) · **Recommended first**++The contract already demands it (CLAUDE.md §5: a feature is a REST route+plus an auto-CLI), the audience already has Jarvis installed, and it costs+zero infrastructure.++```+jarvis plugin init <name> # scaffold the directory, ask the 6 questions+jarvis plugin validate [path] # the exact rules CI and the client apply+jarvis plugin publish [path] # fork + branch + commit + PR, via gh+jarvis plugin status <name> # PR checks, merge state, live in the index?+```++`publish` shells out to `gh` — already a hard dependency of our GitHub+doctrine, already authenticated on a contributor's machine, and it makes the+PR author identical to the publisher, which is what the auto-merge gate+checks. No OAuth app, no server, no secret to hold.++Why first: it is the smallest thing that turns "hand-write JSON" into "one+command", and every later surface calls the same validator underneath.++### Path 2 — Plugins → **Publish** (in-app)++The ClawHub screen, rebuilt inside the desktop app: publisher line, a form+for the six fields, drag a folder or zip onto it, live validation with the+real rule set, a diff-style preview of what the store card will look like,+then one button that runs the same `gh` flow. For contributors who do not+live in a terminal.++Needs a GitHub identity in the app. Use `gh auth token` when the CLI is+present; fall back to the GitHub **device flow** (no client secret, works+headless, prints a code the user types on github.com) — never ask for a PAT+in a text field.++### Path 3 — the storefront form (`personaljarvis.ai/marketplace/submit`)++The only path that reaches someone who has *not* installed Jarvis, and the+one the README already promises. **In build since 2026-08-14** in a parallel+effort — see [github-signin-implementation.md](github-signin-implementation.md)+(GitHub App, cookie session, Cloudflare Pages Functions, bot opens the PR)+and [github-login-analysis.md](github-login-analysis.md) for the decision.+The registry repo is private until that gate is green, so no submission path+is open at all right now. Nothing in this plan duplicates that work: it+supplies the *format* the form will upload and the *validator* it should+call.++---++## 5. One validator, three callers++Today the rules exist twice: `scripts/validate.py` in the registry and+`agent_plugins_loader.py` in the client, kept in sync by hand. A publish CLI+would make it three copies and guarantee drift.++**Decision: the client loader is the single implementation.** Extract the+rule set into `jarvis/marketplace/submission_rules.py`; the registry's CI+job installs the published `personal-jarvis` package and calls it. The+client keeps re-validating at install time — that is not duplication, it is+the deliberate defence against a poisoned index — but it runs *the same+code*, so "CI green" and "the app will accept this" can no longer disagree.++Pin the package version in the workflow and bump it deliberately, so a bad+release cannot silently change what the registry accepts.++---++## 6. Waves++**W1 — Client-side bundles. DONE 2026-08-14.** `convert_package` validates+a package's skills; `bundled_skills.py` writes them under the user's skills+root behind an ownership marker; install rolls back on a name conflict and+uninstall takes only what the plugin owns. `$schema` and the+working-component rule land with it. 10 new tests.++**W2 — Registry side of the same format.** Submission schema gains+`skills: [{name, skill_md}]` for `kind: "plugin"`; `validate.py` applies the+four §3 rules; `expand.py` writes a real `skills/<name>/SKILL.md` into the+package directory; `build_index.py` embeds them. Gate untouched — it is+still one file. *Done when:* a bundle submission auto-merges and the app+installs card plus skill from the live index.++**W3 — One validator.** Extract `submission_rules.py`, point the registry CI+at the installed package, delete the duplicated rules. *Done when:* one rule+change lands in one file and both sides move together, proven by a test that+feeds the same fixtures to both entry points.++**W4 — The CLI.** `init` / `validate` / `publish` / `status`, plus a+`--dry-run` that prints the PR body it would open. *Done when:* a fresh+clone on Windows, macOS, and Linux publishes a bundle end to end with no+hand-written JSON.++**W5 — In-app Publish view.** The form, drop target, live validation,+card preview, device-flow auth (the GitHub App already enables Device Flow+for exactly this — signin spec §7).++**W6 — Storefront upload.** Being built in parallel as the GitHub sign-in+gate; this plan does not duplicate it.++**Multi-file packages** (a skill with `references/`) stay out of scope until+someone asks: they need the gate widened to `submissions/<name>/**` with+path-traversal and size caps, which is a trust change, not a format one.++---++## 7. Open decisions for the maintainer++**D1 — Org publishing.** ClawHub lets you publish as an org. Our gate+compares `publisher` to the PR author, so an org login would never+auto-merge. Supporting it means an extra membership check in the gate.+*Recommended:* personal logins in v1, org support in a later wave — it is a+gate change, not a format change, so it costs nothing to defer.++**D2 — Verified publishers.** Nothing today distinguishes a first-time+account from a known one. A cheap version: badge publishers whose GitHub+account is older than N months or who own a merged package that has survived+M weeks. *Recommended:* defer until there is a listing worth impersonating.++**D3 — Package size cap.** Once `references/` is allowed, the registry repo+grows with every submission and it is the CDN. *Recommended:* 256 KB per+package and 25 files, generous for text and small enough that Pages stays+fast.++**D4 — Delisting on report.** Today: open an issue, maintainer reverts.+*Recommended:* keep it manual — an automated takedown is itself an attack+surface, and revert-plus-redeploy is already minutes.diff --git a/docs/marketplace/validator-parity.md b/docs/marketplace/validator-parity.mdnew file mode 100644index 0000000000..93e368ebbe--- /dev/null+++ b/docs/marketplace/validator-parity.md@@ -0,0 +1,138 @@+# Validator parity — the upload endpoint vs. the registry CI++**Status:** FINDINGS — measured 2026-08-14 ·+**Authority:** `scripts/validate.py` in `PersonalJarvis/marketplace` ·+**Compared against:** `functions/_lib/validate.ts` in the storefront ·+**Related:** [github-signin-implementation.md](github-signin-implementation.md) §4++## Why this file exists++The submit endpoint validates a submission, then opens a pull request that+CI validates again. When the endpoint is *stricter*, an author sees a clear+field error and fixes it — no harm. When the endpoint is *looser*, the+author gets `201 {prUrl}` and a success message, and the pull request then+fails CI and sits open forever. Nobody is watching that pull request: the+author thinks they published, and the maintainer sees a queue of red+strangers.++So the direction of a divergence decides whether it matters. Everything+below is a case where the endpoint accepts what CI rejects.++This is **not** a security hole. `validate.ts` says so itself, correctly:+CI remains the final authority and nothing bad merges. It is a usability+hole that produces orphaned pull requests — with one partial exception,+noted at the end.++## Divergences that produce an orphaned pull request++| # | Rule CI enforces | Where | Endpoint today |+|---|---|---|---|+| 1 | Reserved plugin ids (25 names incl. `github`, `slack`, `stripe`) | `RESERVED_PLUGIN_IDS` | not checked |+| 2 | Reserved skill names (32 incl. every `plugin-*`) | `RESERVED_SKILL_NAMES` | not checked |+| 3 | `skill_md` frontmatter must declare the same `name:` as the submission | `validate_skill` | not checked |+| 4 | `plugin_json.$schema` must equal the Agent Plugins v1.0.0 URL exactly | `validate_plugin` | not checked |+| 5 | `plugin_json.name` must equal the submission name | `validate_plugin` | not checked |+| 6 | `plugin_json.description` is **required** (not just length-capped) | `validate_plugin` | length checked only when present |+| 7 | `extensions["io.github.personaljarvis"]` is required | `validate_plugin` | not checked |+| 8 | `native_tool` is refused (repo-contributed code, not publishable) | `validate_plugin` | not checked |+| 9 | `auth.mode` must be one of the five known modes | `AUTH_MODES` | not checked |+| 10 | `mcp_auth_header_template` needs a `$plugin_…` placeholder and may embed no literal credential | `validate_plugin` | not checked |+| 11 | **No `http://` URL anywhere in the document**, recursively | `reject_http_urls` | only the MCP server url is checked |+| 12 | `headers` on a streamable-http server are refused | `validate_mcp_json` | not checked |+| 13 | Every `env` value must be a `$plugin_…` placeholder | `validate_mcp_json` | not checked |+| 14 | **No** argument may end in `@latest` | `validate_mcp_json` | passes if *any* argument looks pinned, so `npx [email protected] bar@latest` is accepted |+| 15 | `sse` transport is refused by name | `validate_mcp_json` | accepted — an sse server has a `url`, so it takes the hosted branch |+| 16 | Only `mcpServers` is read | `validate_mcp_json` | also accepts `servers` as an alias, which CI then cannot find |+| 17 | Size is measured on the **final file**: pretty-printed, `publisher` and `publisher_id` included | `validate_file` | measured on compact `JSON.stringify` of the value *before* the publisher fields are added |++Number 17 is the quiet one. Indentation on a nested manifest adds real+weight, so a submission that measures just under 128 KB at the endpoint can+cross the limit once written to disk — and it only bites the largest+submissions, which are also the ones an author least wants to redo.++Numbers 1 and 2 are the likeliest to be hit in practice: `github`,+`notion`, `slack` and `stripe` are exactly the names a newcomer reaches for.++## The one with a security edge++`SECRET_PATTERNS` has nine entries in CI and five at the endpoint. Missing:+`github_pat_…`, Google `AIza…` keys, JWTs, and `sk-` keys containing `-` or+`_`. A submission carrying one of those is accepted, and the App **commits+it to a branch** before CI ever runs. The pull request will fail and never+merge, but the credential is in the repository's history from that moment.++The registry repo is private today, which contains the blast radius. It is+scheduled to be reopened once the login path is green (spec §8 step 5), so+this should close before that happens.++## The fix that stops it recurring++`rules.json` is now generated from `validate.py` and published next to the+feed:++```+https://personaljarvis.github.io/marketplace/rules.json+```++It carries the data-driven half — limits, patterns, reserved names, the+launcher and auth allowlists, all nine secret patterns, the plugin schema+URL — with patterns written so Python `re` and ECMAScript both accept them+verbatim. The registry's validate workflow fails if the file drifts from+the validator, so it cannot go stale unnoticed.++Reading that file closes 1, 2, 4, 9 and the secret-pattern gap outright,+and gives 3, 5–8, 10–16 their constants. Those remaining ones are logic+rather than data and still need writing once against the table above.++Number 17 needs no shared data at all: measure the bytes of the exact JSON+the App is about to commit — pretty-printed, publisher fields included —+rather than the payload as received.++## Closed in `jarvis/marketplace/publish.py` (2026-08-15)++| # | Fix |+|---|---|+| 14 | `_validate_mcp` now rejects an `@latest`-suffixed stdio arg on its own, even when a different arg in the same command looks pinned. Previously `npx [email protected] bar@latest` passed because the check only asked "does *any* arg look pinned". |+| 16 | `_validate_mcp` reads only `mcpServers`; the undocumented `servers` alias (which CI has never recognized) is no longer accepted. |++Numbers 1–13 and 15 were left open: closing them well requires either data+this desktop app cannot fetch locally (`RESERVED_PLUGIN_IDS` /+`RESERVED_SKILL_NAMES` — see `rules.json` above, not yet consumed here) or a+change to the `mcp.json` shape this app's Publish form accepts. See+"the mcp.json `type` field gap" below — several of the remaining rows (9,+11 partial, 12, 13, 15, and would-be row 16's structural cousin) are+enforced correctly by `agent_plugins_loader.convert_package`, but that+function REQUIRES a `type: "streamable-http" | "stdio" | "sse"` key per+server (`docs/marketplace/agent-plugins-standard.md` line 21) that neither+`publish.py`'s `_validate_mcp` nor its own tests ever populate — the two+would need to agree on the mcp.json shape before delegation is safe. That is+a form/format decision, not a mechanical fix; flagged for the maintainer in+`status-checklist.md` §4 rather than guessed at here.++## A third comparison axis: this app's own local check vs. its own installer++The rows above are all "the storefront endpoint vs. registry CI". The+desktop app's in-app Publish tab (`publish.py::validate_draft`) is a THIRD+copy of similar rules, one hop earlier — and it had drifted from+`agent_plugins_loader.py`, the authority this same app uses at install time+(`community_install.install_community_skill`,+`bundled_skills.write_bundled_skills`). The failure mode is the same shape+as the table above, just one step earlier in the pipeline: a submission+passes the in-app "Check" and gets a 201, and only fails once someone+(possibly the very same publisher) tries to install it.++Found and closed 2026-08-15, by making `_validate_bundled_skills` and the+`kind: "skill"` branch of `validate_draft` call+`agent_plugins_loader.validate_bundled_skills` directly instead of+re-implementing a subset of it:++| Divergence | Where | Risk before the fix |+|---|---|---|+| Frontmatter `name`/`description` keys not required | `_frontmatter_keys` + local checks never enforced this | A skill missing either key passed the form, then failed `AgentPluginError` at install — for a STANDALONE skill submission, this could also mean it merges into the live index and is broken for every installer, not just the submitter. |+| "may only share the plugin's own name when it is the sole skill" rule | never implemented client-side (the local function had no `plugin_name` parameter at all) | A bundle could smuggle a skill under the plugin's own name alongside other skills, which the loader would refuse at install, again after merge. |+| `risk_policy` forbidden only checked for a plugin's *bundled* skills, never for a standalone `kind: "skill"` submission | `validate_draft`'s skill branch had no risk_policy check at all | A standalone skill declaring `risk_policy` passed the form outright — the loader still refuses it at install (defense in depth held), but the usability failure is identical to the rows above. |+| Error-accumulation bug: an invalid bundled skill was appended to the returned `skills` list in the same branch that recorded its error | old `_validate_bundled_skills` | Dormant under `validate_draft` (a non-empty `errors` list already forces `return None, errors`), but a latent trap for any future direct caller of the helper. |++All four close by construction now: `_validate_bundled_skills` is a thin+try/except around `agent_plugins_loader.validate_bundled_skills`, so there+is no second rule set left to drift.diff --git a/docs/marketplace/wallpapers-lane.md b/docs/marketplace/wallpapers-lane.mdnew file mode 100644index 0000000000..e36a4b2d3d--- /dev/null+++ b/docs/marketplace/wallpapers-lane.md@@ -0,0 +1,172 @@+# The wallpapers lane — community images, notice-and-action++**Status:** built 2026-08-14, converted from pre-moderation to+notice-and-action 2026-08-15, given an in-app publish surface 2026-08-15.+Live in the registry and the feed; the storefront's app-facing half awaits+the push listed at the end. **Related:**+[community-registry.md](community-registry.md),+[install-standard.md](install-standard.md),+[validator-parity.md](validator-parity.md).++## What changed, and why++The lane originally inverted the marketplace's trust model: every wallpaper+waited in a private queue until the maintainer approved it by hand. The+reasoning was sound — no pattern list recognizes a hateful or illegal+picture — but the cost was not payable. Pre-moderation puts one person in+front of every upload forever, and it makes that person look at exactly the+images the queue exists to stop.++So the lane now publishes automatically, like plugins and skills, and moves+its safeguard behind publication. This is the model the law actually asks of+a host: a platform is not liable for what users upload, but becomes liable+for what it leaves up after being told (DSA Art. 6 and 16). Prior inspection+is not required; prompt removal is.++**The trade accepted here is real and should be stated plainly:** a hateful+or illegal image can be publicly reachable under the project's name until+somebody reports it. What buys that back is the speed and completeness of+removal, so anything that slows a takedown is a defect in this lane, not a+nice-to-have.++## The pipeline, end to end++```+uploader — two surfaces, one endpoint, one identity:+ storefront form GitHub session cookie + optional Turnstile+ desktop picker GitHub device-flow token (Bearer), no Turnstile+ │ re-encode before upload: 4K cap, EXIF stripped, WebP+ │ (browser: canvas · app: Pillow, jarvis/marketplace/publish.py)+ │ machine checks: magic bytes, 8 MB cap, license allowlist,+ │ daily per-account quota, rights statement+ ▼+PUBLIC registry PersonalJarvis/marketplace+ submissions/<name>.json + wallpapers/<name>/wallpaper.<ext>+ │ validate (CI) → expand (ledger) → build: Pillow RE-ENCODE + thumbnail+ ▼+GitHub Pages feed …/marketplace/index.json wallpapers[] + image bytes+ │+ ├── storefront gallery /marketplace/wallpapers (live fold-in)+ ├── app wallpaper picker "Community" chip, browse + one-click add+ ├── app import route POST /api/marketplace/community/wallpapers/{name}/install+ └── CLI jarvis marketplace install <name>++ report link on every detail page → registry issue+ maintainer /marketplace/wallpapers/moderate → one-click delist+ │ removes submissions/<name>.json, then the image+ ▼ next Pages build drops it from the feed and every surface+```++## What actually protects the lane++1. **Identity on every upload.** A GitHub sign-in is required and the+ publisher is derived from the verified identity, never from the form. An+ abusive uploader is a named account, not an anonymous POST. Two ways to+ prove that identity, one standard of proof: a session cookie our own+ callback signed, or a Bearer token verified to belong to OUR GitHub App+ before it counts as anything (`checkTokenBelongsToApp` — a token minted by+ a foreign app is not evidence, however valid it is for GitHub). Turnstile+ is skipped on the Bearer path only: the app cannot render the widget, and+ an interactive device-flow approval is the stronger gate. The quota+ applies to both.+2. **A per-account daily quota** (3/day, `MAX_SUBMISSIONS_PER_DAY`), counted+ from the registry's own commit history. Without a human gate this is the+ main brake on a flood, so it is load-bearing rather than hygiene.+3. **An explicit rights and legality statement**, recorded in the submission+ the uploader publishes under their own name. The upload form says plainly+ that nobody checks it first — a form implying later review would make the+ statement feel optional.+4. **Bytes are always re-produced.** The uploader side re-encodes before+ upload — the browser with a canvas, the app with Pillow — and the registry+ build re-encodes a third time before anything reaches Pages. No byte an+ uploader crafted is served as-is. This never judged the depiction and+ still does not; it stops payloads, not pictures. The app's route takes an+ upload ID rather than image bytes, so what is published is the picture the+ owner is looking at, already sanitized when it entered the picker.+5. **A fast, complete takedown.** Every detail page carries a report link;+ `/marketplace/wallpapers/moderate` lists everything published with a+ two-click Delist. Deletion order mirrors publication in reverse — the+ listing first, then the image — so a half-failure leaves an unreachable+ orphan rather than a listing pointing at nothing.++Because the upload form commits the image and the listing together, a+wallpaper pull request can only ever carry one of the two. `automerge_gate.py`+therefore still refuses `kind=wallpaper`, pinned by `scripts/test_wallpapers.py`+— not as a moderation gate any more, but so a listing whose picture is+missing never merges.++## The "Add Wallpaper to Personal Jarvis" button++Unchanged by this conversion. The storefront button POSTs to the running app+at `127.0.0.1:47821/api/marketplace/community/wallpapers/{name}/install`.+`SurfaceSecurity` admits **exactly this path from exactly the configured+storefront origin** (`marketplace.storefront_origin`, default+`https://personaljarvis.ai`, empty string closes the door) — including the+private-network preflight Chrome requires. It deliberately does NOT add the+storefront to the general trusted origins: a compromised storefront must+never be worth more than "a community wallpaper appeared in the picker".+The app resolves the name against the published feed itself; the website can+not hand it a URL. Fallback when no app answers: the three install-standard+surfaces (CLI / uvx / assistant prompt).++Imports land in the existing uploads store with an `origin:+community:<name>` marker (dedupe + future "installed" badges) and show up in+the picker under "Yours". Note the consequence of automatic publication: a+delisted wallpaper disappears from the feed but stays on machines that+already imported it. Removal is not retroactive.++## Who owns what++| Piece | Repo | Key files |+| --- | --- | --- |+| Feed model, import route, storefront door, CLI | PersonalJarvis (app) | `jarvis/marketplace/community_source.py`, `jarvis/ui/web/marketplace_routes.py`, `jarvis/ui/web/surface_security.py`, `jarvis/cli_ctl/commands/marketplace.py` |+| In-app publish + community browsing | PersonalJarvis (app) | `jarvis/marketplace/publish.py`, `jarvis/ui/web/marketplace_publish_routes.py`, `jarvis/ui/web/frontend/src/views/WallpaperView.tsx`, `.../components/wallpaper/PublishWallpaperDialog.tsx`, `.../hooks/useCommunityWallpapers.ts` |+| Validation, never-auto-merge, ledger, site build | marketplace (registry) | `scripts/validate.py`, `scripts/automerge_gate.py`, `scripts/build_index.py`, `scripts/test_wallpapers.py` |+| Gallery, upload, moderation UI, publish + delist functions | personal-jarvis-webui (storefront) | `src/pages/marketplace/wallpapers/*`, `functions/api/marketplace/submit-wallpaper.ts`, `functions/api/marketplace/wallpapers/*`, `functions/_lib/wallpapers.ts` |++Rules that must move together across repos: the `kind` enum (schema,+`validate.py`, `expand.py`, app), the name regex, the license allowlist+(`WALLPAPER_LICENSES` → `rules.json` → storefront), and the install-command+strings (`install_standard.py` ↔ `src/lib/wallpapers-client.ts`).++## Activation checklist (maintainer)++Measured against the live services on 2026-08-15, not against the clones:++1. **Registry — done.** `index.json` is at revision 13 and carries+ `wallpapers[]` with three CC0 seeds; every image URL resolves.+2. **Storefront — deployed, one commit behind.**+ `POST /api/marketplace/submit-wallpaper` answers **401** (it exists and+ demands identity), but the deployed revision is the PRE-conversion one: it+ still writes to the private inbox repo and accepts a browser session only.+ The local `remake` clone holds two commits it has not seen — the+ notice-and-action rewrite, and the Bearer path the desktop app needs.+ **Until that push, in-app wallpaper publishing answers 401 and the+ picker's Share button is the only part that works.**+3. **App — done and committed on main.** The picker browses the community+ lane and offers Share; the button hides itself when+ `publish_wallpaper_endpoint` is empty, so a fork without the lane shows+ nothing broken.++The private `marketplace-inbox` repo is obsolete once step 2 ships. Nothing+in the current code reads or writes it.++## Residual risks, stated honestly++- **An illegal image is public until reported.** This is the accepted trade,+ not an oversight. The report link, the moderation page and the delist+ endpoint exist to make the exposure window short; keep all three working.+- **Child sexual abuse material is the one case where removal is not+ enough.** Delist it, do not forward or archive it, and report the incident+ to the police or a hotline (in Germany: jugendschutz.net / BKA). Every+ other category is handled by taking the image down.+- **A delisted wallpaper survives on machines that already imported it.** The+ feed is the only thing a takedown reaches.+- **The quota is per account, and accounts are free.** A determined attacker+ can make more accounts. If that ever happens in practice, the cheap next+ step is an automated image-moderation check on upload (an OpenAI moderation+ call or Cloudflare Images' classifier), which was the alternative to this+ design and remains the natural escalation.+- The skills `raw_url` download in `jarvis/skills/finder.py` still follows+ redirects (pre-existing); the wallpaper download does not. Worth unifying+ some day.