Skip to content

Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.

fix(release): ship unproven plugins as preview instead of blocking releases

rubenluetke10-beepbf55d009d168
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 9a54d769e5..464eeef5b2 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -251,20 +251,15 @@ jobs:
python-version: "3.11"
- run: python -m pip install -e . --no-deps
- run: python -m pip install pyyaml==6.0.3 pydantic loguru python-dotenv packaging rich typer platformdirs click httpx
- - name: Qualify plugin auth with the v2.9.0 exception
+ - name: Qualify plugin auth and disclose preview plugins
shell: bash
- env:
- RELEASE_REF: ${{ github.ref }}
run: |
- # Maintainer-approved on 2026-10-04 for this immutable release only.
- # Keep structural/security checks and the audit's BLOCKED results intact.
- if [ "$RELEASE_REF" = "refs/tags/v2.9.0" ]; then
- python scripts/ci/check_plugin_auth_contract.py
- echo "::warning::Full plugin acceptance is waived for v2.9.0 only; no plugin E2E PASS is asserted."
- echo "Plugin E2E acceptance: maintainer-approved exception for v2.9.0 only. Existing audit results remain unchanged. All other release checks remain required." >> "$GITHUB_STEP_SUMMARY"
- else
- python scripts/ci/check_plugin_auth_contract.py --require-e2e-pass
- fi
+ # A plugin ships as "verified" only with a completed browser journey;
+ # every other one ships labeled "preview" in the app (AGENTS.md section 3).
+ report="$(python scripts/ci/check_plugin_auth_contract.py --require-e2e-pass)"
+ printf '%s\n' "$report"
+ # The last line counts verified and preview plugins for the release.
+ printf '%s\n' "${report##*$'\n'}" >> "$GITHUB_STEP_SUMMARY"
python-fast:
name: python contracts (fast)
diff --git a/AGENTS.md b/AGENTS.md
index 1f35438980..ad39cb5f0e 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -157,8 +157,10 @@ own-client stays an expert override only. Every catalog change keeps
`docs/marketplace/plugin-auth-audit.md` and `scripts/ci/check_plugin_auth_contract.py`
green; provider error bodies never reach logs, UI, or storage (AP-34).
Keep `docs/marketplace/plugin-e2e-audit.json` backed by real browser evidence;
-new built-ins require PASS and release qualification uses the auth gate's
-`--require-e2e-pass` option. BLOCKED never means provider-verified or complete.
+new built-ins require PASS, and a plugin ships as `acceptance: "verified"` only
+with a completed PASS; every other one ships labeled "preview" in the app.
+Release qualification runs the auth gate with `--require-e2e-pass`, which also
+makes every PASS ship as verified. BLOCKED never means provider-verified or complete.
The rest of the register, one line each, because code comments cite these
numbers: never hardcode an Anthropic/Claude client (AP-6); keep awareness and
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1c2c141ba0..319384e031 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,10 @@ versioning per [SemVer](https://semver.org/).
## [Unreleased]
+### Changed
+
+- Built-in plugins that have not yet passed a full end-to-end sign-in test now show a Preview badge, and releases no longer wait for every plugin to pass that test. A plugin is marked verified only after its complete journey was observed.
+
---
## [2.9.0] — 2026-10-05
diff --git a/docs/ci-pipeline.md b/docs/ci-pipeline.md
index 32c4c9dc1e..d27e236072 100644
--- a/docs/ci-pipeline.md
+++ b/docs/ci-pipeline.md
@@ -26,7 +26,7 @@ agent branch ──► pull request ──► CI (lanes) ──► CI gate ─
| `tests macos 1..3` | Nightly and manual runs only (~10x runner cost). | — |
| `test report + floor` | Proves the six Linux shards cover every discovered file exactly once, enforces the min-passed floor, and on main refreshes the duration cache. Detection freezes one shared duration snapshot for all shards, including partial reruns. | `scripts/ci/ratchet_tests.py` |
| Lanes | `frontend`, `jarvisctl`, `deps`, `realtime` (3 OS + slim container), `updater` (3 OS: in-app update, native handover, restart helper), `dragdrop`, `browser`, `macOS desktop`, `installer smoke` — each only when its paths change. | — |
-| `release qualification` | Tag CI requires a full run with macOS and the real browser-auth E2E evidence gate. Ordinary branch/PR/nightly CI skips this release-only requirement. | `scripts/ci/check_plugin_auth_contract.py --require-e2e-pass` |
+| `release qualification` | Tag CI requires a full run with macOS and the browser-auth E2E evidence gate: a plugin labeled verified needs a completed journey, every completed journey ships as verified, and every other plugin ships labeled preview. The step summary counts both. Ordinary branch/PR/nightly CI skips this release-only job. | `scripts/ci/check_plugin_auth_contract.py --require-e2e-pass` |
| `CI gate` | Aggregates every job. **The only required check.** A missing or skipped selected lane fails. Unselected lanes may skip; nightly is strict except event-specific jobs. | `scripts/ci/required_results.py` |
The realtime lane runs the subscription authentication, direct reasoning,
diff --git a/docs/marketplace/browser-auth-standard.md b/docs/marketplace/browser-auth-standard.md
index 2a835e199f..4c75826209 100644
--- a/docs/marketplace/browser-auth-standard.md
+++ b/docs/marketplace/browser-auth-standard.md
@@ -100,8 +100,11 @@ Enforced by:
- `scripts/ci/check_plugin_auth_contract.py` — blocking configuration and
browser-evidence gate. `plugin-e2e-audit.json` records all seven observed
stages, a real action for each smoke PASS, concrete blockers, and a fingerprint
- of each plugin's auth and execution configuration. New built-ins require PASS;
- `--require-e2e-pass` rejects every remaining blocker for release qualification.
+ of each plugin's auth and execution configuration. New built-ins require PASS.
+ The catalog field `acceptance` is "verified" only with a completed PASS; every
+ other plugin ships as "preview" and the app shows a Preview badge on it.
+ `--require-e2e-pass` (release qualification) also requires every PASS to ship
+ as verified.
Evidence is an attestation of an observed provider journey. A schema check cannot
perform account consent. Neither an opened page, a successful mock, nor MCP tool
diff --git a/docs/marketplace/plugin-auth-audit.md b/docs/marketplace/plugin-auth-audit.md
index c98ff3ef32..f31f864923 100644
--- a/docs/marketplace/plugin-auth-audit.md
+++ b/docs/marketplace/plugin-auth-audit.md
@@ -81,7 +81,7 @@ Shopify (added 2026-09-21, after the RUB-94 observation) has one observed UI att
## Release and maintenance gate
-Run `python scripts/ci/check_plugin_auth_contract.py` for configuration, matrix coverage and attestation integrity. Historical BLOCKED rows may pass this structural check only with concrete blocker evidence; that does not make them standard-ready. An unresolved FAIL is deliberately rejected. Run `python scripts/ci/check_plugin_auth_contract.py --require-e2e-pass` for strict release qualification; current evidence must fail qualification.
+Run `python scripts/ci/check_plugin_auth_contract.py` for configuration, matrix coverage and attestation integrity. Historical BLOCKED rows may pass this structural check only with concrete blocker evidence; that does not make them standard-ready. An unresolved FAIL is deliberately rejected. A plugin may carry `acceptance: "verified"` in the seed catalog only with a completed PASS row; every other plugin ships as "preview" and shows a Preview badge in the app. Run `python scripts/ci/check_plugin_auth_contract.py --require-e2e-pass` for release qualification; it additionally requires every PASS to ship as verified and prints how many plugins ship as preview.
A new built-in plugin must carry a complete real journey PASS before release. Record a genuine functional action for smoke evidence; discovery or listing available tools is insufficient. An auth/execution configuration change invalidates the fingerprint and requires renewed relevant observations. Never refresh a fingerprint to disguise stale successful evidence.
diff --git a/jarvis/marketplace/catalog.py b/jarvis/marketplace/catalog.py
index 193c9df7f3..1267115661 100644
--- a/jarvis/marketplace/catalog.py
+++ b/jarvis/marketplace/catalog.py
@@ -175,6 +175,15 @@ CATEGORY_ORDER: tuple[str, ...] = (
# cannot extend; `longevity_note` must say how often
Longevity = Literal["permanent", "self_renewing", "provider_limited"]
+# Whether the full browser journey of a plugin was observed end to end
+# (docs/marketplace/plugin-e2e-audit.json). Rendered as a badge on the card.
+# verified - a completed PASS row exists for the current auth/execution
+# configuration; scripts/ci/check_plugin_auth_contract.py
+# rejects this label on any other row
+# preview - shipped, but not yet proven end to end; sign-in may still need
+# publisher setup. The default, so nothing is verified by omission.
+Acceptance = Literal["verified", "preview"]
+
class PluginSpec(_BaseAuth):
id: str
@@ -208,6 +217,10 @@ class PluginSpec(_BaseAuth):
# `provider_limited` ("Google requires re-approval every 7 days while the
# OAuth app is in Testing mode"), optional otherwise.
longevity_note: str | None = None
+ # End-to-end acceptance label (see `Acceptance`). Only the seed catalog
+ # sets "verified"; community and uploaded entries carry their own
+ # not-reviewed badges instead.
+ acceptance: Acceptance = "preview"
# Which `<family>_oauth_client_id` secret pair overrides this plugin's
# catalog client, for providers where the downloader must register their
# own OAuth app. Lives here so a new plugin stays a catalog-only change:
diff --git a/jarvis/ui/web/frontend/src/i18n/locales/de.json b/jarvis/ui/web/frontend/src/i18n/locales/de.json
index ad58235ef0..be5a076b14 100644
--- a/jarvis/ui/web/frontend/src/i18n/locales/de.json
+++ b/jarvis/ui/web/frontend/src/i18n/locales/de.json
@@ -1301,6 +1301,8 @@
"add_upload": "Eigenes Plugin hochladen",
"add_community": "Aus dem Community-Marketplace",
"add_marketplace": "Marketplace öffnen",
+ "preview_badge": "Vorschau",
+ "preview_tooltip": "Noch nicht vollständig getestet. Die Anmeldung kann noch zusätzliche Schritte brauchen.",
"filter_label": "Anzeigen",
"filter_recommended": "Empfohlen",
"added": "Hinzugefügt",
diff --git a/jarvis/ui/web/frontend/src/i18n/locales/en.json b/jarvis/ui/web/frontend/src/i18n/locales/en.json
index 9505674d9a..ffaa10b3f1 100644
--- a/jarvis/ui/web/frontend/src/i18n/locales/en.json
+++ b/jarvis/ui/web/frontend/src/i18n/locales/en.json
@@ -1301,6 +1301,8 @@
"add_upload": "Upload your own plugin",
"add_community": "From the community marketplace",
"add_marketplace": "Open the Marketplace",
+ "preview_badge": "Preview",
+ "preview_tooltip": "Not yet tested end to end. Signing in may still need extra steps.",
"filter_label": "Show",
"filter_recommended": "Recommended",
"added": "Added",
diff --git a/jarvis/ui/web/frontend/src/i18n/locales/es.json b/jarvis/ui/web/frontend/src/i18n/locales/es.json
index e1ba7b33fc..242c101831 100644
--- a/jarvis/ui/web/frontend/src/i18n/locales/es.json
+++ b/jarvis/ui/web/frontend/src/i18n/locales/es.json
@@ -1301,6 +1301,8 @@
"add_upload": "Subir tu propio plugin",
"add_community": "Desde el marketplace de la comunidad",
"add_marketplace": "Abrir el Marketplace",
+ "preview_badge": "Vista previa",
+ "preview_tooltip": "Aún no probado de principio a fin. Iniciar sesión puede requerir pasos adicionales.",
"filter_label": "Mostrar",
"filter_recommended": "Recomendados",
"added": "Añadido",
diff --git a/jarvis/ui/web/frontend/src/views/PluginPreviewBadge.test.tsx b/jarvis/ui/web/frontend/src/views/PluginPreviewBadge.test.tsx
new file mode 100644
index 0000000000..5b9648c6c6
--- /dev/null
+++ b/jarvis/ui/web/frontend/src/views/PluginPreviewBadge.test.tsx
@@ -0,0 +1,45 @@
+import { cleanup, render, screen } from "@testing-library/react";
+import { afterEach, describe, expect, it } from "vitest";
+
+import { type Plugin, PreviewBadge } from "@/views/PluginsView";
+
+afterEach(cleanup);
+
+function plugin(overrides: Partial<Plugin> = {}): Plugin {
+ return {
+ id: "github",
+ name: "GitHub",
+ description: "Repos and pull requests",
+ category: "Developer",
+ logoSlug: "github",
+ authMode: "oauth_device_flow",
+ authConfig: { mode: "oauth_device_flow" },
+ status: "available",
+ longevity: "self_renewing",
+ oauthClientConfigured: false,
+ fromMarketplace: false,
+ selfUploaded: false,
+ acceptance: "preview",
+ ...overrides,
+ } as Plugin;
+}
+
+describe("PreviewBadge", () => {
+ it("marks a built-in plugin that has not passed the end-to-end journey", () => {
+ render(<PreviewBadge plugin={plugin()} />);
+ const badge = screen.getByText("Preview");
+ expect(badge.getAttribute("title")).toMatch(/end to end/);
+ });
+
+ it("stays hidden once the journey is verified", () => {
+ const { container } = render(<PreviewBadge plugin={plugin({ acceptance: "verified" })} />);
+ expect(container.textContent).toBe("");
+ });
+
+ it("leaves marketplace and self-uploaded plugins to their own badges", () => {
+ const market = render(<PreviewBadge plugin={plugin({ fromMarketplace: true })} />);
+ expect(market.container.textContent).toBe("");
+ const own = render(<PreviewBadge plugin={plugin({ selfUploaded: true })} />);
+ expect(own.container.textContent).toBe("");
+ });
+});
diff --git a/jarvis/ui/web/frontend/src/views/PluginsView.tsx b/jarvis/ui/web/frontend/src/views/PluginsView.tsx
index 720c633cf7..d8f07ebdda 100644
--- a/jarvis/ui/web/frontend/src/views/PluginsView.tsx
+++ b/jarvis/ui/web/frontend/src/views/PluginsView.tsx
@@ -77,6 +77,9 @@ type PluginStatus = "not_connected" | "connected" | "needs_reauth" | "error";
* already know. */
type Category = string;
type Longevity = "permanent" | "self_renewing" | "provider_limited";
+/** Mirrors `Acceptance` in jarvis/marketplace/catalog.py: "verified" only
+ * after a completed end-to-end browser journey, "preview" otherwise. */
+type Acceptance = "verified" | "preview";
/** Mirrors the backend's REAUTH_* codes (jarvis/marketplace/token_store.py). */
type ReauthReason =
| "provider_rejected"
@@ -130,6 +133,7 @@ interface CatalogPlugin {
featured?: boolean;
longevity?: Longevity;
longevity_note?: string | null;
+ acceptance?: Acceptance;
oauth_client_family?: string | null;
oauth_client_configured?: boolean;
auth: { mode: AuthMode; [key: string]: unknown };
@@ -240,6 +244,8 @@ export interface Plugin {
unavailableReason?: string;
longevity: Longevity;
longevityNote?: string;
+ /** Undefined only in hand-built fixtures; the catalog always states it. */
+ acceptance?: Acceptance;
oauthClientFamily?: string;
reauthReason?: ReauthReason | string;
reauthAt?: string;
@@ -277,6 +283,7 @@ function adapt(p: CatalogPlugin): Plugin {
unavailableReason: p.unavailable_reason ?? undefined,
longevity: p.longevity ?? "self_renewing",
longevityNote: p.longevity_note ?? undefined,
+ acceptance: p.acceptance ?? "preview",
oauthClientFamily: p.oauth_client_family ?? undefined,
reauthReason: p.reauth_reason ?? undefined,
reauthAt: p.reauth_at ?? undefined,
@@ -1554,6 +1561,7 @@ function PluginTableRow({
<div className="min-w-0">
<div className="flex items-center gap-2">
<span className="truncate text-title font-medium">{plugin.name}</span>
+ <PreviewBadge plugin={plugin} />
{plugin.fromMarketplace && <MarketplaceBadge publisher={plugin.publisher} />}
{plugin.selfUploaded && (
<span
@@ -1645,6 +1653,7 @@ function PluginDetail({ plugin, onConnect, onDisconnect }: { plugin: Plugin } &
title={plugin.name}
titleAccessory={
<>
+ <PreviewBadge plugin={plugin} />
{plugin.fromMarketplace && <MarketplaceBadge publisher={plugin.publisher} />}
{plugin.selfUploaded && (
<span
@@ -1901,6 +1910,25 @@ export function BrandTile({ plugin, size = "md" }: { plugin: Plugin; size?: "sm"
* dies weeks later. `provider_limited` carries a note explaining how often — a
* warning without an answer would be worse than none.
*/
+/** "Not yet proven end to end." Built-in plugins only: a marketplace or
+ * self-uploaded plugin already carries its own not-reviewed badge. Theme
+ * tokens only, so it reads in light and dark mode. */
+export function PreviewBadge({ plugin }: { plugin: Plugin }) {
+ if (plugin.acceptance !== "preview" || plugin.fromMarketplace || plugin.selfUploaded) {
+ return null;
+ }
+ const title = translate("plugins_view.preview_tooltip");
+ return (
+ <span
+ title={title}
+ aria-label={title}
+ className="shrink-0 rounded-full border border-border px-1.5 text-micro font-medium text-muted-foreground"
+ >
+ {translate("plugins_view.preview_badge")}
+ </span>
+ );
+}
+
export function LongevityBadge({ plugin }: { plugin: Plugin }) {
const limited = plugin.longevity === "provider_limited";
return (
diff --git a/scripts/ci/check_plugin_auth_contract.py b/scripts/ci/check_plugin_auth_contract.py
index d96affb106..0ebe1475cd 100644
--- a/scripts/ci/check_plugin_auth_contract.py
+++ b/scripts/ci/check_plugin_auth_contract.py
@@ -18,6 +18,10 @@ metadata flag — and fails the build when:
blocked + owner of the remaining step).
6. A PAT entry uses a non-https validation endpoint or documents no
accepted token prefix.
+ 7. A plugin is labeled ``acceptance: "verified"`` without a completed
+ browser journey (docs/marketplace/plugin-e2e-audit.json). Every other
+ plugin ships labeled "preview", so an unproven journey is disclosed in
+ the app instead of blocking the whole release.
Stdlib only, cross-platform (pathlib, UTF-8). Exit 0 = clean, 1 = findings.
"""
@@ -73,8 +77,10 @@ def config_fingerprint(plugin: dict) -> str:
def validate_e2e_audit(plugins: list[dict], audit: object, *, strict: bool = False) -> list[str]:
"""Validate attestations; evidence is manually observed, never inferred here.
- CI tolerates explicit historical external blockers. Release qualification
- uses --require-e2e-pass and accepts only completed journeys.
+ Every run: a plugin labeled "verified" needs a completed PASS, a new
+ built-in needs a PASS, and a legacy BLOCKED row needs a concrete blocker.
+ Release qualification (--require-e2e-pass) additionally requires every
+ PASS plugin to ship labeled "verified", so users see what was proven.
"""
if not isinstance(audit, dict) or audit.get("schema_version") != 1:
return ["e2e: expected audit schema_version 1"]
@@ -105,9 +111,16 @@ def validate_e2e_audit(plugins: list[dict], audit: object, *, strict: bool = Fal
findings.append(f"{pid}: unresolved e2e result {result!r}")
if row.get("config_fingerprint") != config_fingerprint(plugin):
findings.append(f"{pid}: stale e2e evidence; auth/execution configuration changed")
+ acceptance = plugin.get("acceptance", "preview")
+ if acceptance not in ("verified", "preview"):
+ findings.append(f"{pid}: unknown acceptance label {acceptance!r}")
+ if acceptance == "verified" and result != "PASS":
+ findings.append(f"{pid}: acceptance 'verified' needs a completed e2e PASS")
+ if strict and result == "PASS" and acceptance != "verified":
+ findings.append(f"{pid}: completed e2e PASS must ship as acceptance 'verified'")
if result == "BLOCKED":
- if strict or pid not in LEGACY_PLUGIN_IDS:
- findings.append(f"{pid}: completed e2e PASS required for release/new built-in")
+ if pid not in LEGACY_PLUGIN_IDS:
+ findings.append(f"{pid}: completed e2e PASS required for a new built-in")
blocker = row.get("blocker")
if not isinstance(blocker, dict) or any(
not isinstance(blocker.get(key), str) or len(blocker[key].strip()) < 12
@@ -377,6 +390,11 @@ def main() -> int:
print(f" - {f}")
return 1
print(f"auth-contract: clean ({len(plugins)} seed plugins checked)")
+ preview = sorted(pl["id"] for pl in plugins if pl.get("acceptance", "preview") != "verified")
+ print(
+ f"auth-contract: {len(plugins) - len(preview)} verified, {len(preview)} shipped as "
+ f"preview: {', '.join(preview) or 'none'}"
+ )
return 0
diff --git a/tests/unit/ci/test_release_cut_workflow.py b/tests/unit/ci/test_release_cut_workflow.py
index e25be41684..f7158336c4 100644
--- a/tests/unit/ci/test_release_cut_workflow.py
+++ b/tests/unit/ci/test_release_cut_workflow.py
@@ -118,46 +118,30 @@ def _exercise(
return result, commands
- "ref,strict",
- [
- ("refs/tags/v2.9.0", False),
- ("refs/tags/v2.9.1", True),
- ("refs/tags/v2.10.0", True),
- ("refs/tags/v2.9.0-rc.1", True),
- ("refs/tags/v2.9.00", True),
- ("refs/heads/v2.9.0", True),
- ("refs/heads/main", True),
- ("", True),
- ],
-)
-def test_plugin_acceptance_exception_is_exactly_one_release(tmp_path, ref, strict):
+_QUALIFY = "Qualify plugin auth and disclose preview plugins"
+
+
+def test_release_qualification_is_strict_with_no_per_tag_exception(tmp_path):
+ root = Path(__file__).resolve().parents[3]
+ workflow = (root / ".github/workflows/ci.yml").read_text("utf-8")
+ assert "refs/tags/v2.9.0" not in workflow # the one-time exception is gone
result, commands = _exercise(
tmp_path,
- ["Qualify plugin auth with the v2.9.0 exception"],
+ [_QUALIFY],
workflow_name="ci.yml",
job_id="release-qualification",
- RELEASE_REF=ref,
)
assert result.returncode == 0, result.stdout + result.stderr
- command = "python scripts/ci/check_plugin_auth_contract.py"
- assert commands == [command + (" --require-e2e-pass" if strict else "")]
- summary = tmp_path / "summary.txt"
- if strict:
- assert not summary.exists()
- else:
- assert "no plugin E2E PASS is asserted" in result.stdout
- assert "Existing audit results remain unchanged" in summary.read_text("utf-8")
+ assert commands == ["python scripts/ci/check_plugin_auth_contract.py --require-e2e-pass"]
+ assert (tmp_path / "summary.txt").exists()
[email protected]("ref", ["refs/tags/v2.9.0", "refs/tags/v2.9.1"])
-def test_plugin_exception_never_ignores_a_failing_auth_contract(tmp_path, ref):
+def test_release_qualification_fails_on_a_failing_auth_contract(tmp_path):
result, _commands = _exercise(
tmp_path,
- ["Qualify plugin auth with the v2.9.0 exception"],
+ [_QUALIFY],
workflow_name="ci.yml",
job_id="release-qualification",
- RELEASE_REF=ref,
ADMIT_EXIT="1",
)
assert result.returncode != 0
diff --git a/tests/unit/marketplace/test_catalog_seed.py b/tests/unit/marketplace/test_catalog_seed.py
index 28343b1ece..4f7cb94f7c 100644
--- a/tests/unit/marketplace/test_catalog_seed.py
+++ b/tests/unit/marketplace/test_catalog_seed.py
@@ -36,6 +36,26 @@ def test_every_seed_category_is_declared_in_the_display_order() -> None:
)
+def test_acceptance_labels_match_across_python_typescript_and_the_api() -> None:
+ """`acceptance` crosses Python -> JSON -> TS -> UI; one drifted value would
+ hide or misplace the Preview badge."""
+ from typing import get_args
+
+ from jarvis.marketplace.catalog import Acceptance, PluginSpec
+
+ labels = set(get_args(Acceptance))
+ assert labels == {"verified", "preview"}
+ assert PluginSpec.model_fields["acceptance"].default == "preview"
+ view = (
+ Path(__file__).resolve().parents[3] / "jarvis/ui/web/frontend/src/views/PluginsView.tsx"
+ ).read_text(encoding="utf-8")
+ assert 'type Acceptance = "verified" | "preview";' in view
+ assert 'acceptance: p.acceptance ?? "preview"' in view
+ for plugin in _seed().plugins:
+ dumped = plugin.model_dump(mode="json")
+ assert dumped["acceptance"] in labels, plugin.id
+
+
def test_every_seed_plugin_states_its_longevity_note_when_limited() -> None:
"""A `provider_limited` card must say HOW often the user has to come back —
the badge alone would be a warning without an answer."""
diff --git a/tests/unit/marketplace/test_plugin_e2e_audit_gate.py b/tests/unit/marketplace/test_plugin_e2e_audit_gate.py
index b3fd36fe36..21c7d0fb6f 100644
--- a/tests/unit/marketplace/test_plugin_e2e_audit_gate.py
+++ b/tests/unit/marketplace/test_plugin_e2e_audit_gate.py
@@ -37,6 +37,7 @@ def journey():
def test_complete_journey_passes(journey):
plugin, audit = journey
+ plugin["acceptance"] = "verified"
assert gate.validate_e2e_audit([plugin], audit, strict=True) == []
@@ -79,15 +80,41 @@ def blocked(audit):
row["evidence"][stage]["status"] = "BLOCKED"
-def test_legacy_blocker_is_honest_but_not_release_qualification(journey):
+def test_legacy_blocker_ships_as_preview_without_blocking_the_release(journey):
plugin, audit = journey
blocked(audit)
assert gate.validate_e2e_audit([plugin], audit) == []
+ assert gate.validate_e2e_audit([plugin], audit, strict=True) == []
+
+
[email protected]("strict", [False, True])
+def test_verified_label_requires_a_completed_journey(journey, strict):
+ plugin, audit = journey
+ blocked(audit)
+ plugin["acceptance"] = "verified"
assert any(
- "PASS required" in item for item in gate.validate_e2e_audit([plugin], audit, strict=True)
+ "acceptance 'verified' needs" in item
+ for item in gate.validate_e2e_audit([plugin], audit, strict=strict)
)
+def test_release_ships_a_completed_journey_as_verified(journey):
+ plugin, audit = journey
+ assert gate.validate_e2e_audit([plugin], audit) == []
+ assert any(
+ "must ship as acceptance 'verified'" in item
+ for item in gate.validate_e2e_audit([plugin], audit, strict=True)
+ )
+ plugin["acceptance"] = "verified"
+ assert gate.validate_e2e_audit([plugin], audit, strict=True) == []
+
+
+def test_unknown_acceptance_label_is_rejected(journey):
+ plugin, audit = journey
+ plugin["acceptance"] = "beta"
+ assert any("unknown acceptance" in item for item in gate.validate_e2e_audit([plugin], audit))
+
+
def test_new_plugin_cannot_bypass_with_blocked_row(journey):
plugin, audit = journey
blocked(audit)