Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
docs(versioning): design cross-artifact version compatibility (CHOO-1865)
Louis Amaudruz80752a8885fd
diff --git a/docs/VERSIONING.md b/docs/VERSIONING.mdnew file mode 100644index 000000000..76e8630da--- /dev/null+++ b/docs/VERSIONING.md@@ -0,0 +1,583 @@+# Versioning and cross-artifact compatibility++Status: **design, pending approval** · Ticket: [CHOO-1865](https://sandboxquantum.atlassian.net/browse/CHOO-1865)++Switch ships many independently-versioned artifacts. This document defines how+they declare compatibility with one another, and how they are updated safely+while sessions are running.++**Scope.** This document owns *which versions work together, and how transitions+between versions are handled safely*. How each artifact is packaged and+distributed is [CHOO-1864](https://sandboxquantum.atlassian.net/browse/CHOO-1864)+and deliberately not covered here.++---++## 1. The problem++Six shippable artifacts, three disjoint release tags, and no declared+interdependency between any of them:++| Artifact | Version today | Released by |+| --- | --- | --- |+| switch-core (+ `gateway`, `setup` images, Helm chart, standalone compose) | `0.12.3` | tag `switch-v*` |+| switchdash desktop | `0.19.2` | tag `switchdash-v*` |+| `@sandbox-quantum/switch-agent-runtime` | `0.1.5` | tag `switch-agent-runtime-v*` |+| Claude Code connector plugin | `0.7.8` | none — served from the default branch |+| Codex connector plugin | `0.2.0` | none — served from the default branch |+| Remote sidecar | `1.7` | ships inside switchdash |++Nothing states which combinations are valid. Neither a user nor the software+itself can answer *"does this switchdash work against that switch-core?"* except+by trying it.++Four tickets are four edges of this one graph, each solved separately:+[CHOO-1321](https://sandboxquantum.atlassian.net/browse/CHOO-1321) (core ↔+switchdash), [CHOO-601](https://sandboxquantum.atlassian.net/browse/CHOO-601)+(plugin version reporting),+[CHOO-1736](https://sandboxquantum.atlassian.net/browse/CHOO-1736) (managed stack+not following the app's pin), and+[CHOO-1677](https://sandboxquantum.atlassian.net/browse/CHOO-1677) (switchdash+bundling a stale core). Solving them pairwise gives N² special cases and no+shared vocabulary.++### 1.1 What already exists++Two mechanisms in the tree already do this correctly for one edge each, and the+model below generalises them rather than replacing them:++- **`PROTOCOL_VERSION = 1`** (`core/switch_core/bridges/agent/protocol/connections.py`)+ — an integer checked when an agent opens the event stream.+- **The sidecar's major** (`dash/apps/switchdash-desktop/src/sidecar/sidecar-version.ts`)+ — `SIDECAR_VERSION` plus an explicit `MIN_SUPPORTED_SIDECAR_MAJOR` floor, with+ compatibility judged on the major alone. This is the most developed+ compatibility model in the repo and the direct ancestor of §3.++### 1.2 What is missing++**switch-core cannot state its own version.** `0.12.3` lives in+`core/pyproject.toml`, never enters the Python process, and is not stamped into+the image. The only introspection endpoint is `GET /health`+(`core/switch_core/main.py`), which returns `{"status": "ok"}`.++Consequently the only working version check in the repo+(`dash/apps/switchdash-desktop/src/main/core/managed-switch-server/deployed-version.ts`)+reverse-engineers the deployed version from the running container's image tag.+That works only for stacks switchdash itself manages. For a switch-core someone+else operates, switchdash has no visibility at all.++**No client version reaches the server.** There is no version header, no field+on agent registration, and no version column on any client-bearing table.++### 1.3 "Unknown" currently renders as "fine" — in four places++The ticket's non-negotiable rule is that *incompatible*, *untested* and+*unknown* stay distinct, and that unknown never reads as fine. Today it does:++1. `protocol_version` on the events endpoint **defaults to the server's own+ value**, so a client that declares nothing is treated as compatible. No+ shipped client sends it, so the check has never fired in production.+2. `readVersionStatus()` returns `drift: null` — indistinguishable from "no+ drift" — when the deployed version cannot be read.+3. The standalone compose interpolates `${SWITCH_VERSION:-latest}`, so a missing+ env var silently floats the stack to `latest`.+4. `refuseDowngrade()` blocks only when the direction is *provably* a downgrade;+ an uncomparable pair passes through.++These are the same defect four times, and fixing that pattern is a larger share+of the value here than any new machinery.++---++## 2. Artifact versions++**Every artifact uses semantic versioning `MAJOR.MINOR.PATCH`, and an artifact+version describes only that artifact.** It says *where you are*. It carries no+compatibility meaning whatsoever.++| Field | Meaning |+| --- | --- |+| `MAJOR` | Breaking change to how the artifact is used, configured or installed |+| `MINOR` | New functionality |+| `PATCH` | Fixes |++A protocol break will usually ship alongside a major bump, but **nothing may+infer compatibility from an artifact version**. Versions inform people;+contracts (§3) inform machines.++Changes required:++- **Sidecar** moves from two components to three: `1.7` → `1.7.0`. Its major+ **stays at 1** — see §11.1.+- **Gateway** (`gateway/package.json`) has no `version` field. It gets one, or+ is explicitly documented as part of the switch-core release rather than a+ separate artifact.+- **Helm chart** `Chart.yaml` says `0.2.1` and is silently overwritten at+ package time with the tag version. The file stops lying.++**Every artifact gets a changelog.** Today only switch-core and switchdash have+one. Each release produces two outputs from one event: a **changelog entry** for+humans and a **release manifest** (§5) for machines, generated together so they+cannot drift.++---++## 3. Compatibility lives on contracts, not artifacts++A **contract** is a named interface where two artifacts talk. Each contract+carries a single integer, incremented only when that interface changes in a way+the other side can observe.++Each participant declares two numbers per contract:++- **`speaks`** — the newest version it implements+- **`accepts`** — the oldest version it still handles++Two participants are compatible when their ranges overlap: each side's `speaks`+is at or above the other side's `accepts`. They then operate at the lower of the+two `speaks`.++### 3.1 The contracts++| Contract | Between | Today |+| --- | --- | --- |+| `agent-protocol` | switch-core ↔ agent runtime (and the sidecar, which embeds it) | exists as `PROTOCOL_VERSION = 1` |+| `gateway-api` | switch-core ↔ switchdash, gateway frontend | new |+| `stack-compose` | standalone compose ↔ switchdash's managed-stack pipeline | new |+| `sidecar-control` | remote sidecar ↔ switchdash | exists as the sidecar major |+| `db-schema` | switch-core ↔ Postgres | Alembic revision; one-way |++Five edges, not N² pairs. Adding an artifact means declaring which contracts it+participates in; it does not mean touching the others.++### 3.2 The connector plugins are on no contract++A plugin's payload is a skill, a hook and an MCP config. It declares which+runtime version to launch, and the runtime fetches its tool definitions from the+server at startup (`GET /agents/{id}/ops`), so a plugin's tool surface cannot go+stale.++**A plugin's compatibility is the runtime's, inherited.** This is why there is no+plugin contract. It also settles CHOO-601: the server should *know and report*+which plugin version an agent runs, and must never *gate* a connection on it —+the contract underneath decides that.++> **Gate on contracts. Report on releases.**++### 3.3 Where contract numbers live++Contract versions are **baked in at build**, because they describe what the code+does — a component cannot look this up, it *is* this.++One authored file at the repo root is the only thing a human edits. From it,+CI generates a Python module and a TypeScript module so the two languages cannot+disagree, and fails if the generated files do not match their source.++This is deliberate: `COMPATIBLE_SWITCH_VERSION` is currently hand-copied into+`app-identity.canary.ts` with a "keep in sync" comment and nothing enforcing it.++### 3.4 How the numbers are exchanged++- **Client → server** — on the connect it already makes. The events endpoint+ already accepts `?protocol=`; the runtime starts sending it, along with its+ `accepts` floor and its artifact version. The default must be removed: an+ absent declaration is *unknown*, not *current*.+- **Server → client** — **`GET /version`**, returning switch-core's artifact+ version, its `speaks`/`accepts` per contract, and its schema revision.+ Unauthenticated, alongside `/health` in the auth allowlist.++`GET /version` is the one genuinely new primitive, and everything else depends+on it.++---++## 4. Two questions, not one++Every artifact answers two independent questions:++- **Compatible?** — the contract check. A hard gate. Changes rarely.+- **Current?** — is this the build it should be on. Soft. Changes constantly.++Compatibility is not currency. A plugin from three months ago may be perfectly+compatible and still worse: stale skill text, an unfixed hook bug. The contract+model cannot see that, and should not.++The plugin and the sidecar live almost entirely in the second question —+`sidecar-control` will sit at 1 for a long time, so for the sidecar the currency+question does nearly all the work. That is already why its code compares content+hashes first and uses the version only as a tie-break.++**For artifacts we own and that hold no user data, the default is to keep them+current, with the contract check as the brake rather than the engine.**++### 4.1 Compatibility states++| State | Meaning | Behaviour |+| --- | --- | --- |+| `compatible` | Ranges overlap | Silent |+| `outdated` | Compatible; a newer build exists | Informational |+| `deprecated` | Compatible today; the peer has announced it is dropping this version | Warned, while it still works |+| `unknown` | The peer declared nothing, or it could not be read | Shown as unknown — **never as fine** |+| `incompatible` | Ranges disjoint | Blocked at the boundary |++`untested` — ranges overlap but this exact release combination was never+exercised — is a **release-layer advisory generated by CI from what we actually+ran**. It never blocks. Keeping it on a separate layer from `incompatible` is+what stops the tested-combination set from becoming a hand-maintained matrix+that rots.++### 4.2 Never report a mismatch without the remedy++Because both sides declare a range, we know not only that they disagree but+**who must move and to what**. The output is never "incompatible"; it is+"requires switch-core 0.14 or newer, this one is 0.12".++Direction is not symmetric:++- **Client too old** — common. The client updates; for the runtime and sidecar+ we simply replace them.+- **Server too old** — nothing the client does fixes it. For a managed stack,+ offer to update it; for a server we do not manage, report clearly.+- **Either side could move** — prefer moving the **client**. It is cheap and+ reversible. Moving the server means a schema migration, which is not.++---++## 5. Release manifests++Each release publishes a small manifest alongside its changelog entry:++```yaml+version: 0.7.9+contracts:+ agent-protocol: { speaks: 1, accepts: 1 }+disturbance: next-session+atomic: true+reversible: true+```++| Field | Meaning | Source |+| --- | --- | --- |+| `contracts` | This release's `speaks`/`accepts` per contract | Generated by CI from the contract file |+| `disturbance` | What applying it costs — §6 | Declared; often derivable |+| `atomic` | Can a failed apply leave things worse than before? | A property of the installer, constant per artifact |+| `reversible` | Can it be rolled back? | Constant per artifact (switch-core: no) |++Only `disturbance` is a genuine per-release judgement, and even that is often+derivable: an empty contract delta on an artifact read at session start is+`next-session` by construction.++### 5.1 Manifests describe the *move*, not the version++`0.7.8 → 0.8.0` and `0.7.9 → 0.8.0` are different journeys and can have+different answers. Each release declares only what changed **since the release+before it**; a jump **folds** every release in between:++- `disturbance` → the worst on the path+- `reversible` → false if any step on the path was irreversible+- `contracts` → compare the endpoints; the ranges handle it+- `atomic` → unaffected by path length; one install is performed++N releases therefore require N declarations, not N² pairs. **Manifests for all+releases are retained**, not only the newest — a path cannot be folded if it+cannot be seen. A client jumping forward across three breaking releases is told+so, rather than being judged only against the latest.++---++## 6. Disturbance: what is lost, not what restarts++| Level | Meaning | Example |+| --- | --- | --- |+| `none` | Nothing running is affected | A plugin release that only edits its README |+| `next-session` | New sessions get it; running ones keep what they have | A skill-text edit; a new runtime pin |+| `restart-resumable` | The component restarts and everything reattaches | Sidecar upgrade; switch-core patch |+| `restart-visible` | Work resumes, but the user sees an interruption | switchdash app update |+| `restart-lossy` | Work is actually lost | Stack reset; a protocol bump that strands running sessions |++The boundary between `restart-resumable` and `restart-visible` is *does anyone+notice*. The boundary between `restart-visible` and `restart-lossy` is *is+anything lost*. Only the last is worth interrupting someone over.++**`restart-resumable` is not conditional on idleness.** A sidecar upgrade+reattaches by design: sessions live in tmux on the host, sidecar state is+persisted across restarts, and sessions are launched with the *path* to the+endpoint file rather than a port, precisely so a restarted sidecar with a fresh+port stays reachable from panes spawned against the previous process. The event+stream resumes from its cursor, and if it fell outside the buffer it emits a gap+frame rather than pretending to be caught up — so the worst case is *detected*,+not silent.++### 6.1 What happens to sessions on a `next-session` update++Nothing. No prompt, no injection, no restart — if running sessions had to be+restarted it would not be `next-session`.++What we do is **show** it: the directory is on plugin `0.7.9`, two sessions are+running `0.7.8`, displayed as *"2 sessions on the previous version — they pick+it up when they next start"*. They converge as sessions end.++This is the **installed vs running** distinction (§7) at its smallest scale. A+user-initiated "restart this session" affordance is reasonable — resuming a+Claude Code session preserves the conversation — but it is never automatic. The+one case worth actively nudging is when the running version is `deprecated`+rather than merely outdated.++---++## 7. Scope: everything is shared at a different granularity++Per location, the chain is linear:++```+sidecar → agent CLI (Claude Code / Codex) → plugin → runtime → switch-core+```++Each link talks only to the next, so N hosts is N copies of one small pattern+rather than a graph. But each link is **shared at a different granularity**, and+that — not the component count — is the real source of complexity:++| Link | Scope |+| --- | --- |+| switch-core | global |+| plugin + runtime | per host + user |+| sidecar bundle (the version) | per directory |+| sidecar process (what runs) | per agent |++The sidecar bundle is one file per repo directory, while every agent in that+directory runs its own process with its own state. So agents in one directory+share a version involuntarily; agents in different directories drift freely.++**Verdicts are computed per agent and rolled up** through directory and host,+worst state winning. That yields blast radius for free — *"this affects 3 agents+in this directory"* — because scope is modelled rather than derived by hand.++**Track what is installed separately from what is running.** They diverge for+exactly as long as an old process outlives an update, which is §8 in miniature.++### 7.1 Two views, two owners++- **switchdash** is the only component that sees every location → *"are my hosts+ consistent?"*+- **switch-core** is the only component that sees every connected client →+ *"who is talking to me, and on what?"*++Neither can answer the other's question, and neither needs to.++---++## 8. Updates while things are running++Immediately after any update the deployment is, by construction, running mixed+versions. The model must describe that state rather than assume it away.++### 8.1 In-flight sessions, per artifact++- **switch-core** — sessions keep running. This is what the `accepts` floor buys:+ a new core must reach back far enough to cover anything plausibly still alive.+ A session below the floor is refused at its **next reconnect**, with a message+ naming both versions. It is never killed mid-work, and never left running+ while silently dropping events.+- **switchdash** — restarts anyway; the interesting question is the stack it+ manages.+- **Plugin / runtime** — a running session keeps the runtime it started with; new+ sessions get the new one. Mixed by construction and correct.+- **Sidecar** — the existing verdict table (replace on incompatible major,+ reattach when the host has newer, upgrade otherwise) is retained and folded+ into this framework.+- **`db-schema`** — one-way. Forward at boot, no rollback.++### 8.2 The update verdict++The verdict is **computed from the folded manifest plus live state**, never+written by hand:++```+if a contract delta strands a currently-connected peer → BLOCKED+else if not atomic, or not reversible → MANUAL+else if disturbance <= restart-resumable → SILENT+else if restart-visible and nothing live → SILENT+else if restart-visible and something live → PROMPT+else → PROMPT+```++| Verdict | Surface |+| --- | --- |+| `SILENT` | Applied without asking, and recorded |+| `DEFER` | *"will apply when this host is idle"* — rare; only where waiting saves something |+| `PROMPT` | *"this will interrupt 3 sessions"*, with the real count |+| `BLOCKED` | *"needs switch-core 0.14 first"* — names what must move |+| `MANUAL` | Cannot be automated, and why |++Nobody labels a release "silent". Codex plugin updates come out `MANUAL` purely+because that installer has no update verb and does uninstall-then-reinstall — a+failure leaves the agent with **no connector at all**. Making that path atomic+turns it silent automatically, with no policy change.++### 8.3 Safety is computed; autonomy is chosen++`SILENT` is a **ceiling, not an instruction**. Whether we act on it is a separate+per-artifact policy decision. An artifact may be more conservative than its+verdict allows; never less.++> Manage silently the things the user does not know exist. Ask about the things+> they do.++| Artifact | Policy |+| --- | --- |+| Sidecar | Auto-apply. Its version should be invisible |+| Runtime | Auto-apply |+| Plugin | Auto-apply, **recorded** — skill text changes agent behaviour, so it should not need approval but must be findable afterwards |+| switchdash | **Never auto.** It is the supervisor — the component that applies everyone else's updates — and the UI the user is looking at. The verdict gates and annotates the offer instead |+| switch-core | **Never auto.** Data, and one-way migrations |++**Silent is not invisible.** Silent means no prompt, not no record.++This closes a live footgun: switchdash's updater currently applies a build whose+pinned core is *older* than the running stack, which then refuses to start and+tells the user to update switchdash. Under this rule that update is `BLOCKED`,+with a reason, before anything is clicked.++### 8.4 Safe update sequence++Update the depended-upon before the depender: **switch-core, then switchdash,+then the runtime and plugins**, with the sidecar riding switchdash. A server's+`accepts` floor is what covers old clients; an old server cannot know anything+about a new client.++Rollback is safe anywhere in that order **except across a schema migration**,+where the model must refuse rather than warn.++---++## 9. Surfacing++None of this is worth building if it only reaches the logs.++- **switch-core records what each client declared** — artifact version and+ contract versions, per connection. This is the substrate for everything else.+- **Gateway compatibility view** — every connected client, what it speaks, and+ its state; plus the deployment summary: *"3 sessions on `agent-protocol` 1,+ server speaks 2, floor 1."*+- **"What would raising this floor break?"** — answerable before doing it,+ instead of afterwards.+- **switchdash** extends its existing drift notice to servers it does not manage+ via `GET /version`, and stops rendering "could not read" as "no drift".+- **Update availability** is annotated with its verdict, so *"an update exists"*+ and *"this update will interrupt your work"* are visibly different.++---++## 10. Two defaults that carry most of the weight++1. **Unknown never renders as fine.** If a version cannot be read or was not+ declared, say so. (§1.3 lists four places that currently violate this.)+2. **Undeclared never means safe.** A release with no manifest, or a missing+ field, is treated as the most disruptive class and prompts. The failure to+ avoid is someone forgetting to declare and receiving silent auto-updates as a+ reward.++---++## 11. Risks++### 11.1 Renumbering the sidecar++The sidecar is the only artifact whose version *currently is* its compatibility+signal. Moving that into `sidecar-control` is correct, but switchdash installs+already in the field still judge on the major.++**The major must stay at 1** (`1.7` → `1.7.0`). An old client parses major 1+either way. Going to `2.0.0` would make every existing switchdash read the new+sidecar as incompatible — reproducing CHOO-1937, where two installs sharing a+host replace each other indefinitely.++### 11.2 A forgotten counter++The model is only as good as our discipline in incrementing a contract when we+change it. A missed bump means the system reports "compatible" for something+that is not.++Mitigation: CI flags a change to an interface-defining file with no+corresponding counter change. This is a prompt, not a proof, and it is the+honest cost of keeping artifacts independently releasable.++### 11.3 Deploy lock scope (to verify)++`sidecarDeployLockRelPath()` is keyed per **agent**, while the bundle it guards+is shared per **directory**. Two agents in one directory take different locks and+write the same file. Upload is temp-file-then-rename, so nothing tears — but two+switchdash installs with differing bundles could alternate overwriting each+other, which is the shape of CHOO-1937 one level down. Needs verification rather+than assumption.++---++## 12. Rejected alternatives++- **A central matrix of version tuples.** Rots on contact: cannot express "any+ 0.12.x", needs an edit per release per artifact, and goes stale silently —+ the failure mode this work exists to end. Retained only as the non-blocking+ `untested` layer, CI-generated, never hand-written.+- **Semver ranges between artifact versions** (switchdash declares+ `switch-core >=0.12 <0.14`). What CHOO-1321 would have produced. Couples+ release cadence to compatibility and misstates reality in both directions: it+ forces a bump when no interface moved, and stays silent when a patch breaks+ the wire.+- **A single lockstep release train.** Genuinely simpler — one version,+ everything ships together, compatible means equal. Rejected because it ends+ how these artifacts are actually shipped (the plugin releases by merging to+ main; the runtime publishes from a branch so it can be tested without+ merging), and forces version bumps on artifacts that did not change.+- **Exact-match protocol equality** — today's behaviour. No overlap window at+ all, so every protocol change is a flag day. Not a trade-off; a defect.+- **The client asking the server "am I compatible?" as the only mechanism.**+ Fails exactly when the two cannot talk, and cannot gate an update before it is+ installed. Useful as a supplement, not as the base.++---++## 13. Deferred++**The deprecation-window policy** — how long a contract version must be accepted+before its code path is deleted — is layered on top of this model and does not+change its core. It is deferred to a follow-up.++One part is **not** deferrable and is included here: the `accepts` floor must+exist in the declaration and be recorded server-side from the outset. It is the+mechanism the policy will later act on, and without it the question *"who would+raising this floor break?"* cannot be asked at all.++Sketch, for later: a floor rises only after the dropped version was marked+deprecated for at least one minor cycle and nothing observable is still below+it, with a published calendar backstop for deployments we cannot observe; the+end is a hard refusal, not an override; and deprecation is announced to the+specific affected client while it still works.++---++## 14. Implementation plan++Four layers, in dependency order. Each is useful on its own.++**A — make versions legible.**+switch-core learns its own version; `GET /version`; the contract file plus+generated Python and TypeScript with a CI equality check; clients declare on+connect; the server records what they declared. Fix the four defects in §1.3.++**B — enforce ranges.**+Exact-match becomes range overlap; refusal messages name both sides and the+remedy; switchdash gains drift detection for servers it does not manage.++**C — make it visible.**+Gateway compatibility view; mixed-version summary; the "what would raising this+floor break" query.++**D — update safety.**+Release manifests and path folding; the verdict engine; per-artifact autonomy+policy; the switchdash updater gate; atomic Codex reinstall; CI-generated tested+combinations; the documented update sequence.++Splitting by layer is not the per-pair fragmentation CHOO-1865 forbids — it is+one model landing in reviewable pieces. Proposed split: **A + B** in one PR,+**C + D** in a second.