Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
docs(publishing): draft three AWS Builder Center articles and two IBM Community posts
nirholas431bbb66743f
diff --git a/docs/aws-builder-center-claims-ledger.md b/docs/aws-builder-center-claims-ledger.mdnew file mode 100644index 000000000..3eb370d57--- /dev/null+++ b/docs/aws-builder-center-claims-ledger.md@@ -0,0 +1,440 @@+---+title: "The claims ledger: an AI-assisted publishing pipeline that refuses to post what it cannot prove | three.ws on AWS"+venue: AWS Builder Center+account: three.ws (official organization account, byline "three.ws")+status: draft, owner approval required before publishing (external-channel gate in CLAUDE.md)+description: "How three.ws runs its X account from a reviewed queue where a language model may draft and edit, but deterministic code decides what is true: a claims ledger that holds every number and absolute word to checked evidence, probes that prove a feature works, proof reels filmed against production, a model failover chain, review records bound to a content hash, an idempotent publish ledger, and a learning loop that only changes the order posts go out in."+tags: [generative-ai, agentic-ai, llm, automation, open-source]+index: docs/aws-builder-center.md+---++# The claims ledger: an AI-assisted publishing pipeline that refuses to post what it cannot prove++A language model can write a competent product announcement in seconds. That is no longer the hard part of running a company account. The hard part is that a fluent, wrong sentence from a company account costs more than no sentence, and fluent sentences whose relationship to the product is approximate are exactly what a model produces when nothing holds it to the product.++We run the [@trythreews](https://x.com/trythreews) account for [three.ws](https://three.ws), an open-source (Apache-2.0) platform for 3D AI agents, from a reviewed queue. A model drafts posts from an evidence brief, and a model reviews every post before it can be approved. Neither decides whether a post is true. The principle, stated once so the rest of this article can be about mechanism: **a model may draft and edit; deterministic code decides what is true, what is approved, when it posts, and whether it already posted.** Every claim below points at readable source in [the repository](https://github.com/nirholas/three.ws), because an article about refusing unverifiable claims should be checkable in the same sitting.++**Status, plainly, because AWS builders check.** three.ws is a verified AWS Partner. This pipeline does not run on AWS: the platform's runtime is on Google Cloud Run, the publishing tick is a Cloud Scheduler job calling an HTTP handler every 15 minutes, and the ledger lives in Postgres. The model chain's providers are exactly the five that [`api/_lib/x-content/llm.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/llm.js) lists: Claude on Vertex AI, gpt-oss-120b on Groq, Claude through OpenRouter, OpenAI, and Kimi K3 on NVIDIA NIM. There is no Amazon Bedrock rung today. Nothing in the design depends on where it runs, and section 14 maps the pieces onto AWS services for anyone lifting them.++**Contents**++1. The gap a fact check leaves open+2. The pipeline in one picture+3. The claims ledger+4. Probes: proving the feature works, not that the page loads+5. Proof reels: a camera that cannot film a failure+6. Where the models sit, and what they may not decide+7. The model chain, and what failover looked like in practice+8. The voice lint: cheap, deterministic, and sometimes wrong+9. Review records bound to a content hash+10. The publish ledger: retries that cannot double-post+11. An unreadable schedule, and the approval policy+12. The learning loop, and the limits of what it learned+13. What we would build differently+14. What to lift from this+15. Try it++---++## 1. The gap a fact check leaves open++The first version of this pipeline had a good fact check. Every number in a post had to quote a page on the live site, and the check confirmed the quote was there. It caught real errors: one early post said a tool recognised 11 rig conventions when the live page and the code both said 15.++Posts were still taken down, and the pipeline's documentation records the uncomfortable pattern: every post the owner had taken down had passed the fact check. The copy matched the page. The feature did not live up to the page. A quote proves that a page *says* something; it cannot prove that the product *does* it.++The design that follows closes that gap in layers, cheapest first: an offline lint that costs nothing, live evidence checks that cost a browser, feature probes and filmed runs that cost minutes, and a model review that costs a model call. A post that fails a cheap layer never pays for an expensive one, and a post that passes gets a record bound to the exact bytes that passed.++## 2. The pipeline in one picture++```+story file (scenario + copy + claims)+ | prove: run the scenario against production, film it, write a proof record+ v+queue item --> check: voice lint, editorial lint, claims ledger (offline)+ |+ | review: live evidence, probes, links, mentions, spelling, AI editor+ v+review record (bound to a SHA-256 of the item) --> approval: owner, or policy+ |+ v+scheduler tick (every 15 min) --> pre-flight: links + API probes again+ |+ v+publisher --> publish ledger (every X id written the moment it exists)+ |+ v+outcomes: read the account's own posts back, learn, re-rank the queue+```++The queue is one JSON file, [`data/x-content/queue.json`](https://github.com/nirholas/three.ws/blob/main/data/x-content/queue.json). On 8 October 2026 it held 54 items (45 posts and 9 long-form X Articles) carrying 678 declared claims, 1,031 pieces of evidence, and 127 feature probes. The operator CLI and the production cron call the same validator, so "passes locally" and "publishable in production" are one rule.++## 3. The claims ledger++Every number, ordinal and absolute word in the copy must sit inside a declared **claim**. A claim quotes the copy and carries at least one piece of **evidence** a program can re-check. The detector for absolute words is one regular expression in [`api/_lib/x-content/editorial.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/editorial.js):++```js+export const ABSOLUTES = /\b(?:first|only|fastest|largest|biggest|best|leading|never|always|every|no one|nobody|zero|unlimited|instant(?:ly)?|world['’]s|#1|number one)\b/gi;+```++Numbers are found the same way, with exceptions for names that contain digits ("3D", "GPT-5"). Then the ledger is checked in both directions:++```js+const covered = (fragment) => claims.some((claim) => String(claim.says || '').toLowerCase().includes(fragment.toLowerCase()));+for (const text of texts) {+ const { numbers, absolutes } = assertionsIn(text);+ for (const number of numbers) if (!covered(number)) problems.push({ rule: 'claims', severity: 'blocking', message: `"${number}" is an unverified number; declare it in claims with evidence` });+ for (const word of absolutes) if (!covered(word)) problems.push({ rule: 'claims', severity: 'blocking', message: `"${word}" is an absolute; declare the claim it makes, with evidence, or cut it` });+}+```++An uncovered number is blocking. So is a claim whose `says` is not in the copy, and a claim with no evidence. The second direction matters more than it looks: without it, the verified claim can stay in the ledger while the copy quietly says something else.++Here is a real claim from the Machine Atlas post (a Labs page that builds a radial engine from code). The copy says "Nine cylinders to five", and the claim cites two facts a filmed run read off the page:++```json+{+ "says": "Nine cylinders to five",+ "evidence": [+ { "type": "proof", "fact": "cylinders", "equals": "9" },+ { "type": "proof", "fact": "cylindersAfter", "equals": "5" }+ ]+}+```++[`api/_lib/x-content/verify.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/verify.js) checks each evidence type at the moment of review:++| `type` | Passes when |+|---|---|+| `page` | The live page, rendered in a real browser, contains the quoted text |+| `file` | A repository file contains the text, or matches a regular expression |+| `module` | A module's export has a stated length, or equals a stated value |+| `github-issue` / `github-issues` | An issue has a stated state and label, or a repository has at least N matching issues |+| `proof` | The filmed run read the fact off the screen, waited for a text, or got an answer from a named request |++The `page` check renders the page in Chromium through Playwright, waits for network idle, then polls once a second until the body text stops growing, because a client-rendered page can still be empty at network idle. The match is word for word after collapsing whitespace and ignoring case, and nothing looser: no fuzzy matching, no embeddings, no model in the loop.++```js+case 'page': {+ const { status, text } = await pages.text(evidence.url);+ if (status < 200 || status >= 300) return { ...base, ok: false, detail: `page answered HTTP ${status}` };+ const found = text.includes(normalize(evidence.contains));+ return { ...base, ok: found, detail: found ? `live page shows "${evidence.contains}"` : `live page does not show "${evidence.contains}"` };+}+```++## 4. Probes: proving the feature works, not that the page loads++A claim proves the copy. A **probe** proves the feature. A post with no probe fails review with "no feature probe declared; add one to `probes` that proves the feature works, not just that its page loads". Four kinds exist:++| `type` | Passes when | Runs |+|---|---|---|+| `api` | A URL answers 2xx (or an expected status), optionally with a text or a JSON value at a path | at review, and again seconds before publishing |+| `browser` | A scripted session (`goto`, `click`, `expect`) reaches the expected text | at review |+| `command` | A repository test that exercises the claimed behaviour exits 0 | at review |+| `scenario` | The post's filmed scenario passes against production again, and its facts still match | at review |++The `api` probe is cheap and dependency-free enough to run in production, which is why it runs twice. A post that states a count from a live catalog carries a probe on the catalog's JSON length; if the count moved after review, the pre-flight fails and the post is held instead of published wrong.++One lesson from running this: click by exact text before partial text. A step that said "Search" once resolved to a "Clear search" button earlier in the page, and the run filmed the query being wiped instead of run.++## 5. Proof reels: a camera that cannot film a failure++The strongest probe is the one whose output is also the post's media. A **scenario** is a list of steps a person would take on the live site: go to a page, type, click a control by its text, drag across a 3D scene, wait for a text that only appears on success, and read a fact off the screen with a pattern. [`api/_lib/x-content/reel.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/reel.js) runs those steps against production and films them, so one list is the probe, the video, the captions and the evidence at once. If a step fails, no reel is encoded, and a post without its reel does not validate.++**The page clock is stepped, not recorded.** The filming machines have no GPU, and a headless browser renders WebGL in software at one to three frames a second; a screen recorder pads that by repeating frames, which reads as a slide show. So the runner pauses Playwright's page clock and advances it one frame at a time:++```js+async frame() {+ await this.pause();+ await this.page.clock.runFor(this.frameMs);+ if (this.badgeFrames > 0 && --this.badgeFrames === 0) await this.bar.set({ badge: '' });+ if (!this.film) return;+ const shot = await this.page.screenshot({ type: 'png', scale: 'device', timeout: 60_000 });+ // Motion is measured on the product alone, so a caption changing never+ // counts as the feature doing something.+ const hash = createHash('sha1').update(shot).digest('hex');+ if (this.lastHash && hash !== this.lastHash) this.changes++;+ this.lastHash = hash;+ const frame = await this.compose(shot, this.bar.shot, this.size);+ writeFileSync(join(this.framesDir, `f${String(++this.count).padStart(6, '0')}.jpg`), frame);+}+```++Every frame is distinct and exactly 1/30 s of page time apart; the machine's speed only decides how long filming takes. The Machine Atlas reel is 477 frames and 15.9 seconds, and one drag step in it filmed 84 frames in 55.2 seconds of wall time. A browser flag made this practical: with the page composited through the same software Vulkan device as the 3D scene, one frame of a plain list page measured 3.5 to 15 seconds, and with GPU compositing and rasterization disabled it takes about 250 ms while WebGL still runs on SwiftShader.++**Waiting is cut and labelled.** Network waits run on the real clock with the camera off. A wait of 2 seconds or more puts a badge such as `cut 6 s` in the bar for the next 45 frames, so a slow feature is shown as slow. The Drive reel, where an agent answers a spoken question, records cuts of 3.9 and 6.3 seconds and a chat request that answered 200 in 6.8 seconds.++**Nothing is drawn over the product.** Captions, the badge, and a stamp reading `live on three.ws @ <commit>, <date>` sit in a bar under the page. The commit comes from the site's own [`/api/version`](https://three.ws/api/version) endpoint at filming time. Only the pointer is drawn on the page.++**The proof is bound to the post.** The proof record stores hashes of the scenario and of the encoded file, and production checks them without a browser:++```js+if (proof.scenarioHash !== scenarioHash(item.scenario)) problems.push('the scenario changed after it was filmed; prove it again');+const age = (now - Date.parse(proof.ranAt)) / 86_400_000;+if (age > PROOF_MAX_AGE_DAYS) problems.push(`the reel was filmed ${Math.floor(age)} days ago; the product moves, so film it again`);+```++Change a step, swap the file, or let 14 days pass, and the item no longer validates. A reel in which fewer than 15% of frames change is refused too: it is a still with a progress bar, and a still image is the honest format for it.++**Reviews run it again, and hold each fact to how the copy used it.** A fact cited exactly must read today what the reel shows. A fact cited as a floor (`min`) only has to stay at or above it, so a growing count does not send a true post back to be filmed. A fact no claim cites may move freely, because the reel is stamped with its day.++```js+for (const [name, rule] of Object.entries(factRules(item))) {+ const filmed = proof?.facts?.[name];+ const live = facts[name];+ if (rule.exact && live !== filmed) problems.push(`${name} is now "${live}", the reel shows "${filmed}"`);+ if (Number.isFinite(rule.min) && !(factNumber(live) >= rule.min)) problems.push(`${name} is now "${live}", under the floor of ${rule.min} the post claims`);+}+```++That exact-versus-floor distinction is the most reusable idea here. It forces the author to decide whether a number is a fact about today or a promise about a minimum, and the checker holds them to that decision.++## 6. Where the models sit, and what they may not decide++Two jobs call a model, and both are fenced the same way: the model proposes, deterministic code judges.++**The drafter** ([`api/_lib/announce/draft.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/announce/draft.js)) writes a post from an evidence brief: lines harvested from the live page and from repository files before a word is written. Its claims may cite only those candidates, copied exactly, and code enforces it, because a paraphrased candidate is a claim the verifier will fail on the live page:++```js+const allowed = new Set((brief.evidenceCandidates || []).map(candidateKey));+for (const claim of draft.claims) {+ for (const evidence of claim.evidence || []) {+ if (!allowed.has(candidateKey(evidence))) {+ findings.push(`claims: evidence ${JSON.stringify(evidence).slice(0, 120)} is not one of the brief's evidenceCandidates; copy a candidate exactly or drop the claim`);+ }+ }+}+```++Each draft runs through the queue's own lints and claims ledger. Findings go back to the model verbatim, up to 3 attempts. A draft that still fails is returned with its findings rather than written into the queue; the module's comment says it "never decides that something is good enough". The drafter cannot invent a fact, only choose among facts a program already harvested.++**The editor** ([`api/_lib/x-content/editor.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/editor.js)) judges what a lint cannot: whether the first line earns the second, whether the image shows what the copy claims, whether a sentence is literally true but misleading. It sees the copy, images, alt text, the claims with every live verification result, the house voice contract, and the account's best-measured posts for calibration. It cannot watch video, so for a reel it gets four still frames, the captions, and the run record, and is asked whether each caption is true of its frame. It returns a verdict, six scores from 1 to 5 (accuracy, clarity, specificity, voice, professionalism, visual), quoted issues with fixes, and a rewrite. The model does not get the last word on its own verdict:++```js+// The model does not get the last word on its own verdict: a blocking issue+// or a low score always means revise.+if (review.verdict === 'publish' && (review.issues.some((issue) => issue.severity === 'blocking') || SCORE_KEYS.some((key) => review.scores[key] < 4))) {+ review.verdict = 'revise';+}+```++Its rewrite is linted like any draft, with every uncovered number or absolute listed for the human. That rule has a history: an early AI rewrite upgraded our partner credential to stronger wording than the evidence for that post supported, and the review bar caught it. A model that "improves" copy reaches for the more impressive phrase that still sounds true. The editor's prompt now names that exact upgrade as forbidden, and the pipeline never applies a rewrite: the editor never edits the queue, and a person adopts its suggestion or does not.++The asymmetry is deliberate: **the editor can add a blocker, but it cannot remove one.** A failed fact check or blocking lint finding fails the review whatever the model says. A human who disagrees with a `revise` that carries no blocking issue can record an `editorOverride` with a reason, and the approval policy (section 11) then refuses to release that post without a person.++## 7. The model chain, and what failover looked like in practice++Both jobs share one transport, an ordered list of rungs:++```js+export function modelRungs(request, env = process.env) {+ return [+ () => viaVertex(request),+ () => viaChatCompletions(request, { url: 'https://api.groq.com/openai/v1/chat/completions', key: env.GROQ_API_KEY, model: 'openai/gpt-oss-120b', label: 'groq', textOnly: true }),+ () => viaChatCompletions(request, { url: 'https://openrouter.ai/api/v1/chat/completions', key: env.OPENROUTER_API_KEY, model: `anthropic/${EDITOR_MODEL}`, label: 'openrouter', extraHeaders: { 'http-referer': 'https://three.ws', 'x-title': 'three.ws editorial review' } }),+ () => viaChatCompletions(request, { url: 'https://api.openai.com/v1/chat/completions', key: env.OPENAI_API_KEY, model: 'gpt-5.5-pro', label: 'openai' }),+ () => viaChatCompletions(request, { url: 'https://integrate.api.nvidia.com/v1/chat/completions', key: env.NVIDIA_API_KEY, model: 'moonshotai/kimi-k3', label: 'nvidia', maxTokens: RUNG_MAX_TOKENS }),+ ];+}+```++Four rules make it behave. **A rung with no credentials returns `null` and is skipped.** **A text-only rung refuses a request that carries an image**: Groq sits high because it is fast and serves the drafter's text brief, while a review that must see its media skips it and so never gets an image-blind verdict from that rung. **A parse failure is a rung failure**: the caller passes a `parse` function, and prose where JSON was requested falls through to the next rung. **Retries are classified, not blind**: a dropped connection, a 5xx or a plain 429 gets 3 attempts with a growing pause, while a 429 about billing will only repeat, so it falls through at once:++```js+export function isRetryable({ status = null, body = '', networkError = false } = {}) {+ if (networkError) return true;+ if (status >= 500) return true;+ return status === 429 && !/billing|credit|quota|insufficient|not active/i.test(String(body));+}+```++Every review record names the model that wrote it and the failure of every rung above it, which turned out to be the most useful field in the record. All 52 review records committed on 8 October name the last rung, Kimi K3 on NVIDIA NIM. Every one lists OpenRouter answering 402 (out of credits) and OpenAI answering 429 with `billing_not_active`; 32 list Vertex answering 403 for insufficient authentication scopes, and the other 20 found no Google Cloud credentials on the reviewing machine. The bar never stopped, because the chain did its job. The chain also hid, for the three weeks those records span, that three of its four paid rungs were unusable. Section 13 returns to that.++## 8. The voice lint: cheap, deterministic, and sometimes wrong++Formatting rules do not make a post true, but they keep a feed from reading as generated, and they cost nothing. [`quality.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/quality.js) rejects launch-deck openers ("Introducing", "We're excited"), hype phrases, hashtags, emoji, en and em dashes, stacked exclamation marks, and shouting. [`editorial.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/editorial.js) adds brand spelling, anything that reads as a price promise or investment pitch, slang, marketing filler, engagement begging, rhetorical-question openers, more than one link, and media rules (images under 1200 px wide, alt text under 40 characters).++Two rules came from the platform rather than taste. Length is measured as X measures it: every URL counts 23 characters, including a bare domain X decides to link, a lesson from the first test post, where X linked the brand name "three.ws" and created a second link the one-link rule never saw. And X refuses a post with more than one cashtag with a 403 no retry can fix, so that rule moved left into the lint, where it fails at review with a message saying what to write instead.++The honest part: a regular-expression lint has false positives. The engagement-ask list bans "drop a" (as in "drop a comment"), and the same rule blocks "Drop a .glb on Rig Doctor", the sentence our own pipeline documentation uses as its example post. "Awesome" is banned as filler, which made a post about a surface named Awesome 3D Agents unwritable until the rule learned to ban the adjective and keep the proper noun. Deterministic rules are worth their false positives, because a false positive is visible, specific and fixable in a one-line diff, while a model's inconsistent judgment is none of those. They need an escape hatch with a paper trail: the glossary for spelling, the editor override for judgment.++## 9. Review records bound to a content hash++`npm run x:content -- review <id>` runs every layer and writes `data/x-content/reviews/<id>.json`. An item can only be `approved` while a passing record exists whose hash matches the item as it is now. From [`api/_lib/x-content/review.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/review.js):++```js+const subject = {+ kind: item.kind,+ posts: (item.posts || []).map((post) => ({ text: String(post.text || '').trim(), media: (post.media || []).map(media) })),+ article: item.kind === 'article'+ ? { title: item.article?.title, body: fileHash(root, item.article?.body), cover: item.article?.cover ? media(item.article.cover) : null, images: articleImages(root, item).map(media) }+ : null,+ claims: item.claims || [],+ mentions: item.mentions || {},+ probes: item.probes || [],+ // Absent on an item with no scenario, so the hash of every item reviewed+ // before scenarios existed is unchanged.+ scenario: item.scenario,+};+return createHash('sha256').update(JSON.stringify(subject)).digest('hex');+```++`media` hashes each file's bytes with its alt text and video probe, so swapping an image is as visible as editing a word, and an Article's inline images are hashed even though the Markdown that names them did not change. The `scenario` comment is a lesson of its own: adding a field to a hashed subject silently invalidates every record ever written, unless the field is absent when unused.++A record also expires after 14 days, because facts drift, and the production cron enforces both rules from the records shipped in the container image. On 8 October, `npm run x:content:check` refused two approved posts outright because their reviews were 17 and 19 days old; a scheduled tick skips them the same way. The hash also drives holds: a post that fails pre-flight is held against its hash, and any edit releases the hold at once, because a new hash means somebody fixed something.++## 10. The publish ledger: retries that cannot double-post++The most embarrassing automation failure on a social account is a thread posted twice because a job retried. The publisher writes every identifier X returns to a ledger the moment it exists, and consults it before every call. From [`api/_lib/x-content/publisher.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/publisher.js):++```js+for (let index = 0; index < posts.length; index++) {+ if (progress.postIds[index]) continue;+ const post = posts[index];+ const mediaIds = [];+ for (const media of post.media || []) mediaIds.push(await uploadOnce({ client, root, media, progress, persist }));+ const payload = { text: post.text.trim() };+ if (mediaIds.length) payload.media = { media_ids: mediaIds };+ const replyTo = index === 0 ? null : progress.postIds[index - 1];+ if (replyTo) payload.reply = { in_reply_to_tweet_id: replyTo };+ if (index === 0 && quoteId) payload.quote_tweet_id = quoteId;+ const { data } = await client.tweet(payload);+ progress.postIds[index] = data.id;+ await persist();+}+```++A crash between the third and fourth reply resumes at the fourth, threaded under the third's real id. Media ids are cached and re-uploaded only after 23 hours, inside X's 24-hour expiry. An Article's draft id and published id are persisted separately, so a crash between the two never creates a second draft. A half-published item resumes ahead of every pacing rule on the next tick and skips pre-flight, because finishing what was started matters more than re-checking it.++The ledger is one row in a Postgres `app_settings` table, shared by the cron and the CLI, and both take the same lease lock:++```sql+INSERT INTO app_settings (key, value)+VALUES (${LOCK_KEY}, jsonb_build_object('until', extract(epoch from now()) + ${LOCK_TTL_S}))+ON CONFLICT (key) DO UPDATE+ SET value = excluded.value, updated_at = now()+ WHERE (app_settings.value->>'until')::numeric < extract(epoch from now())+RETURNING key+```++The `WHERE` on the conflict branch lets a caller take the row only when the previous lease has expired, and `RETURNING` says whether it won. The lease is 900 seconds, longer than the slowest video processing. The CLI refuses to publish without a database connection, because a publish with no shared ledger is the one way to get a double post.++Failures are classified like the model chain's. A post-specific refusal (a 4xx such as a duplicate) holds that post for 2 hours, then 6, then 24, and the slot goes to the next-best post in the same tick, up to 5 attempts. A refusal of the account (401, 402 for spent API credits, or a non-duplicate 403) stops the tick, because every post would fail the same way, and raises one critical alert a day naming what a person must fix.++## 11. An unreadable schedule, and the approval policy++The queue is a committed file in a public repository, so the day an item is eligible is public by construction. If the posting minute were a plain hash of the item id, it would be public too, and an announcement whose minute is knowable days ahead can be camped. So the jitter is an HMAC under a seed that exists only in production:++```js+export function jitterMinutes(id, windowMinutes, seed = null) {+ const digest = seed+ ? createHmac('sha256', String(seed)).update(String(id)).digest()+ : createHash('sha256').update(String(id)).digest();+ return digest.readUInt32BE(0) % Math.max(1, windowMinutes);+}+```++The HMAC is stable per slot, so a preview, a retry and the real tick agree on the minute, while nobody without the seed can compute it. Without the seed it falls back to the public hash, which keeps tests deterministic, and the planning command says its minutes are placeholders. Which post fills a slot is decided when the slot opens, so a correct guess at the minute would not say which post goes out. The account has three slots a day, one per tier (flagship, feature, proof of work), plus a slot for long-form X Articles every second day.++**Who approves** is [`api/_lib/x-content/approval.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/approval.js). In `owner` mode every post waits for a person. In `auto` mode a post is released without one only when all of these hold:++| Condition | Why |+|---|---|+| Its tier is listed in the policy | Flagship posts carry a partner's name or $THREE, so tier 1 stays with the owner unless the queue lists it |+| It was filmed, and the proof covers it | A post never proven against the product is never released by policy |+| It tags no one | A tag puts a partner's name next to ours, and a person decides that |+| The review passed on the editor's own verdict | An override is a person's judgment, so it needs a person |++X Articles have their own switch: long-form writing rarely has a reel to film, so the review stands in for it (every probe re-run, every link resolved, every sentence of the body held to the claims ledger), while the tag, verdict and proof rules still apply. A release by policy is never immediate: the post waits out a veto window (the code's default is 24 hours; the queue sets its own), the release is written to the ops alerts, and one command takes it back. Human attention goes to tags, flagship claims and disagreements with the editor, not to re-reading a filmed, verified, linted post.++## 12. The learning loop, and the limits of what it learned++The pipeline's original style rules came from a scrape taken before X had loaded the like counts of the account's largest posts. It concluded that video loses to a still and that 100 to 179 characters is the best length. The X API said the opposite on both. So the queue now measures itself.++After a publishing tick, at most once every 6 hours, [`api/_lib/x-content/outcomes.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/x-content/outcomes.js) reads the account's head posts back through the X API, describes each by attributes a draft can be checked for in advance (media kind, length band, link placement, thread, tags), and learns what each was worth over 90 days. The outcome is a log, because a post with 400 likes is not 100 times the post with 4:++```js+export function outcomeOf(row) {+ return Math.log1p(row.likes || 0) + 0.5 * Math.log1p(row.bookmarks || 0) + 0.25 * Math.log1p(row.reposts || 0);+}+```++The guard rails are the part to copy. A post counts once it is 48 hours old, because likes arrive for about two days. Outcomes are recency-weighted with a 45-day half-life. Each attribute's measured difference is shrunk by `n / (n + 8)`, and a value seen on fewer than 3 posts moves no score. The learned part is ignored until 20 posts have matured and is clamped to between -15 and +15 ranking points, so it cannot overrule the owner's explicit boost. Overlapping attributes are not summed: the estimate is the strongest reason for, the strongest against, and the mean of the rest. Because X bills each post a read returns, a refresh only re-reads as far back as metrics can still move (7 days before the last read).++A second model ranks by a different measured outcome: [`data/x-content/volume-model.json`](https://github.com/nirholas/three.ws/blob/main/data/x-content/volume-model.json) is a logistic regression fitted on 313 post moments, 54 of which were followed by a volume response on the $THREE pool. Read honestly, it is weak evidence: of its 9 features, only three (announcing a shipped thing, posting inside US working hours, posting from the company account) have confidence intervals that exclude no effect. It also rewards the word "introducing", which the voice lint bans as an opener. The lint wins, because it is a blocker and the model is a ranking term.++The boundary that matters: no deterministic check reads any of this. The learned lifts and the volume model change which ready post fills a slot, and the best-measured posts are what the editor sees as calibration. A learned preference can reorder true posts; it cannot make a false one publishable.++## 13. What we would build differently++**Alert on degraded failover, not only on total failure.** The chain never stopped the bar, which is why nobody noticed that all 52 reviews ran on the last rung while three paid providers refused us. The records named the model the whole time, and nothing read that field. A chain should alert when the rung that answered is not the rung that should have.++**Keep time-dependent truth out of unit tests.** The suite has a test asserting the committed queue is valid for every approved item. On 8 October it fails, because an approved item's review turned 19 days old. That is the 14-day expiry working as designed, and also a red build in a shared repository that nobody's change caused. Freshness belongs in an operational check with an alert; the unit test should pin `now`.++**Put exact-versus-floor in the schema from day one.** We added `min` evidence after counts that only grow kept sending true posts back to be filmed. If numbers will be held to live data, decide per number whether it is a snapshot or a floor.++**Give every regular-expression rule a corpus of sentences it must accept**, starting with every example in your own docs. The "drop a" false positive sat in the lint while the documentation's example post used the phrase.++**Bind facts to the run, not the page.** Of our 1,031 evidence entries, 519 are file evidence and 410 are page quotes; 95 are facts from a filmed run. Page and file evidence prove the copy matches something we wrote. Only run evidence proves the product did it, and the mix still leans toward the weaker kind.++## 14. What to lift from this++All Apache-2.0, none of it tied to the rest of the platform:++1. **The claims ledger as a schema.** A list of `{ says, evidence[] }`, checked in both directions, with one regular expression for absolute words. It fits release notes, changelogs, docs and sales copy as well as social posts, and it is the cheapest guard rail to put behind a generative step: a model drafting into it can be told its failures verbatim and asked again.+2. **The asymmetric judge.** The model may add blockers and never remove them; code demotes its `publish` when its own scores or issues disagree; its rewrites are linted like any draft.+3. **The model chain as a list of functions.** Each rung returns `null` when it cannot serve a request, throws on failure, and the caller's parser decides whether a reply counts. An Amazon Bedrock rung would be one more function calling the Bedrock runtime; we do not run one today.+4. **Content-hash approval with an expiry.** Hash every byte a reader or a fact depends on, store it in the approval, refuse to act on a mismatch.+5. **The write-ahead publish ledger.** Persist every external id the moment it exists, check before each call, and take a lease lock with a conditional upsert.+6. **Frame-stepped capture.** If you film anything rendered in a headless browser, take over the page clock instead of recording real time.++The tick is a stateless HTTP handler behind a scheduler, a lock row, and files in a container image (with signed, content-addressed bundles laid over them so a post approved after a deploy can still ship). On AWS the same shape would be an EventBridge Scheduler schedule invoking a Lambda function or a container, with the ledger in any transactional database. We have not built that variant, so treat it as a mapping, not a recipe.++## 15. Try it++Everything below is read-only and runs in a fresh clone. None of it posts, edits the queue, or needs an X account.++```bash+git clone https://github.com/nirholas/three.ws && cd three.ws && npm install++# 1. Validate the whole queue offline: voice lint, editorial lint, claims ledger,+# proof bindings, and review-record hashes and ages. Exits 1 and prints each+# problem if any item under review or approved has one.+npm run x:content:check++# 2. Run the claims ledger and the voice lint on a draft of your own.+node --input-type=module -e '+import { lintItem } from "./api/_lib/x-content/review.js";+const item = {+ id: "try-it", kind: "post",+ posts: [{ text: "Introducing the fastest 3D viewer ever: it loads 40 models in 2 seconds. three.ws/viewer" }],+ claims: [],+};+for (const f of lintItem(item, { maximum: 1000 })) console.log(f.severity.padEnd(8), f.rule.padEnd(10), f.message);+'++# 3. Watch the content hash change when one word changes.+node --input-type=module -e '+import { contentHash } from "./api/_lib/x-content/review.js";+const a = { kind: "post", posts: [{ text: "Nine cylinders to five." }], claims: [] };+const b = { kind: "post", posts: [{ text: "Nine cylinders to four." }], claims: [] };+console.log(contentHash(a, process.cwd()).slice(0, 16), contentHash(b, process.cwd()).slice(0, 16));+'++# 4. Read a real proof record and the review record bound to it.+node -e 'const p = require("./data/x-content/proofs/machine-atlas.json"); console.log(p.target, p.facts, p.video)'+node -e 'const r = require("./data/x-content/reviews/machine-atlas.json"); console.log(r.contentHash, r.passed, r.editor.model, r.editor.scores)'++# 5. The pipeline's own tests: scheduling, holds, the ledger, the lint, the hash binding.+npx vitest run tests/x-content+```++Step 2 prints four blocking findings: the banned opener, the two numbers no claim covers ("40" and "2 seconds"), and the absolute "fastest". Declare a claim whose `says` quotes the copy and whose evidence a program can check, and they clear one by one. Step 5 includes the time-dependent test from section 13, so expect it to fail whenever an approved item's review is older than 14 days.++The operator guide is [the pipeline doc](https://github.com/nirholas/three.ws/blob/main/docs/x-content-pipeline.md), and the guide to filmed stories is [Writing a story](https://github.com/nirholas/three.ws/blob/main/docs/x-story-authoring.md). Both are rendered on the live site at [three.ws/docs/x-content-pipeline](https://three.ws/docs/x-content-pipeline).++The part we would most like other builders to argue with is section 6: a model as an editor that may add blockers but never remove them. It is the cheapest way we have found to get a model's judgment without inheriting its confidence.++---++*three.ws is a verified AWS Partner and an open-source platform for 3D AI agents. Previously from us here: [how we metered a SaaS product through AWS Marketplace with the AWS SDK for JavaScript v3](https://builder.aws.com/content/3ESpll50BdSp9eiCEIxcfG9pGUN/how-we-metered-a-saas-product-through-aws-marketplace-with-the-aws-sdk-for-javascript-v3).*diff --git a/docs/aws-builder-center-immutable-embed-releases.md b/docs/aws-builder-center-immutable-embed-releases.mdnew file mode 100644index 000000000..d99362863--- /dev/null+++ b/docs/aws-builder-center-immutable-embed-releases.md@@ -0,0 +1,517 @@+---+title: "A URL is a promise: immutable releases for a 3D web component that runs on pages we do not own | three.ws on AWS"+venue: AWS Builder Center+account: three.ws (official organization account, byline "three.ws")+status: draft, owner approval required before publishing (external-channel gate in CLAUDE.md)+description: "How three.ws releases the <agent-3d> web component: one library build in two formats, a build order that is load-bearing, moving channels beside immutable versioned URLs, a write-once release archive, a ledger of Subresource Integrity hashes the build has to obey, and the incident where rebuilding one version on every deploy broke the integrity pins we had published for it. Plus the second lesson of code that runs on someone else's origin: where its API calls go."+tags: [web-components, cdn, supply-chain-security, release-engineering, open-source]+index: docs/aws-builder-center.md+---++# A URL is a promise: immutable releases for a 3D web component that runs on pages we do not own++Putting a live, animated 3D avatar on a web page takes two lines of HTML: a script tag that loads the `<agent-3d>` custom element from three.ws, and the element itself. That is the product working as intended, and it is also the hardest kind of software we ship, because the moment those two lines are pasted into somebody else's page we lose sight of the code. We do not see the page, we do not run its tests, and we do not get told when it breaks. The page's owner finds out when a visitor does.++So a release of that script is a set of promises to people we will never talk to:++1. **The URL serves bytes.** A versioned path answers today and in five years.+2. **The bytes are exactly those bytes.** A page that pins a hash gets the file the hash describes, or the browser refuses to run it.+3. **The bytes talk to us, not to the host.** Code running on `example.com` has to know where home is.++We broke the second promise, deploy after deploy, for about three and a half months, fixed it on 2026-10-01, and found a bug in the third promise two days later. This is the release pipeline after both fixes, with the incident where it happened. Every claim points at readable source in [the repository](https://github.com/nirholas/three.ws) (Apache-2.0) or at a public URL, and the live hashes can be recomputed with `curl` and `openssl` in the [Try it](#15-try-it) section.++**Status, plainly, because AWS builders check.** three.ws is a verified AWS Partner. The platform's runtime runs on Google Cloud Run, and the release archive described below is a Google Cloud Storage bucket. Nothing in this article runs on AWS, and we would rather say so than let a partner article imply a hosting story that is not ours. The patterns translate directly, though, and section 12 maps each one onto S3 and CloudFront for readers who want to build it there. Treat that section as a mapping, not as a description of something we operate.++**Contents**++1. What we ship, and to whom+2. One build, two formats+3. Why the build order is load-bearing+4. Moving channels and immutable versions+5. What Subresource Integrity promises, and what it does not+6. The incident: one version string, three hashes+7. The fix, part one: a write-once archive+8. The fix, part two: a ledger the build has to obey+9. The fix, part three: documentation is a pin too+10. Cutting a release+11. The second lesson: an embed runs on someone else's origin+12. Mapping this to AWS+13. What we would build differently+14. What to lift from this+15. Try it++---++## 1. What we ship, and to whom++`<agent-3d>` is a custom HTML element that carries the three.ws runtime in a single file: a three.js renderer, the avatar loader, the animation retargeting that lets one idle clip drive skeletons from many different tools, and the optional conversational layer. A bare embed looks like this:++```html+<script type="module" src="https://three.ws/agent-3d/1/agent-3d.js"></script>++<agent-3d body="https://three.ws/avatars/default.glb"+ style="width:480px;height:640px"></agent-3d>+```++The file behind that script tag is 3,308,933 bytes, measured with `curl` against production on 2026-10-08. It is self-contained on purpose: no chunk fetches, nothing for the host page to install. That makes the two-line embed possible and the release problem sharp: one artifact, loaded by everybody, executing with the full privileges of whatever page it lands on.++Its users fall into two camps:++- **The prototype builder** wants fixes to arrive without editing their page. They load a moving channel such as `/agent-3d/1/` and accept that the bytes change underneath them.+- **The production embedder** wants the bytes to never change without their consent. They load an exact version such as `/agent-3d/1.5.2/`, pin its Subresource Integrity hash, and upgrade on their own schedule.++Sections 6 to 9 are what happens when the second camp's promise is kept with the first camp's mechanism.++## 2. One build, two formats++The library is a separate Vite build from the website, selected by an environment variable. From [`package.json`](https://github.com/nirholas/three.ws/blob/main/package.json):++```json+"build:lib": "TARGET=lib vite build",+"build:lib:full": "TARGET=lib LIB_FORMATS=es,umd vite build",+```++`TARGET=lib` switches [`vite.config.js`](https://github.com/nirholas/three.ws/blob/main/vite.config.js) to its library config:++```js+build: {+ outDir: 'dist-lib',+ emptyOutDir: true,+ chunkSizeWarningLimit: 2000,+ lib: {+ entry: resolve(__dirname, 'src/lib.js'),+ name: 'Agent3D',+ formats: process.env.LIB_FORMATS ? process.env.LIB_FORMATS.split(',') : ['es'],+ fileName: (format) => (format === 'es' ? 'agent-3d.js' : 'agent-3d.umd.cjs'),+ },+ rollupOptions: {+ // inlineDynamicImports keeps the output as a single file so CDN+ // consumers get one <script type="module"> with no chunk fetches.+ // The lib IS the third-party embed, so it needs the old-WebView+ // polyfill even more than the app does.+ output: { inlineDynamicImports: true, banner: LEGACY_RUNTIME_POLYFILL },+ },+},+```++Two files come out of `build:lib:full`: `dist-lib/agent-3d.js`, the ES module our docs recommend, and `dist-lib/agent-3d.umd.cjs`, the same code wrapped for environments without ES modules. `inlineDynamicImports` is the line that makes the embed a single request: every dynamic `import()` inside the runtime is folded into the one file, so a host page never has to resolve a chunk path relative to an origin it does not control.++The `banner` prepends a small polyfill (the shipped file's first bytes define `Object.hasOwn` when it is missing), because the embed runs inside other people's WebViews, some of them old.++## 3. Why the build order is load-bearing++The production build is one npm script, `build:gcp`, and its order is not a style choice. Here is the chain from `package.json`, with each `&& npm run` drawn as an arrow:++```+check:conflicts → check:browser-graph → check:tdz-bootstrap → ensure:avatar-studio+→ build:info:snapshot → docs:freshness → build:lib:full → build:avatar-sdk+→ build:chat → build → publish:lib → build:info → check:dist → check:pages+```++Three constraints fix the library's place in it.++**The library has to be built before the avatar SDK.** We also publish the element as an npm package, `@three-ws/avatar`. Its build script, [`avatar-sdk/build.mjs`](https://github.com/nirholas/three.ws/blob/main/avatar-sdk/build.mjs), does not re-bundle anything. It copies the CDN file:++```js+const src = resolve(repoRoot, 'dist-lib', 'agent-3d.js');+```++```js+const outDir = resolve(here, 'dist');+mkdirSync(outDir, { recursive: true });++copyFileSync(src, resolve(outDir, 'index.mjs'));+```++Between those two excerpts sits a guard: when `dist-lib/agent-3d.js` does not exist, the script prints an error telling you to build the library first and calls `process.exit(1)`. One build, two distribution channels, identical bytes. That is the right design, and it creates an ordering dependency that is easy to miss.++**The avatar SDK has to be built before the website.** The website's own SDK page lazily imports `avatar-sdk/src/agent.js`, which in turn does `import('../dist/index.mjs')`. Vite resolves that import at build time. Until 2026-08-17 the deploy chain did not run `build:avatar-sdk` at all, so the frontend build silently resolved whatever stale `avatar-sdk/dist` the build machine happened to hold, and in a fresh checkout it failed outright with `Could not resolve "../dist/index.mjs"`. Commit `83f2cc476` added the step ahead of the frontend build.++**Publishing has to come after the website build.** The frontend `vite build` runs with `emptyOutDir`, so it wipes `dist/`. Anything that writes into `dist/` before it is lost. That is why `publish:lib`, which lays the library out under `dist/agent-3d/`, follows `build` rather than sitting next to `build:lib:full`, and why `check:dist`, which fails a deploy that lacks the library, runs after both.++The general lesson: when one artifact feeds others by copying, the copy creates a build-order edge no bundler knows about. Encode the order in one script, and make each step fail loudly on a missing input instead of using whatever is on disk.++## 4. Moving channels and immutable versions++The same origin serves four paths for the library:++| Path | Response cache header | Meant for |+|---|---|---|+| `/agent-3d/1.5.2/agent-3d.js` | `public, max-age=31536000, immutable` | Production embeds, pinned with SRI |+| `/agent-3d/1.5/agent-3d.js` | `public, max-age=3600, s-maxage=300, stale-while-revalidate=86400` | The newest build on the 1.5 line |+| `/agent-3d/1/agent-3d.js` | same as above | The newest build on the 1.x line |+| `/agent-3d/latest/agent-3d.js` | same as above | Demos only |++Those headers are what production returned on 2026-10-08, and they come from the route table in [`vercel.json`](https://github.com/nirholas/three.ws/blob/main/vercel.json) (the filename is legacy; our own Cloud Run server reads its routes on boot):++```json+{+ "src": "/agent-3d/([0-9]+\\.[0-9]+\\.[0-9]+(?:-[A-Za-z0-9.-]+)?)/(.*)",+ "headers": {+ "cache-control": "public, max-age=31536000, immutable",+ "access-control-allow-origin": "*",+ "access-control-allow-methods": "GET, HEAD, OPTIONS",+ "cross-origin-resource-policy": "cross-origin",+ "timing-allow-origin": "*"+ },+ "dest": "/agent-3d/$1/$2"+}+```++This rule is the whole contract. `immutable` with a one-year `max-age` tells browsers and shared caches that this URL will never change, so a browser holding the file does not even revalidate it. It is a performance gift and an irrevocable statement: if the bytes ever change, caches holding the old copy keep serving it for up to a year, everyone else gets the new one, and visitors split into two groups running different code under one version number.++The other lines matter for embedding. `access-control-allow-origin: *` is required, not optional: a `<script>` tag with an `integrity` attribute must be loaded in CORS mode (`crossorigin="anonymous"`), and a CORS-mode response without that header is blocked. `cross-origin-resource-policy: cross-origin` keeps the file loadable from pages that opt into cross-origin isolation.++The moving channels are the opposite contract: five minutes at the edge, an hour in the browser, and a long `stale-while-revalidate`. Note that `/agent-3d/1.5/` is labeled by a version line but serves the current build, which can be newer than the newest numbered release.++## 5. What Subresource Integrity promises, and what it does not++Subresource Integrity is a hash in the tag:++```html+<script+ type="module"+ src="https://three.ws/agent-3d/1.5.2/agent-3d.js"+ integrity="sha384-KdAiFRsdcCbQMu4O5rkoL8hOEYLkmrdpf9ELd7jro+9NTNipWis3oqqFLll7v20Q"+ crossorigin="anonymous"+></script>+```++The browser downloads the file, computes its SHA-384, base64-encodes it, and compares it with the attribute. On a match the script runs. On a mismatch the browser refuses to execute it and logs an error in the host page's console. Here is the exact message Chromium printed when we loaded the URL above with the hash our own tutorial carried before 2026-10-01:++```+Failed to find a valid digest in the 'integrity' attribute for resource+'https://three.ws/agent-3d/1.5.2/agent-3d.js' with computed SHA-384 integrity+'KdAiFRsdcCbQMu4O5rkoL8hOEYLkmrdpf9ELd7jro+9NTNipWis3oqqFLll7v20Q'.+The resource has been blocked.+```++For an embedder this is the right trade: if anything between them and us ever serves different bytes, malicious or mistaken, their page fails closed. The price is that the pinned URL has to be byte-stable forever, and the browser enforces that with no appeal.++Two things SRI does not cover, and we learned both while writing this article.++**SRI covers the body, not the headers around it.** The UMD build at `/agent-3d/1.5.2/agent-3d.umd.cjs` hashes exactly to its recorded value. It is also served with `content-type: application/node`, because our static server derives the type from the `.cjs` extension, and with `x-content-type-options: nosniff`, which our server sets on everything. Browsers refuse to execute a script whose type is not a JavaScript MIME type when `nosniff` is present. We loaded the documented UMD tag in headless Chromium from a non-three.ws origin and got:++```+Refused to execute script from 'https://three.ws/agent-3d/1.5.2/agent-3d.umd.cjs'+because its MIME type ('application/node') is not executable, and strict MIME+type checking is enabled.+```++The bytes are perfect and the file does not run. The archive copy of the same file is uploaded with `content-type: text/javascript; charset=utf-8`, so the fix is a header rule on our route, not a new release. The ES module, which is the format our docs recommend for production, is served as `text/javascript` and loads. We list this in section 13 because it is open at the time of writing.++**SRI does not promise the URL still exists.** A pinned URL that starts answering `404` fails exactly as closed as one with the wrong hash. Section 6 has an example.++## 6. The incident: one version string, three hashes++Here is what the publish step looked like before 2026-10-01, from the parent of commit [`bde1f4975`](https://github.com/nirholas/three.ws/commit/bde1f4975):++```js+const channels = [version, `${major}.${minor}`, major, 'latest'];+const sri = {};++for (const f of files) {+ if (f.skip) continue;+ const bytes = readFileSync(resolve(srcDir, f.name));+ const hash = createHash('sha384').update(bytes).digest('base64');+ sri[f.name] = `sha384-${hash}`;++ for (const channel of channels) {+ const outDir = resolve(destRoot, channel);+ mkdirSync(outDir, { recursive: true });+ writeFileSync(resolve(outDir, f.name), bytes);+ }+}+```++Look at the first line. The exact version directory is in the same list as the moving channels, and all four are written from `dist-lib/`, which is whatever the current build produced. Nothing about this is wrong on the day a version is cut. It goes wrong the day after, because nothing forces the version string to change when the code does.++Version 1.5.2 was cut on 2026-06-17 (commit `57c315696`). `package.json` then stayed at 1.5.2 until the fix on 2026-10-01. Over those three and a half months, every deploy rebuilt the library from current source and wrote the result to `/agent-3d/1.5.2/`, under a header that told the world those bytes would never change. Each rebuild with different library code meant a different hash behind the same URL.++The evidence was sitting in our own documentation. The fix commit replaced three different hashes for the same "immutable" file:++- our tutorial for embedding a Character Library avatar, and the combined docs page, pinned `sha384-qCG5gH4q2+k2...`,+- the embed instructions we ship to AI coding agents pinned `sha384-xkFDjVP866hY...`,+- production was serving `sha384-KdAiFRsdcCbQ...`.++Three snapshots of one version string, each true on the day it was copied, two of them refused by browsers at the time of the fix. A reader who followed the tutorial to the letter got a blank space and a console error in their own page.++There is a second casualty, quieter. The 1.5.2 commit message says "1.5.1 stays immutable." It did not stay at all. The old publish step only ever wrote the *current* version's directory, into a `dist/` that the frontend build empties on every run, so the deploy that shipped the bump dropped `/agent-3d/1.5.1/` entirely. On 2026-10-08 it answered `404`. That is the first promise from the introduction, broken by the same root cause: the serving side treated released versions as build output instead of as records.++The root cause, stated once: **a release was a side effect of a deploy.** Releases have to be records that deploys read from, never things deploys produce.++## 7. The fix, part one: a write-once archive++A released version now has one source of truth: a public-read Google Cloud Storage bucket, `gs://three-ws-lib-releases`, with each file under `agent-3d/<version>/`. On 2026-10-08 it held two objects, the two files of 1.5.2.++Two mechanisms make it write-once, at two different layers.++**The upload refuses to overwrite.** From [`scripts/release-lib.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/release-lib.mjs):++```js+gcloud([+ 'storage',+ 'cp',+ local,+ dest,+ '--if-generation-match=0',+ `--content-type=${CONTENT_TYPE[file]}`,+ '--cache-control=public, max-age=31536000, immutable',+]);+```++`--if-generation-match=0` is a precondition that only succeeds if no object exists at that path, so a second upload fails instead of replacing the file. The script treats that failure as "already archived" and verifies the archived bytes, so a re-run with *different* bytes cannot alter what is published.++**The bucket refuses to delete.** The bucket carries a retention policy of 315,295,200 seconds, a little under ten years, in effect since 2026-10-01 19:23 UTC. Until an object is older than that, it cannot be deleted or overwritten. The precondition protects against our scripts; the policy protects against a person with credentials and a bad afternoon.++The two overlap on purpose, because a precondition is opt-in per request and a hand-typed upload can forget it. One honest detail: the policy is not locked. A project owner could still shorten or remove it. Locking is irreversible, and section 13 covers why we have not done it yet.++The archive is also a public URL, so anyone can hash it and check that our origin serves what the archive holds.++## 8. The fix, part two: a ledger the build has to obey++The archive holds bytes. The ledger, [`data/agent-3d-releases.json`](https://github.com/nirholas/three.ws/blob/main/data/agent-3d-releases.json), holds the promise about them, committed to git where it can be reviewed and diffed:++```json+"archive": "https://storage.googleapis.com/three-ws-lib-releases/agent-3d",+"releases": {+ "1.5.2": {+ "publishedAt": "2026-10-01T19:22:44.064Z",+ "source": "https://three.ws/agent-3d/1.5.2/",+ "integrity": {+ "agent-3d.js": "sha384-KdAiFRsdcCbQMu4O5rkoL8hOEYLkmrdpf9ELd7jro+9NTNipWis3oqqFLll7v20Q",+ "agent-3d.umd.cjs": "sha384-z74vjNbQ8KKXBZHL6JFhl4h45B+InYbzbV8ArjZSgzR+ATWOEthqu0S/eadTA0XC"+ }+ }+}+```++`publish:lib` now reads the ledger instead of the build for every released version. From [`scripts/publish-lib.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/publish-lib.mjs):++```js+// Moving channels: the current build.+const moving = [`${major}.${minor}`, String(major), 'latest'];+for (const channel of moving) {+ const outDir = resolve(destRoot, channel);+ mkdirSync(outDir, { recursive: true });+ for (const name of built) writeFileSync(resolve(outDir, name), build[name]);+}++// Immutable versions: the archived release bytes, verified.+const releases = releasedVersions(ledger);+for (const v of releases) {+ const outDir = resolve(destRoot, v);+ mkdirSync(outDir, { recursive: true });+ for (const name of RELEASE_FILES) {+ writeFileSync(resolve(outDir, name), await fetchReleaseFile(ledger, v, name, { cacheDir }));+ }+```++Compare that with section 6: the exact version is no longer written from the build. Every released version, not just the current one, is laid out from the archive on every deploy, which also keeps a release from disappearing the way 1.5.1 did.++`fetchReleaseFile` is where the ledger becomes binding. From [`scripts/lib/agent-3d-releases.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/lib/agent-3d-releases.mjs):++```js+const url = archiveUrl(ledger, version, file);+const res = await fetch(url, { signal: AbortSignal.timeout(120_000) });+if (!res.ok) throw new Error(`${url} answered ${res.status}; the archived release is unreachable`);+const bytes = Buffer.from(await res.arrayBuffer());+const actual = sri(bytes);+if (actual !== expected) {+ throw new Error(`${url} hashes to ${actual} but ${LEDGER_REL} records ${expected}; refusing to publish altered bytes`);+}+```++The archive is trusted only as far as it agrees with the ledger. A local cache keeps rebuilds from refetching, and a cached copy is re-hashed on every read, so a corrupt entry is refetched instead of served.++Two guards close the remaining doors. `publish:lib` refuses to run at all when `package.json` names a version the ledger has never released, so "bump the version and deploy" can no longer mint a release as a side effect; its error message names `npm run release:lib` as the way out. And `check:dist`, which runs after publishing in the same chain, re-hashes what actually landed in `dist/` and compares it with the ledger and with the published manifest. From [`scripts/check-dist.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/check-dist.mjs):++```js+const actual = sri(readFileSync(p));+if (actual !== expected) {+ console.error(`[check-dist] dist/agent-3d/${v}/${name} hashes to ${actual} but release ${v} is ${expected}; an immutable URL would change bytes`);+ releaseProblems++;+}+const advertised = manifest?.channels?.[v]?.integrity?.[name];+if (manifest && advertised !== expected) {+ console.error(`[check-dist] versions.json advertises ${advertised ?? 'nothing'} for ${v} ${name}; the release is ${expected}`);+ releaseProblems++;+}+```++The manifest publishing writes, `/agent-3d/versions.json`, is the public face of the ledger. Released versions are marked `"immutable": true` with their hashes; the moving channels are marked `"build": "current"`, with the current build's hashes under a separate `currentBuild` key, so an embedder can tell which hashes are safe to pin. The same JSON is served at `/api/agent-3d/versions` with a content-derived `ETag`, so polling it costs a body-less `304` when nothing changed.++## 9. The fix, part three: documentation is a pin too++The incident was found in documentation, so the fix treats documentation as release surface. Any `<script>` tag in our docs that loads a versioned agent-3d file with an `integrity` attribute is a promise to the reader that those exact bytes exist. The library finds every such tag:++```js+const SCRIPT_TAG = /<script\b[^>]*>/gi;+const PINNED_SRC = /\bsrc\s*=\s*["']?[^"'\s>]*\/agent-3d\/(\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?)\/(agent-3d\.(?:js|umd\.cjs))\b/i;+const INTEGRITY = /\bintegrity\s*=\s*["'](sha384-[A-Za-z0-9+/]{64})["']/i;+```++It walks `docs`, `specs`, `blog`, `examples`, the skill instructions for AI coding agents, the README, and `packages`. Two outcomes need different fixes. A pin whose hash disagrees with a released version is mechanically fixable: `npm run check:sri-pins -- --fix` rewrites the hash from the ledger. A pin on a version that was never released cannot be fixed by rewriting a hash, so it stays an error until a person points it at a real release.++The check runs in three places: as `npm run check:sri-pins` on demand, inside `check:dist` on every production build, and under `npm test`, through a test named "pins every documented agent-3d SRI hash to its released bytes" in [`tests/agent-3d-releases.test.js`](https://github.com/nirholas/three.ws/blob/main/tests/agent-3d-releases.test.js). Test fixtures are deliberately excluded from the scan, because they pin wrong hashes on purpose to exercise the stale-pin detection.++This article is in that scan, too. It lives in our `docs/` folder, so the production snippet in section 5 is checked against the ledger on every build. If we had pasted a stale hash into an article about stale hashes, the deploy would have failed.++## 10. Cutting a release++Cutting a release is now an explicit act:++```json+"release:lib": "npm run build:lib:full && node scripts/release-lib.mjs",+```++It builds both files, hashes them, and checks the ledger before touching the archive. An existing version with the same bytes is a no-op. An existing version with different bytes is a hard stop:++```js+const existing = ledger.releases[version];+if (existing) {+ const same = RELEASE_FILES.every((f) => existing.integrity?.[f] === files[f].integrity);+ if (!same) {+ console.error(+ `[release-lib] ${version} is already released with different bytes. A released version never changes;` ++ ' bump package.json to a new version and release that.',+ );+ process.exit(1);+ }+```++After the upload, the script reads every file back through the same public URL the build will fetch from, and only then writes the ledger entry. Written last, the ledger only ever records releases the archive can prove. Finally the script runs the documentation pin check with `fix: true`, so the docs move to the new release in the same commit as the ledger. `--dry-run` prints the hashes and writes nothing.++End to end: bump `package.json`, run `npm run release:lib`, commit the ledger and any rewritten docs, deploy.++The script has one more mode, and it is how the incident was closed. `--from <origin>` adopts the bytes a live origin is already serving for the current version instead of the local build. On 2026-10-01 we had to pick which of the three 1.5.2 hashes to make permanent, knowing that any choice would leave holders of the other two broken. We froze 1.5.2 at the bytes production was serving at that moment (the ledger's `source` field records `https://three.ws/agent-3d/1.5.2/`), so that anyone loading the file that day, and any pin taken from the live site, kept working. The public changelog entry told everyone else to update their pin once, from `/agent-3d/1.5.2/integrity.json`.++The timeline is short enough to read in one line: the bucket was created at 19:20 UTC, the two files were archived at 19:22, the retention policy took effect at 19:23, the fix was committed at 19:25, and the build that production is still running was made at 19:31 (`/api/version` reports it). No fix for a broken immutability promise keeps every pin working. There is only choosing the least-bad hash quickly and making sure it never moves again.++## 11. The second lesson: an embed runs on someone else's origin++Two days after the freeze, a different embed bug surfaced: the third promise from the introduction, that the bytes talk to us and not to the host.++Inside our own site, `fetch('/api/agents/' + id)` is the obvious way to call our API. A site-absolute path resolves against the page's origin, which on three.ws is us. Inside an embed on `example.com`, the same line asks `example.com` for an agent. It gets a `404` (or, worse, whatever `example.com` happens to serve at that path), and the element falls back to a default body named "Agent". Before commit [`fb067d0f2`](https://github.com/nirholas/three.ws/commit/fb067d0f2), the agent resolver built its endpoint as `` `${location.origin}/api/agents/...` ``, and memory, skills and several other modules used bare relative paths. On three.ws, where the page origin and the API origin are the same, every one of those works, which is why it shipped.++The fix centralizes one decision in one small module, [`src/shared/embed-api-origin.js`](https://github.com/nirholas/three.ws/blob/main/src/shared/embed-api-origin.js), and threads its answer through the callers:++```js+// Resolution order (first match wins):+// 1. the `api-base` attribute, for self-hosted backends and local development;+// 2. the origin serving the element's script (three.ws for the CDN embed);+// 3. https://three.ws when that script came from a public npm CDN (unpkg,+// jsDelivr, ...), which serves files but none of the API;+// 4. the page's own origin, which is only right when the page IS three.ws.+```++The second rule is the clever one, and it is cheap: an ES module knows its own URL through `import.meta.url`. The element computes it once at load (`apiOriginFromScriptURL(import.meta.url)`), so an embed loaded from three.ws calls three.ws from any page. The third rule exists because public package CDNs mirror our npm build but serve none of our API.++The same module decides credentials, and that half is a security decision as much as a correctness one:++```js+export function credentialsFor(base, pageOrigin) {+ return isPageOrigin(base, pageOrigin) ? 'include' : 'omit';+}+```++Cookies ride along only when the API base *is* the page's origin, meaning three.ws embedding itself. Anywhere else the session would be a third-party cookie, and a credentialed request is one our public endpoints refuse for arbitrary origins anyway.++The server side had to meet the client halfway. Public reads that an embed makes from unknown origins (fetching a public avatar by id, for example) used to answer only an allowlist of origins, so those reads were blocked with a CORS error in host pages. They now go through a helper in [`api/_lib/http.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/http.js):++```js+export function embedReadCors(req, res, { methods = 'GET,OPTIONS', authedMethods = methods } = {}) {+ const wanted = req.method === 'OPTIONS' ? String(req.headers['access-control-request-method'] || 'GET').toUpperCase() : req.method;+ const anonymous = !req.headers.cookie && !req.headers.authorization;+ const asksForAuth = /\bauthorization\b/i.test(String(req.headers['access-control-request-headers'] || ''));+ if (wanted === 'GET' && anonymous && !asksForAuth) return cors(req, res, { origins: '*', methods });+ return cors(req, res, { methods: authedMethods, credentials: true });+}+```++The rule is the one worth copying: a request with no cookie and no authorization header can only ever see the public answer, so it is safe to open to any origin with a wildcard. Anything that carries identity keeps the strict allowlist and its credentials. The tests in `tests/api/embed-public-reads-cors.test.js` pin all three cases: an anonymous read from a stranger gets the wildcard, a stranger's read that carries a session cookie gets no CORS grant, and a stranger's preflight for a write is refused.++Now the part that ties the two lessons together. That fix is committed to `main`, and on 2026-10-08 production had not deployed it yet (`/api/version` still reports the 2026-10-01 build). When it deploys, it reaches `/agent-3d/1/`, `/agent-3d/1.5/` and `latest` immediately. It will never reach `/agent-3d/1.5.2/`: those bytes are frozen, so a page pinned to 1.5.2 keeps the old origin handling until its owner moves to a newer release, which will exist once we cut 1.5.3. That is the design, not a flaw in it. A pinned embedder chose to receive no changes without consent, including good ones. The obligation on us is to cut releases often enough that "upgrade to get the fix" is a real option, and section 6 shows that is the habit we have to build.++## 12. Mapping this to AWS++We run none of this on AWS. For readers who would, every mechanism has a direct counterpart, and this is how we would map it.++| What we use | Job it does | Direct AWS mapping |+|---|---|---|+| GCS bucket with a retention policy | Released bytes cannot be deleted or overwritten | Amazon S3 with S3 Object Lock (requires versioning). Compliance mode is the counterpart of a locked retention policy; governance mode is closer to our current unlocked one |+| `gcloud storage cp --if-generation-match=0` | Upload fails if the object exists | S3 conditional writes: `PutObject` with `If-None-Match: *` fails with `412 Precondition Failed` when the key exists |+| `--content-type` and `--cache-control` set at upload | Correct headers travel with the object | `Content-Type` and `Cache-Control` object metadata on `PutObject` |+| Route rule with `immutable` and CORS headers | Versioned paths cached forever, loadable cross-origin | A CloudFront cache behavior for the versioned path pattern with a long-TTL cache policy and a response headers policy for CORS and `Cache-Control` |+| Moving-channel route with `s-maxage=300` | Channels follow the current build | A second CloudFront cache behavior with a short TTL |+| Full-path CDN purge after every deploy | Edges drop stale moving-channel copies | A CloudFront invalidation scoped to the moving-channel paths only. Versioned paths should never need one, which is a useful alarm in itself |+| `data/agent-3d-releases.json` in git | The reviewed record of what each release is | The same file in your repository. A ledger is a code-review artifact, not an infrastructure service |+| `check:dist` and `check:sri-pins` in the build | The build refuses to publish bytes that disagree with the ledger | The same scripts as a step in AWS CodeBuild, or whatever runs your build, before the sync to S3 |++Two notes on that mapping. First, Object Lock retention applies per object version; set a bucket default so every released key carries a retain-until date. Second, if you serve versioned files straight from S3 through CloudFront instead of copying them into an application container as we do, the ledger check runs before the upload to the release prefix, and the read-back goes through the CloudFront URL. The principle holds: record the hash only after the public path serves the bytes.++## 13. What we would build differently++Five things, plainly, including two we found while writing this.++**Make releases records from day one.** The incident follows from one early design choice: the publish step wrote released versions out of the current build. The fix is about 190 lines of shared library plus two small scripts. Not writing them on day one cost three and a half months of broken pins, a lost version, and a hash we had to choose under pressure.++**Do not let unknown versions inherit the immutable header.** The versioned route rule stamps its headers before the server knows whether a file exists, and the server's 404 fallback keeps them. A request for a version that does not exist yet, say `/agent-3d/1.5.3/agent-3d.js`, answers `404` with `cache-control: public, max-age=31536000, immutable`, and our edge cache stored it (a second request came back with an `age` header). Our deploy purges the whole CDN and a release always arrives through a deploy, so the edge copy is cleared in time, but a browser that asked early may keep its cached `404` far longer. The fix is to emit the immutable header only on a successful response.++**Serve every released file with the type it was archived with.** Section 5's UMD finding: the archive stores `text/javascript`, our static server re-derives `application/node` from the extension, and `nosniff` turns that into a refusal. A release record should cover the headers that decide whether the bytes execute, and `check:dist` could assert the served type as easily as the hash.++**Lock the retention policy.** An unlocked policy stops accidents, not a determined owner. Locking is irreversible for the life of the bucket, which is why we have not done it in the first week of the new flow, and it is the right end state for a bucket whose only job is to never change.++**Give embedders a way to hear about new releases.** Pinned embedders opted out of automatic changes, so they do not see fixes like the one in section 11. Our manifest makes polling cheap, but polling is something an embedder has to think of. A release feed carrying the change summary and the new pin would make "upgrade" a decision instead of a discovery.++## 14. What to lift from this++None of this depends on 3D, on our platform, or on a particular cloud. If you ship a script other people paste into their pages:++1. **Separate the release from the deploy.** A versioned URL should only ever be written from an archive of released bytes, never from the current build. Moving channels can follow the build. Exact versions cannot.+2. **Make the archive write-once at two layers.** A precondition on every upload, so your own tools cannot overwrite, and a retention policy on the bucket, so people cannot either.+3. **Keep a ledger in version control, and make the build obey it.** Hash on the way out of the archive and refuse to publish on a mismatch. Write the ledger entry only after the public URL serves the bytes.+4. **Treat every documented pin as part of the release.** Scan your docs for integrity attributes, fail the build when one disagrees with the ledger, and give yourself a `--fix` for the mechanical case.+5. **Resolve your API origin from your own script URL.** `import.meta.url` tells an embed where home is. Never build a request from `location.origin` in code that runs on other people's pages, send cookies only when the page is you, and open anonymous public reads to any origin while keeping everything credentialed on an allowlist.++The scripts are short and readable: [`scripts/release-lib.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/release-lib.mjs), [`scripts/publish-lib.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/publish-lib.mjs), [`scripts/check-sri-pins.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/check-sri-pins.mjs), and the shared [`scripts/lib/agent-3d-releases.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/lib/agent-3d-releases.mjs), which uses Node built-ins only so it can run in a build step without `node_modules`. Lift them whole.++## 15. Try it++Everything below is a read against public URLs. No account, no key. Paste and run.++```bash+# 1. Hash the pinned file exactly as a browser does for SRI+echo "sha384-$(curl -s https://three.ws/agent-3d/1.5.2/agent-3d.js \+ | openssl dgst -sha384 -binary | openssl base64 -A)"++# 2. Hash the same release straight from the write-once archive+echo "sha384-$(curl -s https://storage.googleapis.com/three-ws-lib-releases/agent-3d/1.5.2/agent-3d.js \+ | openssl dgst -sha384 -binary | openssl base64 -A)"++# 3. Compare both with the ledger in the repository, and with the sidecar we serve+curl -s https://raw.githubusercontent.com/nirholas/three.ws/main/data/agent-3d-releases.json | grep '"agent-3d.js"'+curl -s https://three.ws/agent-3d/1.5.2/integrity.json | grep '"agent-3d.js"'++# 4. The two cache contracts: immutable versus moving+curl -sI https://three.ws/agent-3d/1.5.2/agent-3d.js | grep -i '^cache-control'+curl -sI https://three.ws/agent-3d/latest/agent-3d.js | grep -i '^cache-control'++# 5. The release manifest, then a cheap conditional poll that answers 304+curl -s https://three.ws/agent-3d/versions.json+ETAG=$(curl -sI https://three.ws/api/agent-3d/versions | awk -F': ' 'tolower($1)=="etag"{print $2}' | tr -d '\r')+curl -s -o /dev/null -w '%{http_code}\n' -H "If-None-Match: $ETAG" https://three.ws/api/agent-3d/versions++# 6. The exact commit and build production is running+curl -s https://three.ws/api/version+```++Steps 1 to 3 should print the same `sha384-KdAiFRsdcCbQ...` four times. If they ever do not, the system described in this article has failed, and we would like to hear about it.++To see the browser enforce it, put the production snippet from section 5 in an empty HTML file, open it, and then change one character of the `integrity` value. The avatar disappears and the console tells you why. The full attribute, method and event reference is at [three.ws/docs/web-component](https://three.ws/docs/web-component), including the [CDN versioning](https://three.ws/docs/web-component#cdn-versioning) section this pipeline backs, and the [end-to-end tutorial](https://three.ws/tutorials/web-component-end-to-end) walks the element through a real page.++Source: [github.com/nirholas/three.ws](https://github.com/nirholas/three.ws), Apache-2.0.++---++*three.ws is a verified AWS Partner and an open-source platform for 3D AI agents. Previously from us here: [how we metered a SaaS product through AWS Marketplace with the AWS SDK for JavaScript v3](https://builder.aws.com/content/3ESpll50BdSp9eiCEIxcfG9pGUN/how-we-metered-a-saas-product-through-aws-marketplace-with-the-aws-sdk-for-javascript-v3).*diff --git a/docs/aws-builder-center-universal-retargeting.md b/docs/aws-builder-center-universal-retargeting.mdnew file mode 100644index 000000000..d266a9de0--- /dev/null+++ b/docs/aws-builder-center-universal-retargeting.md@@ -0,0 +1,391 @@+---+title: "Any skeleton, one clip library: how we animate every humanoid avatar in the browser without a rig allowlist | three.ws on AWS"+venue: AWS Builder Center+account: three.ws (official organization account, byline "three.ws")+status: draft, owner approval required before publishing (external-channel gate in CLAUDE.md)+description: "Every animation clip on three.ws was baked once, onto one reference skeleton, and plays on avatars exported later by other tools. This is how: a bone-name canonicalizer that knows the naming conventions of the major exporters, a world-space bind correction for rest poses and axis conventions, a limb re-aim for A-pose rigs, root motion rescaled to the avatar's own height, and a fallback ladder that refuses to animate a rig badly. With the tests that pin each convention down, and the places it still fails."+tags: [three-js, animation, gltf, webgl, open-source]+index: docs/aws-builder-center.md+---++# Any skeleton, one clip library: how we animate every humanoid avatar in the browser without a rig allowlist++There are 113 animation clips on three.ws: idles, walks, waves, dances, emotes. We baked each of them once, at build time, onto one reference skeleton. They then play, in the browser, on avatars that other tools exported years apart under other naming schemes: a Mixamo character, a VRoid model, an Unreal mannequin, a Daz figure, a Blender export, a hobby rig whose bones are called `shoulderL` and `kneeL`. Nobody re-authors a clip for any of them, and there is no list of approved rigs anywhere in the code.++This article explains how that works, and where it stops. Mapping bone names is the part everybody expects, and it is the smaller half. The larger half is that two skeletons with identical names can rest in different poses and measure rotations in different frames, and a clip copied across verbatim will tip one avatar onto its back and drive another one's arms through its own chest. We shipped both of those bugs. The fixes, and the tests that keep them fixed, are the substance here.++Everything runs client-side in [three.js](https://threejs.org), and every code sample is an excerpt from [the repository](https://github.com/nirholas/three.ws) (Apache-2.0) with its file path, so you can check each claim in the same sitting.++**Status, plainly, because AWS builders check.** three.ws is a verified AWS Partner. Nothing in this article runs on an AWS service: retargeting happens in the viewer's browser tab, and the platform's own runtime is on Google Cloud Run. We would rather say that than let a partner article imply a hosting story that is not ours. One more date matters: the A-pose limb re-aim described in section 7 merged on 3 October 2026. At the time of writing, production still runs the commit before it (`curl -s https://three.ws/api/version` shows which commit is live), and the published npm package `@three-ws/retarget` 0.1.2 predates it too. Everything else here is live.++**Contents**++1. A clip is a list of bone names+2. One canonical skeleton, and why fingers decide everything+3. Reducing a name: prefixes, separators, alias tables, and their order+4. When spelling cannot decide: the clavicle and the upper arm+5. Why a name map is not enough+6. The bind correction, q' = L · q · R+7. T-pose against A-pose: re-aim the limbs before the motion plays+8. Hips, legs, and root motion+9. Failure modes and the fallback ladder+10. How the tests encode each convention+11. What we would build differently+12. What to lift from this+13. Try it++---++## 1. A clip is a list of bone names++A three.js `AnimationClip` does not store a body. It stores keyframe tracks, and each track is addressed by a string: `Hips.position`, `Spine.quaternion`, `LeftForeArm.quaternion`. When you hand the clip to an `AnimationMixer`, the mixer binds each track by looking up a node with that name in the scene graph. A track whose node does not exist is skipped without an error.++That silence is the whole problem. A rig whose elbow is named `lowerarm_l` loads, renders, runs the mixer at full frame rate, and never moves its elbow. If no track binds, the avatar stands in its bind pose: for many exporters a T-pose with the arms straight out, which reads as broken.++Every tool that exports a humanoid picks its own spelling, and the spellings are not cosmetic variants of one another. Here is one joint, the left elbow, as different conventions name it. Each line was run against the canonicalizer in [`src/glb-canonicalize.js`](https://github.com/nirholas/three.ws/blob/main/src/glb-canonicalize.js) and returns `LeftForeArm`:++```+mixamorig:LeftForeArm Mixamo+lowerarm_l Unreal mannequin+J_Bip_L_LowerArm VRM 0.x / VRoid+leftLowerArm VRM 1.0+lForeArm Daz Genesis+forearm.L Blender Rigify+CC_Base_L_Forearm Reallusion Character Creator+Bip01 L Forearm 3ds Max Biped+ElbowLeft Kinect+mElbowLeft Second Life+elbowL hand-built hobby rigs+ulna.L anatomy-kit and scan rigs+```++A library tied to one spelling animates one family of avatars. We wanted it to animate any humanoid somebody brings, which raised two questions: what turns any of those names into one name, and what else must be true for the motion to look right once it binds?++## 2. One canonical skeleton, and why fingers decide everything++The clips are produced by a build script ([`scripts/build-animations.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/build-animations.mjs)) that retargets source motion onto one reference avatar, an Avaturn-rigged model checked in as `public/avatars/cz.glb`, and writes each clip out as three.js JSON. Every track in the library therefore addresses one of 52 canonical joint names:++```js+export const CANONICAL_BONES = Object.freeze([+ 'Hips', 'Spine', 'Spine1', 'Spine2', 'Neck', 'Head',+ 'LeftShoulder', 'LeftArm', 'LeftForeArm', 'LeftHand',+ 'LeftHandIndex1', 'LeftHandIndex2', 'LeftHandIndex3',+```++That excerpt is the head of the list in `src/glb-canonicalize.js`; the rest repeats the hand for the other side and adds the legs (`LeftUpLeg`, `LeftLeg`, `LeftFoot`, `LeftToeBase` and their right twins).++Count where those 52 joints are. Six are torso and head, eight are arms, eight are legs, and **30 are fingers**. We measured the shipped library: 110 of the 113 clips carry exactly 53 tracks we can retarget (52 joint rotations plus the hip translation), and 30 of those 53 address a finger joint.++That ratio turns out to be load-bearing, because of a gate we will come back to in section 9. The retargeter refuses to play a clip unless at least half of its tracks find a home on the target rig:++```js+export const MIN_COVERAGE = 0.5;+```++(`src/animation-retarget.js`.) A rig whose body maps perfectly but whose hands do not scores 23 of 53, about 43 percent, which is under the gate. So a missing finger convention does not cost you finger animation. It costs you **all** animation: the manager builds no action for that clip, and the avatar stands in its bind pose. We learned this the hard way. Unreal mannequins, VRM 1.0 avatars, Genesis figures, MakeHuman exports and Rigify characters were all doing exactly that until their finger spellings were added,++## 3. Reducing a name: prefixes, separators, alias tables, and their order++`canonicalizeBoneName(name)` takes any string and returns a canonical name or `null`. It works in stages, and the order of the stages is where the subtlety lives.++**Strip namespaces and vendor prefixes.** A leading namespace (`Character1:Hips` from Autodesk HumanIK, `subject:LeftUpLeg` from mocap tools) goes first. Then known vendor prefixes, each one a single line: `mixamorig:` and its numbered variants, Blender's `Armature_`, Rigify's `DEF-`/`ORG-`/`MCH-` layers, CharacterStudio's `CH_`, Reallusion's `CC_Base_`, Sketchfab's `j_`, the 3ds Max `Bip01 ` prefix, and the numbered `5.joint_` form a VRM converter emits.++**Collapse separators and case, then look up.** What remains is folded into one key, and the key is looked up in three tables in a fixed priority order:++```js+ const key = s.replace(/[-_.\s]+/g, '').toLowerCase();+ // Canonical/Mixamo/Rigify spellings first, then the Unreal-mannequin aliases,+ // then the extended VRM/Daz/MakeHuman/simple-rig table (lowest priority, so it+ // only catches names the first two don't already resolve).+ return LOOKUP.get(key) ?? UNREAL_ALIASES.get(key) ?? EXTRA_ALIASES.get(key) ?? null;+```++That one fold is why `Left_Arm`, `left-arm`, `left arm`, `LEFT_ARM` and `upperarm.L`-style names reach the same key. The priority order is a safety property: the lowest-priority table can never shadow a canonical spelling, so adding an alias for an obscure convention cannot break Mixamo.++**Alias tables carry the conventions that share no spelling with the canonical set.** The Unreal mannequin is the cleanest example, because its names are keyed by the same folded form the lookup produces:++```js+const UNREAL_ALIASES = new Map(Object.entries({+ pelvis: 'Hips',+ neck01: 'Neck',+ claviclel: 'LeftShoulder', upperarml: 'LeftArm', lowerarml: 'LeftForeArm', handl: 'LeftHand',+ clavicler: 'RightShoulder', upperarmr: 'RightArm', lowerarmr: 'RightForeArm', handr: 'RightHand',+ thighl: 'LeftUpLeg', calfl: 'LeftLeg', footl: 'LeftFoot', balll: 'LeftToeBase',+ thighr: 'RightUpLeg', calfr: 'RightLeg', footr: 'RightFoot', ballr: 'RightToeBase',+}));+```++Notice what is missing: the Unreal spine chain. `spine_02` folds to `spine02`, which collides with a different convention, the `_02` node-deduplication suffix that glTF and FBX exporters append (`mixamorig:Spine_02`). That suffix must keep resolving to `Spine`, and a test pins it. The conflict was resolved by leaving the Unreal spine unaliased rather than letting the two conventions fight.++**Write the left side once, derive the right.** Most side-paired conventions are listed only by their left spelling, and the right twin is derived by swapping the side token:++```js+ for (const [lv, lc] of SIDED) {+ put(lv, lc);+ const rc = lc.replace(/^Left/, 'Right');+ let rv;+ if (/^left/.test(lv)) rv = lv.replace(/^left/, 'right');+ else if (/^l[A-Z]/.test(lv)) rv = 'r' + lv.slice(1);+ else if (/Left$/.test(lv)) rv = lv.replace(/Left$/, 'Right');+ else if (/L$/.test(lv)) rv = lv.replace(/L$/, 'R');+ else rv = lv;+ put(rv, rc);+ }+```++Halving the table halves the places a typo can hide, and the derivation rule is tested directly: a test feeds every left spelling through and asserts none of them lands on a `Right` bone, and the reverse.++**Retry suffixes only after the full name failed.** Two trailing forms get one retry each: Apple's `_joint` suffix (`left_arm_joint`) and the deduplication suffix (`LeftForeArm_010`, Blender's `.001`). Both retries run only when the unstripped name did not resolve, so a genuinely numbered bone like `left_hand_index_1` maps to `LeftHandIndex1` before its index is ever treated as a suffix.++**Non-Latin conventions get explicit entries.** MikuMikuDance names bones in Japanese with the side as a leading character (`左ひじ` is the left elbow), which the side-swap rule cannot reach, so both sides are listed. Its IK targets and twist bones stay unmapped on purpose: neither is a chain joint, and driving them fights the IK solve or tears the mesh.++The header of the file lists the conventions it handles. Rig Doctor (section 13) fingerprints 15 of them by name and reports which one a file uses.++## 4. When spelling cannot decide: the clavicle and the upper arm++Some collisions cannot be settled by any table, because the same word means different bones on different rigs:++- Rigify names the clavicle `shoulder.L` and the upper arm `upper_arm.L`. Both fold onto `LeftArm`.+- SMPL, the body model many research pipelines emit, names the clavicle `left_collar` and the upper arm `left_shoulder`. Both fold onto `LeftShoulder`, which leaves `LeftArm` empty.+- A hand-built hobby rig names its upper arm `shoulderL` and has no clavicle at all, so "always treat shoulder as the clavicle" would freeze its arms.++Whichever way a name-only rule leans, it breaks one of those three. So `resolveArmShoulderCollisions` resolves the clash by **contention**, not spelling: an entry is only reassigned when two joints contest one canonical target and the sibling target is free. A rig without the collision is never touched: on Rigify the clavicle-spelled contender is demoted to `LeftShoulder`, and on SMPL the shoulder-spelled one is promoted to `LeftArm`.++When spelling leaves more than one candidate, the skeleton breaks the tie: the clavicle is the ancestor of the upper arm. The same resolver runs on the glTF JSON at ingest and on the live scene graph at runtime, and it runs before any first-match claim, because Rigify lists `shoulder.L` first: claiming first would bind the arm rotation to the clavicle, and the arm would swing from the shoulder blade.++After resolution, a canonical name is assigned at most once. When two joints still want the same target (SMPL's ankle and foot, or Reallusion's two neck twist joints), the first in parent-first order wins and the other stays unmapped. Duplicate node names make three.js bind to an arbitrary twin, which is worse than leaving an extra joint still.++## 5. Why a name map is not enough++Suppose every name maps. Copy the clip's rotations across and two things can still go badly wrong, because a rotation track stores **absolute local rotations** measured against one skeleton's rest pose and one skeleton's axes.++**Axis conventions.** Mixamo exports bake a +90 degree rotation about X into the armature node and a -90 degree rotation into the hips. The two cancel, so the avatar stands upright at rest. But the clip was authored on a rig whose hips rest at identity, so the first keyframe of the hip track overwrites the -90 and leaves the armature's +90 uncancelled. The avatar plays its idle lying on its back. Our upright-invariant test suite records how bad this was on a real Mixamo export (`public/avatars/michelle.glb`) before the fix: the hips sat 92 degrees off vertical during `idle`, 94 during `celebrate`, and 115 during `dance`.++**Rest poses.** The reference rig rests in a T-pose. We checked rather than assumed: from the rest positions generated out of `cz.glb` into `src/animation-canonical-rest.js`, both upper arms and forearms point within a fraction of a degree of horizontal. Plenty of avatars rest differently: an A-pose with the arms angled down, bent elbows, splayed legs. The idle's first frame swings the reference rig's upper arms a little over 70 degrees down from horizontal (we measured it by forward kinematics on the reference skeleton). Apply that same rotation to an arm that already hangs well below horizontal at rest, and the hand ends up inside the torso.++Neither of these is a naming problem, and no amount of naming work fixes them. They need the target's own rest pose, measured at load time.++## 6. The bind correction, q' = L · q · R++When the `AnimationManager` attaches to a model, before any clip has moved a bone, it captures each canonical bone's rest rotation twice: in its parent's frame (local) and in the model's frame (world). The retargeter then rewrites every rotation keyframe as `q' = L · q · R`, where the factors are built from the reference rig's rest rotations (`Rs` local, `WS` world, generated from `cz.glb`) and the target's (`Rt`, `WT`):++```js+ if (WS && WT) {+ const C = turns.get(canonical);+ const Cp = turns.get(CANONICAL_PARENT[canonical]);+ // L = Rt · WT⁻¹ · Cp⁻¹ · WS · Rs⁻¹+ L = Rt.clone().multiply(WT.clone().invert());+ if (Cp) L.multiply(Cp.clone().invert());+ L.multiply(WS).multiply(Rs.clone().invert());+ // R = WS⁻¹ · C · WT+ R = WS.clone().invert();+ if (C) R.multiply(C);+ R.multiply(WT);+ if (1 - Math.abs(R.w) < BIND_EPSILON) R = null; // identity post-factor+ } else {+```++(`bindCorrections` in `src/animation-retarget.js`.) Ignore `C` and `Cp` until section 7; without them they are identity. What remains is the standard world-delta-preserving retarget: express the clip bone's motion as a change in world space from the reference rest, and replay that change from the target's rest.++For the Mixamo hips, the world rest matches and only the parent frame differs, so the formula collapses to a pure reframe that keeps the -90 the rig needs. For limbs, it replaced an earlier local-only premultiply (`Rt · Rs⁻¹`), which the code comments record as skewing limbs by about 30 degrees between A-pose and T-pose rigs.++Two details make this cheap and safe. A bone whose correction is within `1e-6` of identity is skipped, so retargeting onto the reference rig itself is byte-for-byte the verbatim path, and a test asserts that track for track. And rest is always read from the bind pose: a preview mixer leaves bones mid-pose when it stops, and measuring "rest" from that state once skewed the second clip a user played by 60 to 90 degrees.++## 7. T-pose against A-pose: re-aim the limbs before the motion plays++The bind correction preserves each bone's world-space motion delta. That is only the right thing to replay when both rigs rest in the same stance. The reference rig rests in a T-pose; replaying "swing the arm about 70 degrees down" onto a rig whose arm already hangs down in an A-pose drives the arm through the body. The commit that fixed this (`8561d5c5d`, "re-aim limb rest direction so A-pose rigs stop clipping through the torso") names the rigs it was found on: MakeHuman-derived bodies and a selfie-derived body in our Avatar Studio.++The fix adds one step before the delta replays: turn each limb bone, in the rig's own world frame, from its rest direction onto the reference rig's rest direction. Only the bones whose direction is a stance convention are listed, each keyed to the child joint that defines its direction:++```js+const LIMB_DIRECTION_CHILD = Object.freeze({+ LeftArm: 'LeftForeArm',+ LeftForeArm: 'LeftHand',+ RightArm: 'RightForeArm',+ RightForeArm: 'RightHand',+ LeftUpLeg: 'LeftLeg',+ LeftLeg: 'LeftFoot',+ RightUpLeg: 'RightLeg',+ RightLeg: 'RightFoot',+});+```++The spine, neck, head and clavicles are deliberately absent, because their rest angles are anatomy rather than a stance convention. The turns are computed parents first, so an elbow only adds the bend its parent did not already remove:++```js+ for (const bone of CANONICAL_TOPO_ORDER) {+ const parentTurn = turn.get(CANONICAL_PARENT[bone]) || null;+ if (WORLD_REST_BONES.has(bone)) continue; // identity: the foot keeps its own rest+ const target = targetDirections.get(bone);+ const source = SOURCE_REST_DIRECTION.get(bone);+ if (!target || !source) {+ if (parentTurn) turn.set(bone, parentTurn);+ continue;+ }+ dir.copy(target);+ if (parentTurn) dir.applyQuaternion(parentTurn);+ const own = dir.dot(source) > ALIGN_DOT ? null : new Quaternion().setFromUnitVectors(dir, source);+ if (own && parentTurn) turn.set(bone, own.multiply(parentTurn));+ else if (own || parentTurn) turn.set(bone, own || parentTurn);+ }+```++(`restAlignments` in `src/animation-retarget.js`.) Three choices in that loop are worth naming:++- **Shortest arc, from the parent-carried direction.** `setFromUnitVectors` gives the minimal rotation between two directions, so no bone picks up a spurious twist about its own axis.+- **Inheritance.** A bone without its own entry (a hand, a finger) inherits its parent's turn, so the hand stays rigid with the forearm and the fingers with the hand.+- **Feet keep their world rest.** The foot is in `WORLD_REST_BONES` and turns by nothing, so a sole that is flat on the floor at rest stays flat whatever angle the shin was re-aimed by. Toes inherit from the foot.++The threshold `ALIGN_DOT = 1 - 1e-6` corresponds to about 0.08 degrees, so two exports of the same skeleton that differ by measurement noise are treated as aligned, a T-pose rig turns by a few degrees at most, and the reference rig turns by nothing and still round-trips unchanged. The turn of the bone and of its parent (`C` and `Cp`) then slot into the correction from section 6, which is exactly the original formula when every turn is identity.++## 8. Hips, legs, and root motion++Legs need nothing special in the naming layer; they are ordinary entries, and the dignity sweep in section 10 checks that they actually swing. They need care in two other places.++**Only motion crosses over.** About a third of the clips (41 of 113) ship a position and a scale track for every bone, because that is how the build bakes them. A bone's local position is its offset from its parent, which is to say the reference rig's bone lengths. Copy those onto an avatar with different proportions and every bone "matches" while the skeleton folds into a heap. So the retargeter takes only what is genuinely motion:++```js+ const retargetable =+ property === 'quaternion' || (property === 'position' && canonical === 'Hips');+ if (!retargetable) continue;+```++Joint rotations, plus the root translation. Every rig keeps its own bone lengths, which is how a stylized character with short legs and a large head keeps its proportions instead of being stretched into the reference body.++**Root motion is rescaled and re-grounded.** The hip track is authored in metres around the reference rig's hip height. The manager measures the target's hip height above its own floor, in the units of the hips' parent, and scales the track to match:++```js+ let hipScale = 1;+ const baseline = clipHipBaselineY(clip);+ if (this._hipTargetLocalY > 0.05 && baseline > 0.05) {+ hipScale = Math.min(200, Math.max(0.2, this._hipTargetLocalY / baseline));+ }+```++(`src/animation-manager.js`.) The upper clamp is 200 rather than something modest because a Mixamo armature exported at `scale 0.01` puts its hips near 100 in local units; matching world heights instead would collapse the hip track to about a centimetre of motion and sink the avatar a metre into the floor. Height is measured from the model's bounding-box bottom rather than world zero, because viewers recenter models and many exports put their floor below the origin. The hip translation is also rotated into the target's hips-parent frame, so root motion travels the same world direction on a Mixamo armature as on the reference rig, and a floor offset lands the clip's ground on the rig's ground.++One more consequence: when a gesture plays over a walk as an additive overlay, every hip, leg and foot track is stripped from the overlay, so a wave never fights the walk cycle's root sway.++## 9. Failure modes and the fallback ladder++The rule we hold is that the library should rather leave an avatar alone than animate it badly, and the next rule is that "alone" must not mean a frozen T-pose when anything better is available. That gives a ladder of gates, each cheaper than the one after it.++**Rung 1: is this a drivable humanoid at all?** The manager's gate needs a skinned mesh and at least eight canonical bones:++```js+function _modelSupportsCanonicalClips(model) {+ let hasSkinnedMesh = false;+ const canonical = new Set();+ model.traverse((node) => {+ if (node.isSkinnedMesh) hasSkinnedMesh = true;+ if (node.name) {+ const c = canonicalizeBoneName(node.name);+ if (c) canonical.add(c);+ }+ });+ return hasSkinnedMesh && canonical.size >= MIN_CANONICAL_BONES;+}+```++`supportsCanonicalClips()` returns that answer, and callers decide what a "no" means for them. A shared scene that needs a performer substitutes a known-good one: the theater stage tries the agent's own avatar, then an assigned fallback avatar, then the plain mannequin, and only an avatar that passes this gate (plus a proportion check that rejects anything far wider than it is tall) gets to perform. A personal view, like an agent's own screen, shows the model as authored. Non-humanoid skeletons (quadrupeds, prop rigs) fail this rung on purpose, because there is no safe automatic mapping for them.++**Rung 2: does this clip find enough of a home?** Coverage below `MIN_COVERAGE` returns `clip: null` with an honest breakdown of matched, total and dropped bones. No half-puppet plays.++**Rung 3: would this clip knock the avatar over?** Before an action plays, the manager poses the hips at the clip's first keyframe and measures how far the bone's up-axis tilts from vertical, once per avatar and clip. Past 45 degrees the clip is disabled for that avatar, the rig keeps its authored pose, and one report goes to our client-error channel. Healthy rigs rest under about 18 degrees.++**Rung 4: the body maps but the arms do not.** If neither upper arm resolved, the clip plays on the torso and legs while the arms stay out. `relaxUndrivenArms` finds those arms geometrically, with no name needed (an upper-body bone pointing mostly sideways, not steeply up), and swings them down to a relaxed rest. It runs only when neither upper arm mapped, so it never touches a rig the clip drives.++**Deliberate refusals.** Blender control rigs of the "tracker" family export deform bones that hang off tracker nodes instead of forming a chain, and glTF cannot carry the trackers' constraints. Rotate one mapped deform bone and the next segment stays put: the arm comes apart. So the canonicalizer leaves that layer unmapped, a test pins the refusal, and those uploads need re-rigging, not renaming.++**Where the ladder still has a hole.** Rung 1 counts bones and rung 2 counts tracks, and a rig can pass the first and fail the second. A 3ds Max Biped skeleton is the live example: its body maps (`Bip01 L UpperArm` resolves), its numbered finger chain (`Bip01 L Finger0`, `Finger01`) does not. We built that skeleton in a scratch test against the current code: the idle finds a home for 21 of its 53 tracks, about 40 percent, so no clip plays; and because the arms did map, rung 4 does not engage either. The avatar shows its bind pose. Biped fingers are written up as an open task in our contribution guide, alongside two other verified gaps: HumanIK names after three.js strips the colon from them on load (`Character1Hips` maps to nothing), and Source-engine `ValveBiped` skeletons.++**What no gate can see.** A rig can map all 52 joints and still move badly if its bones are oriented unusually, and the mesh follows whatever skin weights it shipped with: a sleeve weighted to the wrong bone bends with that bone. Retargeting moves skeletons; it cannot repair skinning.++## 10. How the tests encode each convention++Three layers of tests keep this honest, and each catches a failure the others cannot.++**Name tests, one table per convention.** [`tests/glb-canonicalize.test.js`](https://github.com/nirholas/three.ws/blob/main/tests/glb-canonicalize.test.js) is mostly `it.each` tables of real export spellings, one block per convention. The Unreal block opens like this:++```js+ it.each([+ ['pelvis', 'Hips'],+ ['pelvis_09', 'Hips'], // UE name + de-dup suffix+ ['clavicle_l', 'LeftShoulder'],+ ['upperarm_l', 'LeftArm'],+ ['lowerarm_r', 'RightForeArm'],+ ['hand_r', 'RightHand'],+```++Around those tables sit the tests that encode decisions rather than spellings: the Unreal spine stays unaliased, `J_Bip_` names survive the `j_` prefix strip, side derivation never crosses sides, control-rig bones stay unmapped, metacarpals with no canonical home stay unmapped, a full Unreal mannequin hand maps all 15 joints per side, the Rigify and SMPL collisions split correctly whichever order the joints are listed in, and no canonical name is ever assigned twice. Real fixtures cover both ends: `cz.glb` is a no-op returned by reference, and `michelle.glb` is renamed, axis-folded, and asserted to keep every joint's world matrix, so the rewrite cannot change how the mesh looks. The file ran 477 tests green when we wrote this.++A new convention is always the same four parts: an alias entry, a table of its real spellings, a "never crosses sides" test, and a Rig Doctor fingerprint. That is what "no allowlist" means in practice: teach the canonicalizer once, and every surface that animates avatars learns it at the same moment.++**Upright tests that prove they have teeth.** [`tests/animation-upright-invariant.test.js`](https://github.com/nirholas/three.ws/blob/main/tests/animation-upright-invariant.test.js) retargets the featured clips onto real and synthetic rigs and asserts the hips stay within 40 degrees of vertical at every keyframe. Its best test asserts the opposite (comment lines omitted from the excerpt):++```js+ it('the test has teeth: WITHOUT the bind correction, michelle + celebrate falls flat', () => {+ const { root } = loadBoneGraph(avatar('michelle.glb'));+ const verbatim = scanClip(root, 'celebrate', { withFix: false });+ expect(verbatim).not.toBeNull();+ expect(verbatim.max).toBeGreaterThan(CATASTROPHE_DEG);+ });+```++A regression test that passes with or without the fix tests nothing. Writing the "without" case first is the cheapest way we know to prove a test can fail.++**A motion sweep, because names passing is not limbs moving.** A rig can map its hips and animate as a torso with four frozen sticks attached, and every name test stays green. `scripts/animation-dignity-sweep.mjs` drives the real idle and walk clips onto ten differently named rigs, down both production paths (canonicalize at ingest, then retarget; and retarget straight onto raw names at runtime), and measures per-limb rotation swing plus hand and foot travel through space in hip-heights, by composing world matrices. We ran it for this article: 10 of 10 conventions animate both arms and both legs on both paths, with coverage from 94 percent (the `shoulderL` hobby rig, 49 bones mapped) to 100 percent. On that hobby rig, the walk swings the legs by 50.7 and 54.4 degrees.++One honest caveat about that sweep: its ten rigs are one synthetic T-pose skeleton renamed ten ways. That is deliberate, so a difference in results is a difference in naming and never in proportions, but it also means the sweep says nothing about rest poses. Rest poses are covered by the upright suite, the byte-for-byte reference-rig test, and the real Mixamo fixture. The section 7 re-aim has no dedicated test yet; see the next section.++## 11. What we would build differently++**Derive descriptive claims from data.** Writing this article, we found our own comments disagreeing: one says the reference rig rests in an A-pose, the comment added by the October fix says T-pose, and the generated rest data settles it (T-pose). A comment cannot fail a test. Where a stance or a number can be computed from reference data, compute it or assert it.++**Write the test in the same commit as the geometric fix.** The A-pose re-aim shipped with no unit test of its own. It deserves the same treatment as the upright suite: a synthetic A-pose rig, an assertion that the idle's hands end outside the torso's bounds, and the "without the fix" case that proves the assertion can fail.++**One retarget entry point, not three.** The runtime manager, `retargetClipToObject` and `retargetClipToRig` each assemble their own inputs. The re-aim reached the first two. `retargetClipToRig`, used by the Animation Studio's preset gallery and its animated-GLB export, does not pass rest directions yet, although the helper it needs (`canonicalRestDirectionMapFromRig`) already exists. Entry points that assemble their own inputs drift.++**Make the gates agree.** The bone-count gate and the coverage gate answer different questions, which is how the Biped case slips between them. Gate on what the library will actually do (would any clip clear coverage?), not on a proxy. Likewise Rig Doctor uses the same canonicalizer but not the clavicle/upper-arm resolver, so on a Rigify rig it reports a collision the runtime resolves. A diagnostic that disagrees with the runtime teaches the wrong thing.++## 12. What to lift from this++All Apache-2.0, none of it tied to our hosting:++1. **`@three-ws/retarget`** on npm packages the canonicalizer, retargeter and `AnimationManager`, with `three` as a peer dependency. Its 0.1.2 release predates the A-pose re-aim; read that piece in `src/animation-retarget.js` until the next release.+2. **The bind correction** (section 6) works for any two skeletons given their local and world rest rotations: about thirty lines of three.js quaternions.+3. **Gate on coverage, not on a rig list.** A 50 percent coverage floor and a first-keyframe hips-tilt check are cheap, rig-agnostic, and catch the two failures that look worst on screen.+4. **Keep structure out of motion.** Skip per-bone position and scale tracks, and proportions survive.+5. **Test the absence of the fix.** For every geometric correction, keep a test that runs without it and asserts the catastrophe.++## 13. Try it++Everything below is public and keyless.++- **Rig Doctor**, with a Mixamo rig preloaded: [three.ws/rig-doctor?sample=/avatars/michelle.glb](https://three.ws/rig-doctor?sample=/avatars/michelle.glb). It names the convention, scores torso, arms, hands and legs, lists every bone rewrite, and plays idle, walk, wave and dance on the rig through the same manager the platform uses. Drop your own `.glb` on [three.ws/rig-doctor](https://three.ws/rig-doctor): the file is read in your browser and never uploaded. The [Rig Doctor reference](https://three.ws/docs/rig-doctor) explains each verdict.+- **The Animation Studio** at [three.ws/pose](https://three.ws/pose) plays the rest of the library on a loaded avatar and exports an animated GLB.+- **The clip library itself:**++```bash+# the 113 clips and their metadata+curl -s https://three.ws/animations/manifest.json | head -c 600++# one clip: tracks addressed by canonical bone name+curl -s https://three.ws/animations/clips/idle.json | head -c 400++# which commit production runs (the A-pose re-aim is 8561d5c5d)+curl -s https://three.ws/api/version+```++- **The tests and the sweep**, from a clone of [github.com/nirholas/three.ws](https://github.com/nirholas/three.ws):++```bash+npx vitest run tests/glb-canonicalize.test.js tests/animation-upright-invariant.test.js+node scripts/animation-dignity-sweep.mjs --verbose+```++If your rig uses a naming convention the tables do not know, Rig Doctor lists the unrecognised names, and [the contribution guide](./first-contribution.md) walks through teaching the canonicalizer a new one, and lists the three open conventions from section 9 ready to pick up.++---++*three.ws is a verified AWS Partner and an open-source platform for 3D AI agents. Previously from us here: [how we metered a SaaS product through AWS Marketplace with the AWS SDK for JavaScript v3](https://builder.aws.com/content/3ESpll50BdSp9eiCEIxcfG9pGUN/how-we-metered-a-saas-product-through-aws-marketplace-with-the-aws-sdk-for-javascript-v3).*diff --git a/docs/ibm-community-doc-freshness-post.md b/docs/ibm-community-doc-freshness-post.mdnew file mode 100644index 000000000..00e839248--- /dev/null+++ b/docs/ibm-community-doc-freshness-post.md@@ -0,0 +1,891 @@+---+title: "Documentation drift detection from git history: docs that tell readers, and AI agents, whether to trust them"+venue: IBM Community, Three.ws User Group (blog post)+account: nich (nich8)+description: "A long technical write-up for IBM developers, engineering leads, and platform teams: how three.ws binds every doc to the source files it names, asks git what moved since the doc was last written or reviewed, records human reviews against a specific commit, exempts dated records from rewriting, regenerates the result inside the deploy build, and shows every reader a freshness status. Includes real command output, two months of measured history, a zero-dependency Node script readers can run on their own repo in an afternoon, the honest limits of git-history staleness, and the lesson for teams whose docs are now read as instructions by AI coding agents."+status: draft, not yet posted+meta_title: "Documentation Drift Detection from Git History"+meta_description: "Bind each doc to the code it names, ask git what changed since it was last checked, and show readers and AI agents a freshness status. Runnable script."+slug: documentation-drift-detection-git-history+featured_image: "https://three.ws/api/page-og?v=carbon&s=learn&t=Documentation+that+tells+you+whether+to+trust+it&d=Measure+doc+drift+against+git+history%2C+record+human+reviews+against+a+commit%2C+and+show+every+reader+a+freshness+status.&p=%2Fdocs%2Ffreshness"+framing_notes: |+ Every framing rule in docs/ibm.md applies. three.ws is an IBM Business Partner, and+ nothing in this post is an IBM product, uses an IBM service, or had IBM involvement:+ the freshness system is three.ws engineering practice built on git and Node.js. The+ affiliation line near the top says so and must survive any edit.+ Per docs/ops/seo-keyword-plan.md this is a standalone tutorial with no crypto-cluster+ content. Every quoted doc path, command row, and commit subject was chosen to carry+ none; rows of real output that did were trimmed and the trim is marked in the text.+ Featured image caveat: the ?v=carbon card renders a fixed "Built on IBM watsonx.ai"+ footer (api/page-og.js, carbonCard). The platform is built on watsonx.ai, but next to+ a docs-freshness title the line can read as if this feature were. If that matters at+ posting time, use the same URL without `v=carbon` (the default dark card), or drop the+ featured image.+ All numbers are dated in the text. Production figures are from the deploy of commit+ 76081013b (2026-10-01); local figures are from commit fe2a8b24f (2026-10-08). Re-run+ `npm run check:docs-freshness` and re-read https://three.ws/docs-freshness-summary.json+ before posting and refresh sections 9 and 10 if they moved.+ Canonical URL: leave blank (original post).+---++# Documentation drift detection from git history: docs that tell readers, and AI agents, whether to trust them++_Posted in the [Three.ws User Group](https://community.ibm.com/community/user/groups/community-home?communitykey=e71510cc-d953-408f-9a1c-019f5c0a7016) on IBM Community._++Nobody edits a document to make it wrong. The code it describes moves, and the page keeps saying what used to be true. For most of software history the only signal that a doc had rotted was a confused reader, usually days after the rot set in.++That was tolerable when the only reader was a person who could notice that a command failed and go ask someone. It is not tolerable anymore. On a growing number of teams, documentation now has a second reader: the AI coding agent. Agents read your README, your runbooks, and your agent instruction files, and they act on what those say. A human treats a stale sentence as a hint that something is off. An agent treats it as an instruction.++This post is a complete tour of how three.ws handles that. Every doc in our repository is bound to the source files it names. Git decides whether any of those files moved after the doc was last written or last reviewed. The result is a status per doc (verified, watch, stale, conceptual, or record), a ranked work queue for the people who fix docs, and a small badge at the top of every published doc page that tells the reader how much to trust what they are about to read. The measurement is regenerated inside the deploy build, so the badge describes the commit that is actually serving.++It is a general engineering practice, not a product. Nothing here depends on our stack beyond git and Node.js, and section 11 gives you a zero-dependency script you can run against your own repository this afternoon.++**The affiliation, stated exactly.** three.ws is an IBM Business Partner. The system described in this post is three.ws's own engineering practice, built on git and Node.js. It is not an IBM product, it does not use any IBM service, and IBM had no involvement in it. I am sharing it here because the problem it solves is one every team adopting AI coding agents will meet.++**Contents**++1. Why stale docs got more expensive: your docs now have a second reader+2. The model in one paragraph+3. What "stale" is computed from+4. Five statuses, and why two of them are not failures+5. The second clock: recording a human review against a commit+6. Records of a moment: why dated docs are exempt from rewriting+7. Working the queue+8. Where it runs: the gate, and inside the deploy build+9. What readers see+10. What two months of history say+11. Build your own in an afternoon+12. The neighbors: link audits and an instruction file that checks itself+13. Honest limits+14. The lesson for teams adopting AI coding agents+15. Questions for the group++---++## 1. Why stale docs got more expensive: your docs now have a second reader++three.ws is built by a small human team working alongside AI coding agents. The agents write code, run builds, prepare deploys, and, as you will see, do most of the documentation repair. They all start from the same place: an operating file at the root of the repository (ours is `CLAUDE.md`; many teams use `AGENTS.md` or a similar convention) plus whatever docs the task leads them into.++That operating file contains a section called "Keeping this file true", and its opening paragraph is the clearest statement of the problem I know:++> This file is the operating brain for every agent here, and agents execute what it says verbatim. A stale line in it does not read as stale, it reads as an instruction, and the cost is a wasted session per drift.++That is not hypothetical. The guard script that checks the file (`scripts/check-claude-md.mjs`, covered in section 12) carries comments recording the drifts that actually happened:++- The file once described a database migration command as a dry run. The npm wrapper hardcodes `--apply`, so the "dry run" writes to the production database immediately. A human might hesitate. An agent told it is safe will run it.+- The file quoted a scheduled-job count. The real list in the config grew from 89 to 100 without anyone updating the sentence.+- The file told agents to push with `git push threews main`. On 2026-07-30 one worktree had no remote by that name, so the exact push command the file prescribes failed. As the script's comment puts it, a push instruction that does not resolve "fails at the moment the owner asked to ship."++Each of those cost a session. None of them was a dramatic failure. They were small truths that stopped being true, read by a reader that does not second-guess.++The same is true of every other doc an agent is pointed at. A runbook that names a script, a tutorial that tells you to run a command, an API reference that lists a route: all of them are now executable in a loose sense. So the question "is this doc still true?" stopped being a documentation-quality question and became an operational one.++---++## 2. The model in one paragraph++A doc already tells you what it is about: it names files, `npm run` commands, and `/api/` routes in its own text. Read those references, resolve the ones that point at real source files, and treat them as the doc's dependencies. Then ask git one question per dependency: was this file committed to after the doc was last written or last reviewed? Code that moved after the page was last checked is code the page has never been checked against. Weight each moved file by how exclusively this doc claims it, sum the weights, and turn the sum into a status. Rank the stale ones into a queue, give each one the exact commits to read, let a reviewer record "I read it and it was right" against a specific commit, and publish the result to every reader.++That is the whole system. Everything below is detail, and most of the detail exists because a simpler version produced noise that taught people to ignore it.++---++## 3. What "stale" is computed from++The implementation is one script, [`scripts/doc-freshness.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/doc-freshness.mjs), run as `npm run docs:freshness`. It needs no annotations in the docs. It reads the references authors already write.++### 3.1 Which docs++Every Markdown file under `docs/`, plus three root files that matter: `README.md`, `STRUCTURE.md` (the map from product surface to directory), and `CLAUDE.md`. Three generated aggregates that restate the whole corpus (`docs/ALL.md`, `docs/EVERYTHING.md`, `EVERYTHING.md`) are skipped. More on why in section 3.5, because the reason is one of the most useful lessons in this post.++### 3.2 What a doc depends on++Three reference shapes, because those are the three ways our docs actually name code:++```js+// scripts/doc-freshness.mjs (excerpt)+function extractDeps(markdown, selfPath) {+ const deps = new Set();++ // Explicit paths. The character class is deliberately narrow so a sentence+ // ending in a period does not swallow the next word into the match.+ for (const m of markdown.matchAll(/(?:^|[\s`("'[<])((?:\.\/)?[\w.-]+(?:\/[\w.-]+)*\.\w{1,5})/g)) {+ const rel = resolveRef(m[1], selfPath);+ if (rel) deps.add(rel);+ }++ // npm scripts: a doc telling you to run something depends on what it runs.+ for (const m of markdown.matchAll(/npm run ([\w:-]+)/g)) {+ for (const rel of scriptTargets(m[1])) deps.add(rel);+ }++ // API routes documented as endpoints map to their handler. The two shapes+ // this repo uses are api/<route>.js and api/<route>/index.js.+ for (const m of markdown.matchAll(/\/api\/([\w/-]+)/g)) {+ const route = m[1].replace(/\/$/, '');+ for (const candidate of [`api/${route}.js`, `api/${route}/index.js`]) {+ if (existsSync(path.join(ROOT, candidate))) deps.add(candidate);+ }+ }++ return [...deps];+}+```++The `npm run` rule is the one people underestimate. A doc that says "run `npm run docs:review`" never names `scripts/docs-review.mjs`, but it depends on it completely. The script reads `package.json`, finds the files that script command executes, and binds them to the doc.++Each candidate path then has to survive `resolveRef`, which rejects anything that would flood the graph with false positives: a file without a code extension, a path outside the top-level directories that hold shippable code, a directory rather than a file, a path that does not exist, and the doc's own filename echoing back at it.++### 3.3 One pass over history++Asking git about each dependency separately would be thousands of invocations. Instead the script reads the whole history once and indexes it two ways, by file and by commit:++```js+// scripts/doc-freshness.mjs (excerpt)+const raw = execFileSync(+ 'git',+ ['log', '--format=%x1e%H|%ct|%s', '--name-only', '--no-renames'],+ { cwd: ROOT, maxBuffer: 256 * 1024 * 1024 },+).toString();+```++The `%x1e` record separator keeps a commit subject containing odd characters from corrupting the parse. `%ct` is the committer timestamp, which becomes the clock everything is compared against. (Section 13 covers what that choice costs.)++### 3.4 The baseline, and the weight++For each doc, the baseline is whichever is newer: the doc's last commit, or its last recorded review (section 5). Every dependency committed to after that baseline is drift.++Not all drift means the same thing, and this is the refinement that made the ranking usable. Our route configuration file, `vercel.json` (a legacy name; the file is the live route table for our server), is named by 72 docs. If every one of them went red whenever a route was added, the dashboard would be red everywhere and nobody would read it. But excluding shared files outright is wrong too, because the doc that really is about the route table should notice when it changes.++So each dependency is weighted by how exclusively this doc claims it:++```js+// scripts/doc-freshness.mjs (excerpt)+const specificity = (share) => 1 / Math.max(1, share);+```++A file only one doc names counts fully. A file forty docs name counts for a fortieth. The comment in the source calls it what it is: the inverse-document-frequency idea from search ranking, applied to the binding between docs and code. The doc that is uniquely responsible for a file rises above the doc that happened to mention a busy one.++The signal is the sum of the weights of the drifted files, and the status falls out of it:++```js+// scripts/doc-freshness.mjs (excerpt)+function classify(deps, signal) {+ if (!deps.length) return 'unverifiable';+ if (signal <= 0) return 'fresh';+ if (signal >= 1) return 'stale';+ return 'watch';+}+```++The threshold of 1.0 reads directly: "the equivalent of one file that only this doc documents has changed." That is exactly when a human should re-read the page.++### 3.5 What is excluded, and why exclusion beats down-weighting++Three classes of file are removed from the graph entirely rather than weighted lightly:++- **Build outputs.** The sitemap, the machine-readable `llms.txt`, the changelog feeds, page indexes, build info. The changelog gaining an entry is not evidence that a tutorial went wrong. Their churn is mechanical, not semantic.+- **The machine-translated locale tree.** Locale files are extracted from annotated HTML and machine-translated into other languages. A translation pass that touches a hundred files was reading as a hundred docs going stale until commit `2bf55b582` (2026-08-14) excluded them.+- **The system's own bookkeeping.** The review store and the budget file are rewritten by every review and every budget change. The doc that explains the system names both, so recording reviews pushed that very doc straight back to stale: a false positive produced by using the system exactly as intended. Commit `302d0ae31` (2026-09-18) removed them.++The generated aggregates (`docs/ALL.md` and friends) are excluded as docs for a reason that is easy to miss and worth measuring. They restate the whole corpus, so they name nearly every file every other doc names. That does not just make them permanently stale. It **doubles the share count of almost every file in the repo**, which halves every weight, which quietly pushes real drift below the threshold everywhere.++I measured this with the minimal script from section 11, run against our repository at commit `fe2a8b24f` on 2026-10-08. With `docs/ALL.md` included (it names 2,089 files):++```+809 docs: 58 stale, 359 watch, 165 verified, 227 conceptual, 0 snapshot+```++With it excluded (`--ignore '^docs/(ALL|EVERYTHING)\.md$'`):++```+808 docs: 122 stale, 294 watch, 165 verified, 227 conceptual, 0 snapshot+```++Removing one generated file more than doubled the number of docs that cross the stale line. Nothing about the other docs changed. The aggregate was simply diluting everyone's evidence. If you build this yourself, finding and excluding your own aggregates (generated API references, concatenated handbooks, search-index dumps) is the first thing to do.++---++## 4. Five statuses, and why two of them are not failures++| Status | Shown to readers as | Meaning |+|---|---|---|+| `fresh` | Verified | Nothing it documents has changed since it was written or reviewed. |+| `watch` | Watch | Some movement, below the threshold. Worth a skim, not urgent. |+| `stale` | Stale | The equivalent of one file only this doc documents has changed. Someone should re-read it. |+| `unverifiable` | Conceptual | Names no code, so there is nothing to check. Not a problem, and not counted as drift. |+| `snapshot` | Record | A dated record of a moment. Code moving under it is not drift. |++The two that are not failures matter as much as the three that might be.++**Conceptual** exists because plenty of good docs name no code at all: architecture rationale, product narratives, design principles. Calling those stale, or calling them unknown in a way that looks like a warning, would be noise that trains people to ignore the signal. They are counted separately so coverage stays visible (128 of 809 docs in production on 2026-10-01) without polluting the queue.++**Record** exists because some docs are not descriptions of current behavior at all. Section 6 covers them.++---++## 5. The second clock: recording a human review against a commit++The first version of this system had one clock: the doc's last commit. That clock is wrong in one very common case. Someone reads a page end to end, checks every claim against the source, and finds nothing to fix. The work happened. The doc is verified. The measurement still says stale, because there is no diff to show for it.++Before review stamps existed, the only ways to clear such a page were a cosmetic edit, which lies in the history about what happened, or leaving a permanent false positive, which teaches everyone that the dashboard cries wolf. The sweep on 2026-08-05 ended with 8 docs in exactly that state, and the 2026-09-01 sweep ended with 10.++A review stamp is the second clock:++```bash+# You read the doc, checked it against the code, and it was right.+npm run docs:review -- docs/forge-pipeline.md --note "lanes, timeouts and failover verified"++# Several at once, against the commit you actually reviewed.+npm run docs:review -- docs/a.md docs/b.md --at 53f2a200f --note "checked in the September sweep"++# A dated record that must not be rewritten to match today's code.+npm run docs:review -- docs/security/review-2026-06-24.md --snapshot++npm run docs:review -- --list # every stamp, newest first+npm run docs:review -- --prune # drop stamps for docs that no longer exist+```++Stamps live in one JSON file, `data/docs-freshness-reviews.json`, keyed by doc path and sorted on write "so two agents stamping different docs never fight." This is the real entry for the doc that describes the freshness system itself:++```json+"docs/doc-freshness.md": {+ "commit": "302d0ae31",+ "kind": "verified",+ "by": "agent",+ "note": "re-read against the exclusion change"+}+```++Three design decisions are worth copying.++**A stamp records a commit, not a date.** "Reviewed on Tuesday" says nothing about which version of the code the reviewer read. "Reviewed against `302d0ae31`" does. The freshness clock uses that commit's timestamp, and anyone can check out that commit and see exactly what the reviewer saw.++**A stamp naming a commit the repository does not have is a hard error, not a silent skip.** It means someone hand-edited the store, or stamped against a branch that never landed. Either way the baseline it claims is fiction, and a fictional baseline silently suppresses real drift, which is the precise failure the system exists to prevent:++```js+// scripts/doc-freshness.mjs (excerpt)+const commit = byCommit.get(entry.commit);+if (!commit) {+ problems.push(`${docPath}: stamped at ${entry.commit}, which is not a commit in this repo`);+ continue;+}+```++The gate refuses to pass with a broken stamp outstanding, and a separate unit test (`tests/docs-freshness-reviews.test.js`) checks the store on every test run: every stamped doc exists, every stamp names a Markdown file, every stamp names a commit the repository has, and every stamp declares a kind the analyzer understands. That test runs in the normal suite precisely because the full measurement is too slow for it, and a careless bulk commit or a hand edit is exactly what would break the store.++**The newer clock wins, in both directions.** A doc edited after its last review is measured from the edit. A doc reviewed after its last edit is measured from the review. And if you stamp a doc that has uncommitted edits, the review script tells you so, because the stamp cannot mask the pending edit: the measurement will still count the doc's real last commit.++The dashboard and the badge both surface the stamp. A row on the dashboard says who checked the page and against which commit, and a page nobody has checked says so plainly and gives the command to record a check after reading it.++---++## 6. Records of a moment: why dated docs are exempt from rewriting++Some docs record a moment rather than describe current behavior: a dated security review, an incident writeup, a test log, a finished work order, an upstream bug report draft. They name plenty of code, so they will always look stale. And they must never be "fixed," because rewriting a security review to match today's code destroys the record it exists to keep.++These get `--snapshot`. Their status becomes `snapshot` regardless of drift, and they are never ranked as work:++```js+// scripts/doc-freshness.mjs (excerpt)+// A snapshot documents a moment, so today's code cannot make it wrong.+// Its dependencies stay visible (the evidence is still useful) but it is+// never ranked as work.+status: review?.kind === 'snapshot' ? 'snapshot' : classify(deps, signal),+```++Six docs carry this status today, including two security reviews, a production test log, and an upstream issue draft. The readers' badge says: "Written [date]. This page records what was true at that point, so it is kept as written rather than updated to match today's code."++There is a subtle benefit here beyond silencing noise. An agent asked to "bring the docs up to date" will happily rewrite an audit to match the current code, because from its point of view the audit is just another stale doc. The snapshot status is a machine-readable statement that this document is history, and our sweep workflow (section 7) tells its reviewers in so many words that a record must not be rewritten.++---++## 7. Working the queue++`npm run docs:freshness` ranks everything and writes the report. The command a writer actually reaches for is `--doc`, which explains one page and prints its reading list: every drifted file, the commits behind it, and their subjects.++Here is real output from 2026-10-08, for a doc about how avatar preview images are rendered:++```+$ npm run docs:freshness -- --doc docs/avatar-thumbnails.md++Avatar thumbnails+docs/avatar-thumbnails.md++ status stale (signal 1.045)+ last edited 2026-10-01 (68c8ec4db), 6d ago+ documents 29 file(s)++ 2 of them changed after this doc was last edited:++ scripts/regenerate-avatar-thumbnails.mjs (1 commit)+ 2026-10-01 309cb37e8 fix(avatars): give each thumbnail re-render two minutes, so one stuck model cannot stall the regeneration++ src/animation-retarget.js (1 commit, also documented by 21 other doc(s))+ 2026-10-03 8561d5c5d fix(animation): re-aim limb rest direction so A-pose rigs stop clipping through the torso++ Read those commits, fix anything the doc gets wrong, then record the check:+ npm run docs:review -- docs/avatar-thumbnails.md --note "<what you checked>"+```++The arithmetic is visible in the output. The regeneration script is named only by this doc, so it counts 1.0. The animation file is shared with 21 other docs, so it counts 1/22, about 0.045. Total 1.045: just over the line, and the line was crossed by the file this doc is uniquely responsible for.++Is the doc wrong? I read the commit. It added a two-minute cap per render, so one model that never finishes no longer holds a whole regeneration pass hostage (the commit's own comment says one once held a pass for seven hours). The doc's existing claims about the script (dry run by default, `--apply` to overwrite, resumable with `--offset`) are all still true. What it does not mention is the new cap. That is the typical case: not a lie, but a missing fact an operator would want. The reviewer's choice is a one-sentence addition or a stamp with a note saying the omission is deliberate, and either one clears the page honestly.++That judgment is the part the tool cannot make, and it is designed not to try.++### 7.1 The gate and the budget++`npm run check:docs-freshness` is the same measurement in gate mode. It does not write anything. It counts stale docs against a budget in `data/docs-freshness-budget.json` and fails when the count is over:++```js+// scripts/doc-freshness.mjs (excerpt)+if (over > 0) {+ printTable(results, 15);+ console.error(+ `\nOver budget by ${over}. Either refresh a doc above (run ` ++ `\`npm run docs:freshness -- --doc <path>\` for the exact commits to read), ` ++ `or raise maxStale in data/docs-freshness-budget.json with a reason.`,+ );+ process.exit(1);+}+// The budget only ever ratchets down. Reporting the slack is what makes that+// happen: a number nobody sees is a number nobody lowers.+if (over < 0) console.log(`${-over} under budget. Lower maxStale to lock the gain in.`);+```++The budget is a ratchet with its reasons recorded in the file itself: 50 on 2026-07-31 (the first measurement), 15 on 2026-08-05 (after the first full sweep), 5 on 2026-09-17 (after the sweep that reached zero, with 5 kept "as deliberate headroom for normal churn between passes rather than a backlog allowance").++Here is the gate as it stands on 2026-10-08, at commit `fe2a8b24f`. It took about two seconds, exited 1, and is honest about why. I removed ten of the fifteen table rows because they concern product areas outside this post's scope; the rows shown are untouched:++```+$ npm run check:docs-freshness+docs freshness: 69 stale / 470 watch / 144 fresh / 128 unverifiable / 6 snapshot (budget 5 stale)++15 doc(s) describing code that changed since they were written:++ !! docs/guards.md signal 5.33 18 file(s) since 2026-09-21+ scripts/build-openai-portal-fields.mjs, public/tour-builder/tour.global.js, pages/tour-atlas.html, +15 more+ !! docs/announcement-factory.md signal 5.31 13 file(s) since 2026-09-21+ api/_lib/announce/draft.js, api/_lib/announce/ledger.js, api/_lib/announce/kit.js, +10 more+ !! docs/ops/guard-wiring.md signal 2.67 10 file(s) since 2026-09-17+ tests/deploy-artifacts.test.js, scripts/check-gcloudignore.mjs, public/tour-builder/tour.global.js, +7 more+ !! docs/prompts/13-chat-gateways.md signal 2.35 5 file(s) since 2026-09-22+ api/gateway/telegram.js, api/gateway/discord.js, api/cron/changelog-push.js, +2 more+ !! docs/notifications.md signal 2.33 6 file(s) since 2026-09-17+ src/notifications-page.js, src/notifications.js, src/push-notifications.js, +3 more++Over budget by 64. Either refresh a doc above (run `npm run docs:freshness -- --doc <path>` for the exact commits to read), or raise maxStale in data/docs-freshness-budget.json with a reason.+```++Note who is on that list: `docs/guards.md` and `docs/ops/guard-wiring.md` are the docs that describe our quality gates. `STRUCTURE.md`, the map from every product surface to its directory, is in the omitted rows. The docs about the system that keeps docs honest drift like everything else, which is the strongest argument I know for measuring instead of trusting.++### 7.2 Sweeps, run by agents++The queue is drained in sweeps, and on our team the sweeps are run by AI coding agents through a workflow script (`.claude/workflows/docs-freshness.js`). It has three phases:++1. **Queue.** One agent runs the measurement and groups the stale docs into batches of at most five by subject area, so one reviewer reads each product surface's code once.+2. **Review.** One agent per batch, in parallel. Each must read the exact drifted commits and the current source, check every concrete claim (routes, environment variables, script and function names, behavior, numbers, paths, UI copy), correct rather than rewrite, and return one verdict per doc from a three-word vocabulary: `EDITED`, `VERIFIED`, or `SNAPSHOT`.+3. **Stamp.** One agent records every verdict with `npm run docs:review`. Stamping is deliberately centralized: the store is one file, and several agents writing it at once would lose stamps to a last-write-wins race.++The comment above the verdict schema states the point of the whole second clock: "A doc that was read and found correct is as finished as one that needed a fix... without VERIFIED the queue can only be drained by editing."++---++## 8. Where it runs: the gate, and inside the deploy build++The measurement runs in two places, and the second is the one I would most encourage you to copy.++**In the gate.** `check:docs-freshness` is part of `npm run gate`, our umbrella of repository health checks. The gate is run on demand, not by the deploy pipeline. Section 13 is honest about what that has cost.++**Inside the deploy build, before the frontend build.** Our production build is one chained npm script, and its order is load-bearing:++```+check:conflicts → check:browser-graph → check:tdz-bootstrap → ensure:avatar-studio+→ build:info:snapshot → docs:freshness → build:lib:full → build:avatar-sdk → build:chat+→ build → publish:lib → build:info → check:dist → check:pages+```++`docs:freshness` writes `public/docs-freshness.json` and its compact summary. The frontend build (`build`, a Vite build) copies `public/` into the deployable output. So the freshness step must run before the frontend build, and it does. That one placement decision is what makes the badge honest: every deploy measures the commit being deployed, and the published dashboard and badges describe exactly that code, not whatever a person last measured by hand.++You can check this from outside. On 2026-10-08, the live version endpoint reported:++```+$ curl -s https://three.ws/api/version+{"status":"ok","version":"1.5.2","commit":"76081013b368d6754c427f304c14ee58bbc1112e",+ "commitShort":"76081013b", ..., "commitTime":"2026-10-01T19:30:12+00:00",+ "builtAt":"2026-10-01T19:31:45.051Z", ...}+```++And the live freshness summary:++```+$ curl -s https://three.ws/docs-freshness-summary.json | head -8+{+ "$generated": "npm run docs:freshness (scripts/doc-freshness.mjs)",+ "$doc": "/docs/freshness",+ "generatedAt": "2026-10-01T19:30:52.693Z",+ "commit": "76081013b",+```++Same commit. The measurement ran about 40 seconds after the commit and about 52 seconds before the build finished, inside the build. If you take one idea from this post into your own pipeline, take this one: generate trust signals at build time, from the commit being shipped, and ship them with the artifact.++---++## 9. What readers see++### 9.1 The badge++Every published doc page and every tutorial renders a small freshness line above the article. It is mounted by [`public/doc-freshness.js`](https://github.com/nirholas/three.ws/blob/main/public/doc-freshness.js), which the docs reader and the tutorial viewer load with a dynamic import, so pages that never show a doc never pay for it.++These are real badge texts, read from the live site with a headless browser on 2026-10-08 (status glyphs omitted):++| Page | Badge |+|---|---|+| [/docs/forge](https://three.ws/docs/forge?utm_source=ibm-community) | **Verified against the code.** Last written Sep 30, 2026. None of the 12 source files this page documents has changed since. |+| /docs/doc-freshness | **Lightly out of date.** Checked against the code Sep 18, 2026. 1 file it documents changed since, so a detail here may have moved. *See what changed* |+| /docs/how-it-works | **Lightly out of date.** Last written Aug 13, 2026. 6 files it documents changed since, so a detail here may have moved. *See what changed* |+| /docs/guards | **The code this page describes has changed.** Checked against the code Sep 17, 2026. 18 files it documents changed since, and nobody has re-checked the page against them. *See what changed* |++Notice the wording rules, which are written into the module:++- **It says what happened, never what the reader did wrong.** "18 files it documents changed since, and nobody has re-checked the page against them" is a fact the reader can weigh. It does not tell them to go away.+- **A review leads when there is one.** "Checked against the code Sep 17" is a stronger promise than "last written," so the review date is shown first when a stamp exists.+- **Only drifted pages link out.** On a verified page the dashboard is a distraction from what the reader came for, so the "See what changed" link appears only on watch and stale.+- **A missing entry renders nothing.** A brand-new page, or a build where the generator never ran, shows no badge at all. The comment in the source: "an empty box that says 'unknown' is worse for the reader than no box."+- **It never blocks the content.** The reader came for the doc. The honest thing is to tell them how much to trust it while they read.++### 9.2 Keeping the badge cheap++The full report is large because it carries every drifted file and every commit behind it. On 2026-10-08 the live file was 1.63 MB raw (about 238 KB gzipped). A badge needs five fields per doc, so the script writes a separate summary with single-letter keys, documented in place:++```js+// scripts/doc-freshness.mjs (excerpt)+// Short keys: this file is fetched by every docs page that renders a+// badge, so its bytes are on the reader's critical path.+// s status, g signal, d date last edited, f files drifted, n deps known,+// v date last reviewed (absent when the doc has never been stamped)+```++That summary was 97 KB raw and about 13 KB gzipped for 809 docs, fetched once per page load and cached. A real entry:++```json+"docs/doc-freshness.md": {"s":"watch","g":0.014,"d":"2026-09-18","f":1,"n":5,"v":"2026-09-18"}+```++That `g` of 0.014 is 1/72: the one drifted file is the route table, named by 72 docs. The doc is "lightly out of date" in the most honest sense possible.++### 9.3 The dashboard++[The docs freshness dashboard](https://three.ws/docs/freshness?utm_source=ibm-community) is the people-who-fix-docs view of the same data. On 2026-10-08 its header read "809 docs bound to 4101 source files. Measured at 76081013b on Oct 1, 2026." Each status is a toggle filter showing its count (64 stale, 472 watch, 139 verified, 128 conceptual, 6 records on that deploy), and there is a text filter that matches a title, a path, or a source file that drifted, so you can ask "which docs moved when this file changed?" Sorting covers most drift, most files drifted, oldest doc, and path.++Each row expands to the evidence: every drifted file with its weight ("only this doc documents it" or "also documented by N other docs"), the commits behind it, who last checked the page and against which commit, a link to read the page, a link to edit it, and a button that copies the exact `--doc` command for that page.++---++## 10. What two months of history say++Because the report is committed alongside the code, its own git history is a time series of documentation health. These are the totals from every committed version of the report, read with `git show` on 2026-10-08 (selected rows; the rest follow the same pattern):++| Report date | Docs | Stale | What happened |+|---|---|---|---|+| 2026-07-31 | 469 | 50 | Measurement and dashboard ship; budget set to 50 |+| 2026-08-05 | 484 | 55 to 8 | First full sweep: 47 corrected, 8 verified accurate as written |+| 2026-08-14 | 551 | 148 | Nine days of feature work |+| 2026-09-01 | 628 | 174 to 14 | Second sweep: 164 corrected or extended, 10 verified |+| 2026-09-15 | 718 | 106 | Two weeks of feature work |+| 2026-09-18 | 733 | 0 | Third sweep plus review stamps: 82 corrected, 22 verified, 6 records |+| 2026-09-21 | 747 | 13 | Three days later |+| 2026-10-01 | 809 | 64 | Production deploy of `76081013b` |+| 2026-10-08 | 809 | 69 | Local measurement at `fe2a8b24f`; gate over budget by 64 |++Three things stand out.++**When the tool says stale, the doc is usually actually wrong.** Across the three sweeps, 339 flagged docs were read against the code, and 293 of them were corrected or extended: about 86 percent. The commit messages list what was wrong, and it is exactly the kind of thing a reader acts on: an environment-variable reference listing two variables no code reads, a runbook whose teardown command now refuses to run without `--force`, a command documented as writing two files when it writes four to two destinations, a tutorial whose sanity check pointed at the hosted platform instead of the stack the reader had just built, retired model names, outdated tool and job counts, and the deploy runbook's build chain. A signal with that hit rate is worth putting in front of readers.++**The corpus grows faster than you think.** 469 docs to 809 in ten weeks. Every one of the new ones names code, and every one of them starts drifting the day it lands.++**Drift comes back on a schedule.** Zero on 2026-09-18, 13 three days later, 64 at the next deploy two weeks after that. A sweep is not a fix. It is maintenance, and it has to recur. Section 13 is about what that means for where the gate lives.++---++## 11. Build your own in an afternoon++Here is a complete, zero-dependency Node.js script that implements the core model: extraction, the one-pass history index, specificity weighting, the five statuses, review stamps with a hard error on unknown commits, a budget gate, and JSON output. It needs Node 18 or newer and git. Save it anywhere and run it from inside any git repository.++It is deliberately smaller than ours, and it fixes one blind spot ours has (section 13.3): it resolves `../` links relative to the doc's own directory.++```js+#!/usr/bin/env node+// doc-drift.mjs: rank Markdown docs by how much the code they name has changed+// since each doc was last edited (or last reviewed). Zero dependencies.+//+// node doc-drift.mjs rank docs/ and README.md, print the queue+// node doc-drift.mjs --top 25 show more rows+// node doc-drift.mjs --json > r.json machine-readable report+// node doc-drift.mjs --check 10 exit 1 when more than 10 docs are stale+// node doc-drift.mjs --ignore '^docs/generated/' leave generated files out+// node doc-drift.mjs --stamp docs/setup.md [--snapshot] record a review at HEAD+//+// Optional review stamps live in .doc-reviews.json at the repo root:+// { "docs/setup.md": { "commit": "1a2b3c4", "kind": "verified" },+// "docs/incident-2026-03.md": { "commit": "1a2b3c4", "kind": "snapshot" } }+import { execFileSync } from 'node:child_process';+import { existsSync, readFileSync, writeFileSync } from 'node:fs';+import path from 'node:path';++const git = (...args) =>+ execFileSync('git', args, { maxBuffer: 512 * 1024 * 1024 }).toString();+const ROOT = git('rev-parse', '--show-toplevel').trim();+process.chdir(ROOT);++const argv = process.argv.slice(2);+const opt = (name, fallback) => {+ const i = argv.indexOf('--' + name);+ return i >= 0 && argv[i + 1] ? argv[i + 1] : fallback;+};++const IGNORE = new RegExp(opt('ignore', '$^'));+const CODE = /\.(js|mjs|cjs|ts|tsx|jsx|py|go|java|kt|rb|rs|cs|sh|sql|json|ya?ml|toml)$/;+const tracked = new Set(git('ls-files').split('\n').filter(Boolean));+const docs = [...tracked].filter((f) => ((f.startsWith('docs/') && f.endsWith('.md')) || f === 'README.md') && !IGNORE.test(f));+const npmScripts = existsSync('package.json')+ ? JSON.parse(readFileSync('package.json', 'utf8')).scripts || {}+ : {};++// 1. What does each doc claim to document? Any token that resolves to a tracked+// code file, read relative to the repo root or to the doc's own directory.+function depsOf(doc, text) {+ const deps = new Set();+ const add = (rel) => {+ const p = path.posix.normalize(rel);+ if (p !== doc && p !== '.doc-reviews.json' && CODE.test(p) && tracked.has(p) && !IGNORE.test(p)) deps.add(p);+ };+ for (const [, raw] of text.matchAll(/(?:^|[\s`("'[<])((?:\.{1,2}\/)*[\w.-]+(?:\/[\w.-]+)*\.\w{1,5})/g)) {+ const clean = raw.replace(/^\//, '');+ add(clean);+ add(path.posix.join(path.posix.dirname(doc), clean));+ }+ for (const [, name] of text.matchAll(/npm run ([\w:.-]+)/g)) {+ for (const [file] of (npmScripts[name] || '').matchAll(/[\w./-]+\.(?:mjs|cjs|js|ts|sh|py)/g)) add(file);+ }+ return [...deps];+}++// 2. One pass over history: file -> commits that touched it (newest first).+const byFile = new Map();+const byCommit = new Map();+for (const chunk of git('log', '--format=%x1e%H %ct %s', '--name-only', '--no-renames').split('\x1e')) {+ const [header, ...files] = chunk.trim().split('\n');+ if (!header) continue;+ const [sha, ts, ...subject] = header.split(' ');+ const commit = { sha, ts: Number(ts), subject: subject.join(' ') };+ byCommit.set(sha, commit);+ for (const f of files) if (f) (byFile.get(f) || byFile.set(f, []).get(f)).push(commit);+}++// 3. Review stamps: a second clock. A stamp naming an unknown commit is fatal,+// because a baseline nobody can check would silently hide real drift.+const stamps = existsSync('.doc-reviews.json') ? JSON.parse(readFileSync('.doc-reviews.json', 'utf8')) : {};+if (opt('stamp', null)) {+ const doc = opt('stamp');+ if (!tracked.has(doc)) throw new Error(`${doc} is not a tracked file`);+ stamps[doc] = { commit: git('rev-parse', '--short=9', 'HEAD').trim(), kind: argv.includes('--snapshot') ? 'snapshot' : 'verified' };+ writeFileSync('.doc-reviews.json', JSON.stringify(stamps, null, 2) + '\n');+ console.log(`Stamped ${doc} as ${stamps[doc].kind} at ${stamps[doc].commit}.`);+ process.exit(0);+}+const stampOf = (doc) => {+ const s = stamps[doc];+ if (!s) return null;+ const commit = [...byCommit.values()].find((c) => c.sha.startsWith(s.commit));+ if (!commit) throw new Error(`${doc}: stamped at ${s.commit}, which is not a commit in this repo`);+ return { ...commit, kind: s.kind === 'snapshot' ? 'snapshot' : 'verified' };+};++// 4. Score. Each drifted file counts 1/N, where N is how many docs name it.+const parsed = docs.map((doc) => ({ doc, deps: depsOf(doc, readFileSync(doc, 'utf8')) }));+const share = new Map();+for (const { deps } of parsed) for (const d of deps) share.set(d, (share.get(d) || 0) + 1);++const report = parsed.map(({ doc, deps }) => {+ const stamp = stampOf(doc);+ const baseline = Math.max(byFile.get(doc)?.[0]?.ts || 0, stamp?.ts || 0);+ const drift = deps+ .map((file) => ({ file, weight: 1 / share.get(file), since: (byFile.get(file) || []).filter((c) => c.ts > baseline) }))+ .filter((d) => d.since.length)+ .sort((a, b) => b.weight - a.weight);+ const signal = drift.reduce((sum, d) => sum + d.weight, 0);+ const status =+ stamp?.kind === 'snapshot' ? 'snapshot'+ : !deps.length ? 'conceptual'+ : signal === 0 ? 'verified'+ : signal >= 1 ? 'stale'+ : 'watch';+ return { doc, status, signal: +signal.toFixed(3), deps: deps.length, drift };+}).sort((a, b) => b.signal - a.signal);++const count = (s) => report.filter((r) => r.status === s).length;+const max = opt('check', null);+if (argv.includes('--json')) {+ // exitCode, not process.exit(): exiting early can cut off a large piped write.+ console.log(JSON.stringify(report.map((r) => ({ ...r, drift: r.drift.map((d) => ({ ...d, since: d.since.map((c) => `${c.sha.slice(0, 9)} ${c.subject}`) })) })), null, 2));+} else {+ console.log(`${report.length} docs: ${['stale', 'watch', 'verified', 'conceptual', 'snapshot'].map((s) => `${count(s)} ${s}`).join(', ')}\n`);+ for (const r of report.filter((x) => x.status === 'stale').slice(0, Number(opt('top', 10)))) {+ console.log(`${r.signal.toFixed(2).padStart(6)} ${r.doc}`);+ for (const d of r.drift.slice(0, 2)) console.log(` ${d.file} (weight ${d.weight.toFixed(2)}): ${d.since[0].sha.slice(0, 9)} ${d.since[0].subject.slice(0, 70)}`);+ }+}+if (max !== null && count('stale') > Number(max)) {+ console.error(`\n${count('stale')} stale docs, budget ${max}.`);+ process.exitCode = 1;+}+```++One bug I hit while writing it, kept in as a comment because you will hit it too: the first version called `process.exit(0)` right after printing the JSON report, and piping that output into another program cut it off at exactly 128 KB. Setting `process.exitCode` and letting Node exit on its own fixed it.++### 11.1 Watch it work in sixty seconds++This builds a throwaway repository with three docs (a setup guide, an incident record, and a design note that names no code) and walks the full lifecycle. The `sleep 1` calls matter; section 13.4 explains why.++```bash+mkdir drift-demo && cd drift-demo && git init -q+cp /path/to/doc-drift.mjs .+mkdir -p docs src+echo 'export const PORT = 8080;' > src/server.js+echo 'export const RETRIES = 3;' > src/boot.js+printf '# Setup\n\nThe server in `src/server.js` listens on port 8080.\n' > docs/setup.md+printf '# Incident 2026-03-02\n\nBoot retried 3 times (`src/boot.js`) and gave up.\n' > docs/incident-2026-03-02.md+printf '# Design notes\n\nWhy we chose a monolith.\n' > docs/design.md+git add docs src && git commit -qm "docs: setup guide, incident record, design notes"; sleep 1++# The code moves. Neither doc does.+echo 'export const PORT = 9090;' > src/server.js+echo 'export const RETRIES = 5;' > src/boot.js+git commit -qam "feat: port 9090, five boot retries"; sleep 1+node doc-drift.mjs --check 0+```++Real output from running exactly that on 2026-10-08 (your hashes will differ):++```+3 docs: 2 stale, 0 watch, 0 verified, 1 conceptual, 0 snapshot++ 1.00 docs/incident-2026-03-02.md+ src/boot.js (weight 1.00): 9d2b1369d feat: port 9090, five boot retries+ 1.00 docs/setup.md+ src/server.js (weight 1.00): 9d2b1369d feat: port 9090, five boot retries++2 stale docs, budget 0.+```++The exit code is 1. Now resolve both, the right way for each: fix the setup guide, and mark the incident as a record, because an incident writeup that said "retried 3 times" was true on the day and must stay that way.++```bash+sed -i 's/8080/9090/' docs/setup.md && git commit -qam "docs(setup): the port is 9090 now"; sleep 1+node doc-drift.mjs --stamp docs/incident-2026-03-02.md --snapshot+node doc-drift.mjs --check 0+```++```+Stamped docs/incident-2026-03-02.md as snapshot at 7af8a68a8.+3 docs: 0 stale, 0 watch, 1 verified, 1 conceptual, 1 snapshot+```++(On macOS, use `sed -i ''` instead of `sed -i`.) Now the false positive every team meets in week one: a commit that touches the file without changing anything the doc says.++```bash+echo '// bind on all interfaces' >> src/server.js && git commit -qam "chore(server): add a comment"; sleep 1+node doc-drift.mjs+```++```+3 docs: 1 stale, 0 watch, 0 verified, 1 conceptual, 1 snapshot++ 1.00 docs/setup.md+ src/server.js (weight 1.00): c029f69ba chore(server): add a comment+```++The setup guide is still correct. Without a second clock you would have to make a cosmetic edit to clear it. With one, you read it and record the fact:++```bash+node doc-drift.mjs --stamp docs/setup.md+node doc-drift.mjs --check 0+```++```+Stamped docs/setup.md as verified at c029f69ba.+3 docs: 0 stale, 0 watch, 1 verified, 1 conceptual, 1 snapshot+```++Commit `.doc-reviews.json` along with any doc edits, so the review is part of history like everything else. Finally, prove the store cannot lie. Hand-edit a stamp to point at a commit that does not exist:++```+$ echo '{"docs/setup.md":{"commit":"deadbeef0","kind":"verified"}}' > .doc-reviews.json+$ node doc-drift.mjs+Error: docs/setup.md: stamped at deadbeef0, which is not a commit in this repo+```++Exit code 1, as it should be. One more thing worth seeing for yourself: in an earlier version of this demo, both docs named `src/server.js`. Each then got a weight of 0.5 and both landed in watch rather than stale. That is the specificity rule working exactly as designed, and it is why the demo gives each doc its own file.++Against our real repository (809 docs, 12,267 commits of history on 2026-10-08), the script runs in about 2 to 3 seconds.++### 11.2 Adapting it to your repository++A checklist for the afternoon:++1. **Point it at your docs.** Change the `docs` filter on line 33 if your docs live elsewhere (a `handbook/` tree, `*.adoc`, per-package READMEs).+2. **Teach it your reference shapes.** Paths and `npm run` are covered. If your docs name Make targets, Gradle tasks, CLI subcommands, or REST routes, add a rule that maps each to the file that implements it. The route rule in section 3.2 is a ten-line template.+3. **Exclude generated output first.** Run it once with `--json`, look for docs with absurd dependency counts and files that appear in hundreds of docs, and add them to `--ignore`. Section 3.5 shows why this changes the answer, not just the noise.+4. **Pick a budget you can hold.** Run it, take today's stale count, and set `--check` to that number. Ratchet it down after each sweep, and record the reason when you do.+5. **Put it in the build that ships the docs.** Write the JSON into whatever your docs site publishes, at build time, from the commit being built. Then render a badge from it.+6. **Give your agents the same view.** If your coding agents have an instruction file, tell them to run the `--doc`-style explanation before trusting a page, and to stamp or fix rather than make cosmetic edits.++---++## 12. The neighbors: link audits and an instruction file that checks itself++Freshness measures whether a doc's prose may have drifted. Two sibling checks cover what can be verified exactly.++### 12.1 `npm run audit:docs`: the mechanical layer++[`scripts/audit-docs.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/audit-docs.mjs) catches the classes of doc rot that have a yes-or-no answer:++1. Relative links in Markdown resolve to a real file on disk.+2. Site links (`/create`, `/docs/x`) resolve to a real route: declared in the page catalog, matched by a route rule, or backed by a file.+3. `npm run <script>` references name a script that exists, and `node scripts/<x>` references name a file that exists.+4. Every directory under `packages/` and `workers/` carries a README.+5. Every public doc is registered in the page catalog, so it reaches the sitemap and `llms.txt` instead of being live and invisible to every crawler.+6. The mirror of that: every doc route declared in the catalog has a Markdown file behind it.++It deliberately skips fenced code and inline code spans (those show templates, not links), external URLs (network-dependent), and generated aggregates. Real output on 2026-10-08:++```+$ npm run audit:docs++Directories required to carry a README.md (7):+ packages/agent-cli directory has no README.md+ packages/agents-sdk directory has no README.md+ packages/mcp directory has no README.md+ packages/mcp-policy directory has no README.md+ workers/agent-gateway directory has no README.md+ workers/browser-gateway directory has no README.md+ workers/signal-bridge directory has no README.md++docs audit: 7 finding(s) across 1803 files.+```++It ran in just over a second, and yes, it is currently red. One of the seven, `workers/browser-gateway`, is an empty directory on the machine I ran it on and is not in git at all (the audit reads the disk, which is the right call for a local check). The other six are real, committed directories. Hold that thought for section 13.++### 12.2 `npm run check:claude`: the agent instruction file checks its own claims++This is the piece most directly relevant to teams adopting coding agents. [`scripts/check-claude-md.mjs`](https://github.com/nirholas/three.ws/blob/main/scripts/check-claude-md.mjs) treats the agent operating file as code with assertions. Its header explains why:++> CLAUDE.md is the operating brain for every agent in this workspace: it names npm scripts, file paths, and runbook steps that agents execute verbatim. When the repo moves and CLAUDE.md doesn't, agents follow instructions into dead ends (a renamed script, a moved runbook, a deleted directory) and burn a session discovering the drift. This check makes that drift a red build instead of a wasted session.++It does not check prose by vibes. It **re-derives each checkable claim from its source of truth** on every run:++- Every `npm run <script>` and every backticked script name in the file must exist in `package.json`.+- Every concrete repo path must exist on disk (placeholders, globs, URLs, and build artifacts are skipped, so it only fails on paths an agent would try to open and not find).+- The scheduled-job count quoted in prose must equal the length of the `crons` array in the config.+- The deploy runbook must list the real build chain, every step, in the real order. This is the assertion that keeps the chain in section 8 honest:++```js+// scripts/check-claude-md.mjs (excerpt)+const chain = (scripts['build:gcp'] || '').match(/npm run ([a-z0-9:._-]+)/g)?.map((s) => s.replace('npm run ', '')) ?? [];+const runbookOrder = chain.filter((step) => md.includes(`\`${step}\``));+const positions = runbookOrder.map((step) => md.indexOf(`\`${step}\``));+const documentedInOrder = positions.every((p, i) => i === 0 || p > positions[i - 1]);+```++- The migration warning must match what the migration script actually does: if the npm script passes `--apply`, the file must say it applies immediately, and if it stops passing `--apply`, the warning must go.+- The documented push remote must actually be configured.+- Every cloud build config must pin the service account the file says it must.+- The subagent definition files are held to the same standard as the main file.++Real output on 2026-10-08:++```+$ npm run check:claude+[check-claude] OK: every script, path, and typography rule in CLAUDE.md matches the repo+```++That "OK" turns out to be the most instructive line in this post. Keep reading.++---++## 13. Honest limits++Everything above is useful. None of it is magic. Here is what git-history staleness cannot do, with a real example for each where we have one.++### 13.1 It detects movement, not wrongness++A comment-only commit makes a doc stale (the demo in 11.1 shows exactly that). A refactor that changes nothing observable makes a doc stale. Our sweeps measured about an 86 percent hit rate (section 10), which means roughly one flagged doc in seven was already right. That is good enough to act on and bad enough that you need the second clock, or people will learn to ignore the signal. The real example from section 7, the avatar thumbnails doc, is typical: the drifted commit added a behavior the doc does not mention, and a human has to decide whether that omission matters.++### 13.2 It cannot see claims that name nothing++If a doc says "the server listens on 8080" without naming the file that sets the port, the measurement has no edge to follow. The same goes for behavior that lives in environment configuration, infrastructure, a database, or a third-party API: none of it is in the file graph. Docs that name no code at all are honestly labeled conceptual and never measured. The practical consequence is a writing rule: **name the file that is the source of truth**, because a named file is a checkable claim and an unnamed one is not.++### 13.3 Extraction has blind spots++Our extractor binds a reference by its repo-relative path. A link like `[selfie-capture.js](../src/selfie-capture.js)` has a bare filename as its visible text and a parent-relative path as its target, and the extractor does not resolve `../` against the doc's directory, so neither form binds. I confirmed this on 2026-10-08 with our avatar creation doc: it links three such source files, all of which exist, and none of the three appears among the 7 files the live report says it documents. The script in section 11 resolves relative links against the doc's own directory and binds 10 files for the same doc; the difference is exactly those three links. There will be others in your repo, which is why the first `--json` run is worth reading by hand.++### 13.4 Timestamps are not ancestry++The clock is the committer timestamp, and comparisons are strict. That has three consequences. A code change committed in the same second as the doc edit is invisible (hence the `sleep 1` calls in the demo, which you will never need in a real repo but will notice in a scripted one). Rebases and cherry-picks rewrite committer dates. And a commit made on a long-lived branch before a doc was edited on main, then merged after, carries its older timestamp, so it is filtered out even though it landed after the doc was last checked. A fully correct version would ask "is this commit an ancestor of the doc's baseline commit?" instead of comparing dates. For trunk-based teams the date approximation is very close. For teams with long-lived branches, it is worth the extra `git merge-base --is-ancestor` calls.++### 13.5 The weighting hides shared-file drift by design++The route table is named by 72 docs. When it changes, each of them moves by 0.014, and the one doc that really is the route-table reference will not cross the line on that file alone. In production on 2026-10-01, 472 of 809 docs sat in watch. Watch is a large, quiet bucket, and in practice nobody works it. That is a deliberate tradeoff, but it is a tradeoff.++### 13.6 A review stamp is trust, not proof++A stamp records that someone claims to have checked a doc against a commit. It does not record what they checked. Every one of the 111 stamps in our store today says `"by": "agent"`, which is the default, and the sweeps that recorded them were run by AI coding agents. The note field is free text. If your team needs reviews to carry accountability, require a named reviewer and keep stamps in the same review process as code.++### 13.7 A gate that is not in the ship path drifts red++This is the one I most want other teams to learn from us rather than repeat. `check:docs-freshness` runs in `npm run gate`, which is run on demand. The deploy pipeline regenerates the report (section 8) but does not enforce the budget. Stale went from 0 on 2026-09-18 to 64 at the 2026-10-01 deploy and 69 on 2026-10-08, more than ten times the budget of 5, and the 2026-10-01 deploy shipped normally with the gate in that state, because nothing in the deploy path asks. `audit:docs` is in the same position and is red today with 7 findings. The badges kept telling readers the truth the whole time, which is the system working. But the queue was not worked, because nothing forced it to be. If drift matters to you, either put the budget check in the path that ships, or schedule the sweep the way you schedule dependency updates.++### 13.8 A guard keyed to the wording of a claim stops guarding when the wording changes++And now the "OK" from section 12.2. The agent operating file states that README coverage under `packages/`, `workers/`, and `services/` is 100 percent. One of its assertions exists to check exactly that claim, and it only fires when the file contains the phrase "is currently 100%":++```js+// scripts/check-claude-md.mjs (excerpt)+if (coverageGaps.length && /Coverage under `packages\/`, `workers\/`, and `services\/` is currently 100%/.test(md)) {+```++On 2026-09-10, commit `5a431ea37` reworded that sentence for a good reason: it quoted directory counts that kept going stale, so the counts were replaced with a command that prints the gaps. That is exactly the advice the file gives itself ("name the source of truth instead of quoting a number from it"). The new sentence says "is 100%" instead of "is currently 100%". The assertion's pattern stopped matching, so the assertion stopped asserting, silently.++Between 2026-09-21 and 2026-09-25, seven new directories were committed without a README: the six real ones `audit:docs` lists in section 12.1, plus one under `services/`, which `audit:docs` does not cover. On 2026-10-08 the operating file still says coverage is 100 percent, the guard says OK, and an agent reading the file has no reason to doubt it.++The lesson generalizes well beyond our repo. **A check that skips itself when its trigger text is missing has a failure mode that looks exactly like success.** If you write assertions about prose, make a missing claim an error ("the README-coverage sentence is gone, either restore it or remove this assertion"), the way the same script already does for its cron-count sentence. And do not take a green check on an instruction file as proof that the file is true; take it as proof that the specific assertions that ran passed.++---++## 14. The lesson for teams adopting AI coding agents++If your team is bringing coding agents into its workflow, your documentation just changed category. It used to be reference material that people consulted with judgment. It is now, in part, a program that agents execute. Three things follow.++**Treat your agent instruction file like code.** Every checkable claim in it (a script name, a path, a count, an order of steps, a safety property like "this command does not write") should be re-derived from its source of truth by a check that runs often. Prefer wording that cannot go stale: point at the source of truth rather than quoting a number from it. And make your checks fail loudly when the claim they guard disappears, not quietly pass.++**Give every doc a machine-readable trust signal, and put it where the agent will see it.** A human skims a "Lightly out of date" badge and adjusts. An agent can be told to run the per-doc explanation before trusting a page, and to treat a stale status as "verify before acting." The signal costs one build step and a few kilobytes per page.++**Let agents do the maintenance, inside guardrails that keep it honest.** On our team the documentation sweeps are run by agents: a queue agent, parallel reviewers with a three-word verdict vocabulary, and a single stamping agent. That works because the guardrails are mechanical. Reviewers must read the exact drifted commits. A verified verdict is as good as an edit, so nobody fakes a change. A dated record cannot be rewritten. A stamp against a commit that does not exist fails the build. Agents are very good at reading 110 docs against 110 sets of commits in an afternoon. They are not good at knowing which docs are history unless something tells them.++The underlying point is older than any of this. A document that cannot tell you whether to trust it is asking for blind trust, and blind trust in a document has always been a bad deal. The new part is that some of the readers now extend blind trust by default. Give them, and your people, a signal that is honest about its own age.++---++## 15. Questions for the group++If you want to see the moving parts, the [doc freshness reference](https://three.ws/docs/doc-freshness?utm_source=ibm-community) describes the system end to end, the [freshness dashboard](https://three.ws/docs/freshness?utm_source=ibm-community) shows the live ranking with every drifted file and commit, and the [three.ws changelog](https://three.ws/changelog?utm_source=ibm-community) carries the 2026-09-17 entry where this shipped to readers, titled "Documentation that tells you whether to trust it."++I would genuinely like to hear how other teams in this group handle this:++1. **How do you keep agent instruction files true?** If your coding agents read an `AGENTS.md`, a rules file, or a runbook before they act, what checks it, and how often? Have you had an agent follow a stale instruction into a real incident?+2. **Where does your docs check live relative to your ship path?** We regenerate the signal on every deploy but only enforce the budget on demand, and section 13.7 shows the result. Has anyone made a docs budget a hard deploy gate without it turning into a rubber stamp that everyone raises?+3. **What do your docs depend on that is not a file?** Environment configuration, infrastructure definitions, third-party API behavior, database schemas: if you have bound docs to any of those, I would love to compare approaches.++Run the script from section 11 against your own repository and post your stale count and your weirdest false positive. Mine is a doc that went stale because someone added a comment.diff --git a/docs/ibm-community-long-running-mcp-tools-post.md b/docs/ibm-community-long-running-mcp-tools-post.mdnew file mode 100644index 000000000..367979328--- /dev/null+++ b/docs/ibm-community-long-running-mcp-tools-post.md@@ -0,0 +1,893 @@+---+title: "Long-running MCP tools: how to design a tool that outlives the tool call, from the three.ws 3D Studio server"+venue: IBM Community, Three.ws User Group (blog post)+account: nich (nich8)+description: "A long technical write-up for IBM developers and agent-framework builders: how an MCP server whose work takes from fifteen seconds to several minutes delivers every result anyway, inside host timeouts and per-turn call budgets. Ten generalizable patterns (job handles, a collector tool, call budgets, submit tickets, polling guidance in tool text, interface-side collection, structured content, honest annotations, tiers and refinement, splitting anonymous from authenticated servers), each shown in the real three.ws implementation with live JSON-RPC requests and responses, plus resources and guided prompts, a runnable build-your-own server, and notes for MCP builders targeting watsonx Orchestrate."+status: draft, not yet posted+meta_title: "Long-running MCP tools: job handles, check_job and call budgets"+meta_description: "How to build MCP tools that outlive a host's tool-call timeout: job handles, a collector tool, call budgets, submit tickets and honest metadata, with live examples."+slug: long-running-mcp-tools-outlive-the-tool-call+featured_image: "https://three.ws/api/page-og?v=carbon&s=build&t=Long-running%20MCP%20tools%20that%20outlive%20the%20call&d=Job%20handles%2C%20check_job%2C%20call%20budgets%2C%20and%20honest%20metadata%20from%20the%20three.ws%203D%20Studio%20MCP%20server&p=%2Fdocs%2Fmcp-studio"+framing_notes: |+ Every framing rule in docs/ibm.md applies to this draft and must survive any edit:+ three.ws is an IBM Business Partner; the /api/ibm/* surfaces and the open-source+ @three-ws/ibm-watsonx-mcp connector are independent developer tools built on IBM's+ publicly available Granite models, are not IBM products, are not partnership+ deliverables, and are not endorsed by IBM. Nothing here claims a watsonx Orchestrate+ listing or integration: the Agent Catalog listing is a not-yet-done path+ (docs/partners/ibm-partner-plus.md), and section 13 says so plainly. The Orchestrate+ CLI example in section 13 is quoted from IBM's documentation and has not been run+ against an Orchestrate tenant; keep that caveat if the section is edited.+ Per the group's posting rules (docs/ops/seo-keyword-plan.md) there is no+ crypto-cluster content anywhere in this post: the free 3D Studio server it is about+ carries no payment surface, and the account resources and guided prompts in section+ 11 are described only through their non-financial members. Keep it that way.+ Every response below was captured live on 2026-10-08 against production and trimmed+ only where marked. Canonical URL blank (original post). The featured image card's+ fixed footer reads "Built on IBM watsonx.ai"; the post itself makes no claim that the+ 3D Studio runs on watsonx.ai (section 9 states the prompt director's actual status).+---++# Long-running MCP tools: how to design a tool that outlives the tool call++_Posted in the [Three.ws User Group](https://community.ibm.com/community/user/groups/community-home?communitykey=e71510cc-d953-408f-9a1c-019f5c0a7016) on IBM Community._++Most MCP tutorials build a tool that answers in milliseconds: fetch the weather, read a row. The hard case nobody writes up is a tool whose real work takes longer than the host is willing to wait.++Ours is 3D generation. A text prompt goes in; a textured, downloadable GLB comes out, anywhere from about fifteen seconds to several minutes later, depending on the lane, the quality tier, and whether a GPU worker has to boot first. The MCP hosts calling us have their own clocks. ChatGPT ends any tool call still open at 60 seconds. Other hosts wait longer, but none wait forever, and a model mid-turn has its own patience and its own budget of tool calls.++This post covers the ten design decisions that let the three.ws 3D Studio MCP server finish every model anyway. Each applies to anyone wrapping a slow pipeline (batch inference, document processing, a build) as an MCP tool, and each is shown in the real implementation, with file paths in the [three.ws repository](https://github.com/nirholas/three.ws) and requests you can run against the live endpoint with no account, key, or payment. Most were learned the hard way, so this is also an account of what broke and how it was fixed, dated from our commit history.++**The affiliation, stated exactly.** three.ws is an IBM Business Partner. The `/api/ibm/*` developer surfaces and the open-source `@three-ws/ibm-watsonx-mcp` connector mentioned in section 13 are independent developer tools three.ws built on IBM's publicly available Granite models. They are **not** IBM products, **not** official partnership deliverables, and **not** endorsed by IBM. The 3D Studio server is a three.ws product. Nothing below is an IBM release or IBM guidance.++**Contents**++1. The problem: work that takes minutes, hosts that wait seconds+2. The server, live: initialize and tools/list+3. Pattern 1: never fail work that is still running, return a handle+4. Pattern 2: make the collector a first-class tool+5. Pattern 3: budget every call against the host, not against the work+6. Pattern 4: a handle that exists before the job does+7. Pattern 5: put the polling guidance where the model reads it+8. Pattern 6: let the interface collect, not the model+9. Pattern 7: structured content is the contract, text is the narration+10. Pattern 8: annotations are a safety contract, so verify them+11. Beyond tools: resources and guided prompts+12. Patterns 9 and 10: tiers, refinement, and checking the shelf first+13. Splitting anonymous tools from authenticated ones, and watsonx Orchestrate+14. The pattern checklist+15. Build your own: a long-running MCP tool in 150 lines of Node+16. Honest limits+17. Try it, and three questions for the group++---++## 1. The problem: work that takes minutes, hosts that wait seconds++The protocol has room for progress notifications and server-initiated streams, but the hosts people actually use treat a tool call as a function call with a timeout. When it fires, the host gives up. What happens to the work is up to you.++The naive slow tool accepts the call, submits the job, polls until it finishes, and returns the result. When the poll outlasts the host, three bad things compound:++- **The user sees a failure for work that will succeed.** The job keeps running, a good result lands in storage minutes later, and nobody learns it exists.+- **The model retries.** A failed call invites another, which starts a second job, which also outlives the call.+- **You pay for all of it.** Orphaned work costs exactly as much as delivered work.++We shipped the naive version first. A fix commit on 2026-07-25 records what it did in production: with our self-hosted TRELLIS lane as the primary engine, a generation took four to six minutes, the inline wait was three, so every hosted call to the generation tools answered "taking longer than expected" while the model quietly finished. We verified it twice: both "failed" generations were complete in our creations table one to three minutes after the tool gave up.++Everything that follows answers that observation. **A timeout is not a failure of the work; it is a failure of the wait.** Design for the wait ending early, and keep the work reachable afterwards.++---++## 2. The server, live: initialize and tools/list++The free 3D Studio server is `https://three.ws/api/mcp-studio`: Streamable HTTP (JSON-RPC over `POST`), protocol `2025-06-18`, no authentication. The full reference is the [3D Studio MCP server documentation](https://three.ws/docs/mcp-studio?utm_source=ibm-community).++```bash+curl -s https://three.ws/api/mcp-studio \+ -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}'+```++```json+{+ "jsonrpc": "2.0",+ "id": 1,+ "result": {+ "protocolVersion": "2025-06-18",+ "serverInfo": { "name": "three-ws-3d-studio-free", "version": "1.0.0" },+ "capabilities": {+ "tools": { "listChanged": false },+ "resources": { "listChanged": false, "subscribe": false },+ "logging": {}+ },+ "instructions": "three.ws 3D Studio turns a text prompt or an image into an interactive, downloadable 3D model (GLB), free. ... If a result comes back with status \"pending\", the model is still rendering: call check_job(job_id) after the suggested wait to collect it. Before generating a prop, character or animation, search_catalog(q) checks the thousands of ready-made CC0 props, rigged characters and motion clips three.ws already publishes ..."+ }+}+```++(Trimmed: `instructions` is one paragraph covering every tool.) The server teaches the long-running contract before the model has called anything: if you see `pending`, call `check_job` later.++`tools/list` returns fourteen tools, in about 130 ms in our measurements:++```bash+curl -s https://three.ws/api/mcp-studio \+ -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | jq '[.result.tools[].name]'+```++```json+[+ "forge_free", "text_to_avatar", "mesh_forge", "rig_mesh", "forge_avatar",+ "refine_model", "check_job", "look_at_model",+ "search_catalog", "get_catalog_item", "get_item_source",+ "create_agent_persona", "get_agent_persona", "persona_say"+]+```++Six generators, a collector (`check_job`), an inspector (`look_at_model`), three catalog reads, and three persona tools. A second front door, `https://three.ws/api/mcp-chatgpt`, serves the same handler with only the eight 3D tools, for the ChatGPT app listing:++```bash+curl -s https://three.ws/api/mcp-chatgpt \+ -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | jq -c '[.result.tools[].name]'+```++```json+["forge_free","text_to_avatar","mesh_forge","rig_mesh","forge_avatar","refine_model","check_job","look_at_model"]+```++Both doors are rows in one `SURFACES` table in [`api/_mcp-studio/dispatch.js`](https://github.com/nirholas/three.ws/blob/main/api/_mcp-studio/dispatch.js), each naming its catalog, widgets, model instructions and, for ChatGPT only, a call budget:++```js+const SURFACES = {+ full: {+ server: 'mcp-studio',+ catalog: [...TOOL_CATALOG, ...CATALOG_TOOL_CATALOG, ...PERSONA_TOOL_CATALOG],+ tools: { ...TOOLS, ...CATALOG_TOOLS, ...PERSONA_TOOLS },+ personas: true,+ instructions: [...BASE_INSTRUCTIONS, ...CATALOG_INSTRUCTIONS, ...PERSONA_INSTRUCTIONS].join(' '),+ },+ chatgpt: {+ server: 'mcp-chatgpt',+ catalog: [...TOOL_CATALOG],+ tools: { ...TOOLS },+ personas: false,+ instructions: [...BASE_INSTRUCTIONS, ...CHATGPT_INSTRUCTIONS].join(' '),+ callBudgetMs: CHATGPT_CALL_BUDGET_MS,+ },+};+```++**A host is a deployment target with its own constraints, so model it as data.** Adding a host with a different timeout is a new row, not a fork.++---++## 3. Pattern 1: never fail work that is still running, return a handle++When a generation outlives the inline wait, the tool returns a **success** result carrying a pollable handle, and the job keeps running. Here it is live on the ChatGPT surface, where every call is bounded to 40 seconds (section 5), using the fastest free tier, `draft`:++```bash+curl -s https://three.ws/api/mcp-chatgpt \+ -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":5,"method":"tools/call","params":{"name":"forge_free","arguments":{"prompt":"a small cast-iron teapot with a bamboo handle","tier":"draft"}}}'+```++HTTP 200 after 40.16 seconds:++```json+{+ "jsonrpc": "2.0",+ "id": 5,+ "result": {+ "content": [+ {+ "type": "text",+ "text": "The model is still rendering (heavier scenes take a few minutes) (roughly 14s to go). It keeps running: call the check_job tool with this job_id in ~14s to collect it, or poll https://three.ws/api/gpt-forge?job=f1.eyJwIjoiZ2NwIiwiayI6bnVsbC... until status is \"done\", then use its glb_url (view at https://three.ws/viewer?src=<glb_url>)."+ }+ ],+ "structuredContent": {+ "status": "pending",+ "jobId": "f1.eyJwIjoiZ2NwIiwiayI6bnVsbC...",+ "pollUrl": "https://three.ws/api/gpt-forge?job=f1.eyJwIjoiZ2NwIiwiayI6bnVsbC...",+ "stage": "mesh",+ "etaRemainingSeconds": 14,+ "prompt": "a small cast-iron teapot with a bamboo handle"+ }+ }+}+```++(Trimmed: the handle is a few hundred characters.) Four deliberate details:++- **It is not `isError`.** The call succeeded: it accepted the job and says where it is. An error would invite the model to retry the generator, the worst possible next step.+- **The handle is public, self-contained, and signed.** It is the same auth-free token our public REST lane gives anonymous callers, so any HTTP client can poll `pollUrl` without speaking MCP. It records which backend and task to poll, and it is HMAC-signed ([`api/_lib/forge-job-token.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/forge-job-token.js)), because otherwise a caller could forge a handle and poll arbitrary upstream tasks.+- **`stage` says which half of the pipeline is running,** `mesh` or `rig`, so a client collecting an avatar knows whether it will get a bare mesh or the finished rig.+- **`etaRemainingSeconds` is the running lane's typical duration minus elapsed time,** so the caller waits instead of retrying blind. Section 7 covers what we got wrong about it.++One helper, `pendingResult()` in [`api/_mcp-studio/tools.js`](https://github.com/nirholas/three.ws/blob/main/api/_mcp-studio/tools.js), builds the envelope, and every generator ends the same way:++```js+if (job._timedOut && job.job_id) return pendingResult({ base, jobId: job.job_id, what: 'model', prompt, ...pendingTiming(job) });+if (job._timedOut || !job.glb_url) return toolError('Generation is taking longer than expected. Please try again.');+return ok({ glbUrl: job.glb_url, base, kind: 'model', prompt, referenceImageUrl: job.preview_image_url });+```++A timed-out job **with** a handle is pending. A timeout with nothing accepted is a genuine error, and only then. The original bug was that the first two lines were one.++The helper's comment adds a rule worth stealing: every timing value comes from the job's own status payload, "nothing here is derived from a local clock, so a job handed back after a client reconnect reports the JOB's age."++---++## 4. Pattern 2: make the collector a first-class tool++A handle is only useful if the model can redeem it. The first fix returned `pollUrl` alone, and as the code comment says, "the only way back to a pending job was browsing the raw poll URL." A model inside a host cannot browse. It can call a tool. So the server publishes `check_job`, built on three rules: **one probe, no loop; three clear outcomes; only an unrecognized handle is final.**++Collecting the teapot right after the pending result:++```bash+curl -s https://three.ws/api/mcp-studio \+ -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":6,"method":"tools/call","params":{"name":"check_job","arguments":{"job_id":"f1.eyJwIjoiZ2NwIiwiayI6bnVsbC..."}}}'+```++This first check took 15.9 seconds and returned the finished model (`result.structuredContent`, with the `spatial` block trimmed):++```json+{+ "kind": "model",+ "glbUrl": "https://three.ws/cdn/forge/anon/d0bf4918-36c5-4716-a476-58b4885e6dc2.glb",+ "viewerUrl": "https://three.ws/viewer?src=https%3A%2F%2Fthree.ws%2Fcdn%2Fforge%2Fanon%2Fd0bf4918-36c5-4716-a476-58b4885e6dc2.glb",+ "arUrl": "https://three.ws/api/ar?src=https%3A%2F%2Fthree.ws%2Fcdn%2Fforge%2Fanon%2Fd0bf4918-36c5-4716-a476-58b4885e6dc2.glb&title=a%20small%20cast-iron%20teapot%20with%20a%20bamboo%20handle",+ "format": "glb",+ "prompt": "a small cast-iron teapot with a bamboo handle",+ "referenceImageUrl": "https://three.ws/cdn/forge/refs/30921a8f-7b7d-4886-a971-8826d128ff66.jpg",+ "spatial": { "spatialMcpVersion": "0.1", "kind": "model", "...": "..." }+}+```++The GLB is real: 1,530,768 bytes, served as `model/gltf-binary`. A second check of the same job took **0.136 seconds** and returned the identical envelope.++That difference is the most instructive number here. The **first** check that finds a job finished does real work: it copies the mesh into our storage so it outlives the provider's URL, records the creation, and runs an automated quality gate. Later checks are served from a cached finished frame. The cache exists because of a failure measured on 2026-08-27: every poll of a finished job re-ran that work, 13 to 33 seconds **per poll**, timing out a 15-second status tool on roughly every other read. A finished job is terminal, so its first frame is now kept for six hours and returned verbatim.++The lessons:++- **A collector may be slow exactly once.** Do the post-processing on the first collection; cache the terminal frame.+- **Separate "the check failed" from "the job failed."** A check that failed while the job is fine (a timeout, a busy rate bucket, the slow first save-and-score) carries `retryable: true`.+- **Make the one final error say what to do.** An unknown handle:++```bash+curl -s https://three.ws/api/mcp-studio \+ -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":8,"method":"tools/call","params":{"name":"check_job","arguments":{"job_id":"f1.not-a-real-handle"}}}' \+ | jq -c .result.structuredContent+```++```json+{"error":true,"message":"That job id is not recognized (it may be mistyped or expired). Start a new generation to get a fresh one."}+```++No `retryable`, because retrying cannot succeed, and the message names the one action that will. The handler, from `api/_mcp-studio/tools.js` (lightly condensed: production also normalizes the prompt's type):++```js+async function handleCheckJob(args, _auth, req) {+ const base = originFromReq(req);+ const jobId = String(args.job_id || '').trim();+ if (!jobId) return toolError('Provide the job_id a pending generation returned.');+ let data;+ try {+ data = await pollOnce(base, jobId);+ } catch (err) {+ return toolError(failureMessage(err), err?.code === 'unknown_job' ? {} : { retryable: true });+ }+ if (data.status === 'done' && data.glb_url) {+ const refined = finishRefinement(base, data, args.refine);+ if (refined) return refined;+ return ok({ glbUrl: data.glb_url, base, kind: 'model', prompt: data.prompt || undefined, referenceImageUrl: data.preview_image_url });+ }+ if (data.status === 'failed') {+ return toolError(data.error ? `Generation failed: ${data.error}` : failureMessage({ code: 'generation_failed' }));+ }+ return pendingResult({ base, jobId, what: 'model', prompt: data.prompt || undefined, ...pendingTiming(data) });+}+```++Two decisions sit outside it. **Collection never spends the generation quota:** [`api/_mcp-studio/handler.js`](https://github.com/nirholas/three.ws/blob/main/api/_mcp-studio/handler.js) excludes `check_job` from the rate-limited set, "or collecting a pending job could be rate-blocked by the very generation that created it." And **a result collected later is the same envelope as a result returned inline,** so every consumer handles one success shape.++---++## 5. Pattern 3: budget every call against the host, not against the work++Pattern 1 makes a timeout survivable. Pattern 3 makes sure the **host's** timeout never fires. If the host kills the call, the response is lost, including the handle you were about to return. So the server needs its own deadline, comfortably inside the host's, flowing into every wait. On the ChatGPT surface, from `dispatch.js`:++```js+// 40 s of work, leaving headroom under the host's 60 s limit for a submit that+// needs its guaranteed floor (gpt-forge-client.js SUBMIT_FLOOR_MS) and the+// response's own trip back through ChatGPT.+export const CHATGPT_CALL_BUDGET_MS = 40_000;+```++The dispatcher turns the budget into an absolute deadline once, when the call arrives:++```js+const ctx = surface.callBudgetMs ? { deadline: started + surface.callBudgetMs } : {};+const result = await tool.handler(args, auth, req, ctx);+```++The prompt director, the submit and the poll loop all shrink to fit it. In [`api/_mcp-studio/gpt-forge-client.js`](https://github.com/nirholas/three.ws/blob/main/api/_mcp-studio/gpt-forge-client.js) the effective deadline is the earlier of the lane's timeout and the call's, and no probe or sleep may run past it:++```js+const deadline = Math.min(Date.now() + tMs, callDeadline || Infinity);+const left = () => deadline - Date.now();+```++Why 40 and not 59? The remaining twenty seconds must hold a submit that is guaranteed an 8-second floor even near the deadline (without an accepted job there is nothing to hand back), plus the response's trip back through the host. Our teapot call returned at 40.16 seconds measured from our side of the network: the budget plus about 160 ms.++- **Use absolute deadlines, not durations.** "You have 40 seconds" handed to three nested functions gives each of them 40 seconds. "Be done by 12:00:40" gives all of them the same truth.+- **Different hosts, different budgets.** `/api/mcp-studio` keeps a long inline wait (three minutes by default), because other hosts wait and an inline model beats a handle when allowed. `tests/mcp-chatgpt-budget.test.js` pins both: "stays under the 60 s host limit with room for a floored submit" and "leaves the full MCP surface unbounded, where hosts wait minutes."+- **Back off, and never let a blip end the loop.** Polling starts at three seconds and stretches by 1.35x toward ten, about a third of the status calls a fixed cadence would make. A network error, a 429 or a 5xx during a rolling deploy is retried; only a definitive 4xx ends the loop. After twenty consecutive soft failures the loop still returns the pending shape: "a pollable handle, never a dead error."++---++## 6. Pattern 4: a handle that exists before the job does++The budget created a subtler bug. On 2026-09-30 we measured text-to-3D **submits** taking 30 to 42 seconds in production: the server paints a reference image before accepting a job, and that step was falling through a ladder of image providers. With a 40-second budget, the client aborted a 42-second submit. The server, unaware, finished it and created a real job. The tool had already answered "Generation is taking longer than expected," and the job id was never delivered. The fix commit records that two test cases of our OpenAI App Directory resubmission failed exactly this way.++The fix, in [`api/_lib/forge-submit-ticket.js`](https://github.com/nirholas/three.ws/blob/main/api/_lib/forge-submit-ticket.js), is a **submit ticket: a job handle that exists before the job does.**++1. The client mints a ticket (a timestamp plus a random UUID) and sends it with the submit in an `x-forge-ticket` header.+2. The server marks it `submitting` before starting work, and records the submit's response under it when done, **whether or not the caller is still listening.**+3. A caller whose submit ran out of time returns the ticket's handle (`t1.<ticket>`) as its pending job id.+4. Polling that handle resolves to the real job once the submit lands.++Client side, inside the submit's timeout handler:++```js+// The submit is still running server-side and will record its job under+// the ticket, so hand back the ticket's handle rather than submit a+// second job or report a timeout.+if (ticket) return { status: 'submitting', job_id: ticketHandle(ticket) };+```++`check_job` and the viewer widget needed no changes: they already poll whatever handle they are given. Three details make it robust:++- **A grace window.** A poll can arrive before the `submitting` mark is visible. A ticket younger than 90 seconds with no record is "still being submitted," not unknown; the mint timestamp inside it makes that decidable.+- **A staleness ceiling.** A ticket silent for five minutes died with its instance and is reported failed rather than spinning forever. The slowest submit we measured was 42 seconds.+- **No raw tickets in key listings.** Records are keyed by a hash of the ticket.++If you take one idea from this post, take this one: **any tool that starts asynchronous work should be able to name that work before it finishes starting it.** It is an idempotency key applied to the start of a long job: the client chooses the identifier, so a lost response never means a lost job, and a retry finds the same work instead of creating a second copy.++The same commit fixed a failure that is easy to miss: provider **billing notices** accepted as output. The prompt director rewrites a short prompt into a fuller brief, and a provider's "raise your key budget at https://..." reply was being forwarded as the brief, which, the code comment records, "is how 22 production generations came out of a billing message." The validator now rejects any rewrite containing a link, any that does not end as a finished sentence, and any no longer than what the user typed, and falls back to the user's own words. **Treat upstream text as untrusted until it passes a shape check, especially text that becomes the input to an expensive job.**++---++## 7. Pattern 5: put the polling guidance where the model reads it++An MCP server can teach a model in three places, and a long-running tool needs all three:++1. **Server `instructions`** at `initialize`: the contract in one paragraph.+2. **Tool descriptions** at `tools/list`: `check_job` says "While it is still rendering you get updated timing; call again after the suggested wait."+3. **The result text:** "call the check_job tool with this job_id in ~14s to collect it."++The third is the one most servers skip, and it works best, because it arrives exactly when the decision is made. Its machine-readable twin, `etaRemainingSeconds`, sits beside it in `structuredContent`.++Guidance is also **per host.** The ChatGPT surface appends:++```js+const CHATGPT_INSTRUCTIONS = [+ 'In ChatGPT the inline viewer collects a pending job by itself and shows the model when it lands, so tell the',+ 'user it is rendering and do not loop on check_job; call check_job only when the user asks about the job.',+];+```++That is a per-turn call budget expressed as an instruction. Where the interface collects the job (pattern 6), a model looping on a status tool spends the turn's tool calls on work already being done. Elsewhere the model is the collector, and the base instructions tell it to collect.++**Honest numbers are part of the guidance.** On 2026-09-09 we found the remaining-time estimate floored at five seconds, so once a job outran its estimate every poll said "roughly 5s to go" for as long as it ran. Measured that week: a 12.5-minute job claimed five seconds remaining for its final eleven minutes. The fix commit called it "a fabricated reading, not a slow one." Past the estimate the field is now absent, consumers fall back to "it keeps running in the background," and `elapsedSeconds` carries a real number that moves.++**Name the state, not just the time.** A job queued on a scale-to-zero GPU worker is waiting for a boot, not a slow render, and that changes what a sensible client does. Since 2026-09-08 the text says "The GPU worker for this model is waking up (about Ns of boot left)" and `structuredContent` carries `coldStart: true` with the boot budget, every number from the job's own status. The retry hint points at the end of the boot, because "telling a caller to come back in the render ETA when the worker answers sooner wastes the difference." With no reported budget, the server names the state and promises no time.++**Every number you show a model will be acted on, so a wrong number is worse than none.**++---++## 8. Pattern 6: let the interface collect, not the model++The best collector is often the interface the host renders. In ChatGPT each 3D Studio tool links an Apps SDK widget through `_meta["openai/outputTemplate"]` and is marked `openai/widgetAccessible: true`, which lets the widget call tools itself through `window.openai.callTool`. When a result is pending, the widget takes over. From [`api/_mcp-studio/component.js`](https://github.com/nirholas/three.ws/blob/main/api/_mcp-studio/component.js):++```js+var POLL_FIRST_MS = 5000;+var POLL_MAX_MS = 12000;+var POLL_GIVE_UP_MS = 12 * 60 * 1000;+var MAX_POLL_FAILURES = 6;+```++It polls `check_job` from five seconds, backing off 1.3x toward twelve, shows a live elapsed timer, and drops the model in when it lands. When the envelope says `"next": "rig"` (an avatar mesh whose skeleton is still to come), it calls `rig_mesh` itself, and if rigging fails it keeps the mesh: "The mesh is real and saved; a failed rig must not throw it away." Three failure choices worth copying:++- **Retryable failures retry on cadence;** only six in a row surface, as "Lost contact with the job while it was rendering. It keeps running, so checking again usually finds it," with a resume button.+- **Twelve minutes is a soft limit, not a verdict:** "It is still running, and you can keep waiting for it," with a resume that picks up the same job.+- **The finished model is saved to widget state,** so reopening the conversation shows it instead of restarting the wait.++A host without widget tool calls gets a designed "still rendering" panel, and the model stays the collector. **An interface-side collector is an optimization, never the only path.** This is what the 2026-09-29 changelog entry, "3D Studio in ChatGPT finishes every model, however long it takes," describes from the user's side.++---++## 9. Pattern 7: structured content is the contract, text is the narration++Every result has two channels: `content`, text for the model and for hosts that render nothing, and `structuredContent`, a stable JSON contract for the widget and client code. The contract is deliberately minimal. Compare the raw job status our public REST endpoint returns for the teapot:++```bash+curl -s "https://three.ws/api/gpt-forge?job=f1.eyJwIjoiZ2NwIiwiayI6bnVsbC..." | jq -c '{creation_id, status, durable, score: .quality.score, triangles: .quality.metrics.triangleCount}'+```++```json+{"creation_id":"d0bf4918-36c5-4716-a476-58b4885e6dc2","status":"done","durable":true,"score":0.914,"triangles":7800}+```++The REST frame carries internal identifiers, quality metrics and the storage bucket URL. The MCP envelope carries a GLB link on our own domain, viewer and AR links, the format, the prompt and the concept image. The `tools.js` header states the rule: every internal identifier (creation id, prediction id, backend name, trace) is stripped per the host's data-minimization policy, with one exception, the public poll handle, because "without it the still-running work would be unreachable." Four more choices:++- **Rewrite URLs to your own origin.** Bucket GLB URLs become first-party `/cdn/` paths, because a sandboxed widget needs CORS headers the bucket does not send, and your own URL survives a storage move.+- **Additive, open shapes.** Every result also carries a `spatial` block, an open artifact format for 3D results from the repository's Spatial MCP specification, so other renderers can display it without breaking readers of the older fields.+- **Pictures, when the consumer is a model.** `look_at_model` returns rendered views as MCP `image` content blocks, each preceded by a text block naming the angle, so a multimodal model can judge its own output and regenerate naming the fault. The [3D vision documentation](https://three.ws/docs/3d-vision?utm_source=ibm-community) covers it.+- **Protocol errors and tool errors stay separate.** A malformed request is a JSON-RPC error the host can catch as a bug:++```bash+curl -s https://three.ws/api/mcp-studio -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":9,"method":"tools/call","params":{"name":"check_job","arguments":{}}}'+```++```json+{"jsonrpc":"2.0","id":9,"error":{"code":-32602,"message":"invalid params for check_job: (root) must have required property 'job_id'"}}+```++A tool that ran and failed returns a normal **result** with `isError: true` and a sanitized message the model can act on. Arguments are validated against the published `inputSchema` (with Ajv) before the handler runs, so the schema a client reads is the one the server enforces, and failure text passes through a sanitizer so provider internals never leak.++One note this group should have precisely, since it touches IBM. The prompt director is written to try Granite on watsonx.ai first when watsonx credentials are configured, and to fall through a model chain otherwise, under its own time cap. Production does not carry watsonx credentials today, so the fallback chain does that work. The pattern is the point: a quality step on the critical path must be bounded and must fail soft to the user's own words.++---++## 10. Pattern 8: annotations are a safety contract, so verify them++Tool annotations (`readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`) are how a client decides whether a tool may run without asking the user. A read-only tool runs silently. That makes them a security contract.++`check_job` reads like a status probe, and for a while we annotated it as one: read-only and idempotent. Both were wrong. The 2026-09-12 fix:++```js+annotations: {+ readOnlyHint: false, // polling can persist a model and submit recovery jobs+ destructiveHint: false,+ idempotentHint: false, // later checks can advance recovery to another provider+ openWorldHint: true,+},+```++The first check of a finished job copies the mesh into storage, records the creation, fills the result cache and runs the quality gate, and it can route failed work to another provider; later checks hit the cache. So it writes, and repeating it is not the same as making it once. A client using the old hints to decide what was safe to repeat was told the wrong thing, and the same claim had gone to the OpenAI app directory twice. Our own submission-review tooling caught it; the fix landed in the code, the docs, and a golden fixture of the public tool contract.++The generators are honest the other way: not read-only (they create an asset), not destructive (they never modify or delete), not idempotent ("same prompt, a fresh, different mesh each call"), open-world (work runs against external model services). `look_at_model` is genuinely read-only and idempotent.++What keeps this true is mechanical verification. `npm run audit:mcp-safety` parses every tool definition in the repository and checks its annotations against what its handler does, and `audit:mcp-golden` fails when a tool's public contract changes without a deliberate fixture update. The contract is written up in the [MCP tool safety documentation](https://three.ws/docs/mcp-safety?utm_source=ibm-community), and every tool on every three.ws server, with its safety class, is in the [searchable MCP tool catalog](https://three.ws/mcp-tools?utm_source=ibm-community), generated from the tool source on every build.++**If a tool that sounds like a read writes on any path, it is not read-only.**++---++## 11. Beyond tools: resources and guided prompts++MCP servers have two more primitives, **resources** and **prompts**, and both are underused. Resources let a client read state without a tool call; prompts let a server ship a whole workflow as a menu item.++The free studio uses resources only for its widgets and lists no prompts. The four account-facing three.ws servers (the main server at `/api/mcp` and three siblings) use both since 2026-09-22, in the changelog's words: "Claude and other AI clients can now read your three.ws account and follow guided flows." The design, in [`api/_mcp/resources.js`](https://github.com/nirholas/three.ws/blob/main/api/_mcp/resources.js) and [`api/_mcp/prompts.js`](https://github.com/nirholas/three.ws/blob/main/api/_mcp/prompts.js), has decisions worth copying.++**Resources are live, read-only views under a URI scheme:** `three://me` (your account and quota), `three://agents` (the agents you own), `three://models` (every model an agent can think on). Each declares one access rule (public, any signed-in user, or a list of scopes) and reads the same tables as the REST route behind the dashboard, "so a resource can never disagree with the dashboard." An anonymous `resources/list` returns only the public ones, three today.++**Two refusals, two codes:**++```bash+curl -s https://three.ws/api/mcp -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":1,"method":"resources/read","params":{"uri":"three://me"}}'+```++```json+{"jsonrpc":"2.0","id":1,"error":{"code":-32001,"message":"three://me is account data: sign in with three.ws OAuth or an API key to read it","data":{"uri":"three://me","hint":"Connect this server with OAuth, or send Authorization: Bearer sk_live_... from /dashboard/api-keys."}}}+```++`-32001` means "authenticate or grant a scope first"; `-32002` is MCP's "resource not found." Someone else's resource answers exactly like a missing one, "so a URI cannot be used to probe which ids exist."++**A fallback for clients without a resource reader:** every resource is also reachable through a `read_resource` tool on the same server. Host support for resources is uneven; the data should not be.++**Prompts are rendered against the live tool catalog,** never a hand-kept list. A prompt is listed only where every tool it needs exists; each tool name passes through a function that throws if the server does not publish it; a test renders every prompt on every server against `tools/list`; confirmation flags are read from each tool's own `inputSchema`. The main server lists fourteen, among them `get-started`, `create-agent`, `hire-agent`, `review-costs` and `embed-avatar`.++**Prices in metadata must be real.** `three://models` is what an assistant reads to compare model costs. On 2026-09-25 we fixed it so free models read `[0, 0]` rather than `null`, and a key that routes to another model is priced at the model that answers. The concrete case was IBM's: our `ibm-granite` key names no priced model itself, so it read `null`. Live today:++```bash+curl -s https://three.ws/api/mcp -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":1,"method":"resources/read","params":{"uri":"three://models"}}' \+ | jq '.result.contents[0].text | fromjson | .agent_models[] | select(.id == "ibm-granite")'+```++```json+{+ "id": "ibm-granite",+ "label": "IBM Granite 3.8B",+ "network": "IBM watsonx.ai",+ "tier": "balanced",+ "max_output_tokens": 4096,+ "available": true,+ "free": false,+ "price_usd_per_mtok": [0.05, 0.1],+ "description": "IBM’s open, enterprise-governed foundation model on watsonx.ai."+}+```++Four days later the same resource needed a second fix: it imported a function the catalog module no longer exported, so every read answered "Could not read right now." **A resource is code and needs a test that reads it end to end.**++Discovery follows the same thinking. The 2026-09-29 entry "One link teaches any AI agent to use three.ws" is the [connect page](https://three.ws/connect?utm_source=ibm-community), with one-click setup for Claude, ChatGPT, Cursor and VS Code, plus a single Agent Skill file at `three.ws/skill.md` that tells an assistant how to reach the platform, starting with the free server for clients with no account. Every hosted server is also listed with its auth model in `https://three.ws/.well-known/mcp.json`; the 3D Studio entry reads `"auth": "none"`. A model cannot use a server it cannot find, or connect correctly to one whose auth it must guess.++---++## 12. Patterns 9 and 10: tiers, refinement, and checking the shelf first++### Pattern 9: tiers that degrade honestly, refinement that survives the wait++`forge_free` takes `tier`: `draft`, `standard` (the default) or `high`. Tiers let a caller trade time for quality, and the rule is that **a tier may degrade only in the direction its description promises, and only when nobody paid for the difference.** Free `high` runs on a scale-to-zero self-hosted GPU worker; if that lane refuses or cannot accept the job within the submit window, the call degrades to `standard` rather than failing the conversation, and the description says so ("high is slower and may fall back to standard under load"). The submit client has a `strictTier` mode that throws instead, for any surface where the caller paid for the higher tier, because silently serving less than was bought is a correctness bug. That is also why `standard`, not `high`, is the default: a default that sometimes quietly becomes something else is a broken promise.++`refine_model` iterates in plain language ("make it metallic", "add wings"). It is a real anchored regeneration, not a fake diff: the prior prompt is folded together with the change, optionally anchored by a reference image. Each refinement is appended to an immutable **version lineage** in `structuredContent.lineage`, and the lineage lives **with the client**: it passes the array back as `parent_lineage` on the next call, or targets `parent_index` to branch. Reverting is a pointer move, with no server session to expire.++A refinement can outlive the budget, so its pending result carries a `refine` object, the history the new model will join. Passed back to `check_job` unchanged, it makes the collector append the finished model exactly as `refine_model` would have inline; if it is malformed, the plain model comes back instead ("the model itself is never lost"). **When a long job continues a conversation, the continuation state must ride with the handle,** or the result arrives without its context. Related, from 2026-09-03: refine used to replay the prompt without pinning randomness, so "you asked for this model, better, and got another model, better." A seed now rides along on refine, retry and automatic engine switches.++### Pattern 10: check the shelf before you build++The cheapest long-running job is the one you never start. Before generating, search the catalog of ready-made CC0 props, rigged characters and motion clips:++```bash+curl -s https://three.ws/api/mcp-studio -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"search_catalog","arguments":{"q":"wooden chair","limit":3}}}' \+ | jq -r '.result.content[0].text'+```++Under half a second:++```text+10 matches for "wooden chair" (showing 3 from offset 0):++- `object:painted_wooden_chair_01` | Painted Wooden Chair 01 | CC0 | chair, wood, painted, farmhouse+- `object:painted_wooden_chair_02` | Painted Wooden Chair 02 | CC0 | old, wooden, vintage, antique+- `object:WoodenChair_01` | Wooden Chair 01 | CC0 | wood, prop, vintage, furniture++Kinds: object 10+Categories: furniture (10), seating (10), office (1)+More: call again with offset 3.++Next: get_item_source with an id above returns code you can paste.+```++`structuredContent` carries the items with license, size and thumbnail, plus facets, `next_offset` and the catalog `total` (3,961 items at the time of writing). `get_item_source` returns paste-ready code: a `<model-viewer>` tag pinned with its integrity hash, three.js, React, or the `<agent-3d>` web component. The result's last lines tell the model its next step, and the server instructions add: "Say whether you used an existing asset or generated a new one." A half-second search that saves a three-minute generation is this server's best latency optimization.++---++## 13. Splitting anonymous tools from authenticated ones, and watsonx Orchestrate++The catalog tools taught us the most transferable lesson here, and it is about authentication, not time.++Our main server, `https://three.ws/api/mcp`, has always listed `search_catalog`. Most of its tools act on an account, so it uses OAuth 2.1, and the 401 that starts sign-in arrives on **`initialize`**, before any tool is chosen:++```bash+curl -s -o /dev/null -D - https://three.ws/api/mcp \+ -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \+ -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"1"}}}' \+ | grep -iE '^HTTP|www-authenticate'+```++```text+HTTP/2 401+www-authenticate: Bearer resource_metadata="https://three.ws/.well-known/oauth-protected-resource", resource="https://three.ws/api/mcp"+```++That is correct MCP authorization, and Claude clients follow it automatically. But it means **a keyless tool on a server that challenges at `initialize` is not keyless for any real MCP client.** Our docs told people to add `/api/mcp` to search the catalog with no account; a real client that did was sent to sign-in. Only plain JSON-RPC calls (curl, a script that does not speak the MCP transport) were served anonymously.++The 2026-10-01 fix ("Search the 3D asset catalog from any AI client without an account") moved nothing and duplicated nothing: the keyless studio now serves the same three catalog tools from the main server's own definitions. One implementation, two transports. From [`api/_mcp-studio/catalog-tools.js`](https://github.com/nirholas/three.ws/blob/main/api/_mcp-studio/catalog-tools.js):++```js+function libraryTool(name) {+ const def = libraryTools.find((d) => d.name === name);+ if (!def) throw new Error(`api/_mcp/tools/library.js no longer defines ${name}`);+ return { ...def, description: def.description.replace(MAIN_SERVER_LEAD, 'Free, no account needed. ') };+}+```++If the main server renames a tool, the keyless server refuses to load rather than silently serving a stale copy. **Auth is a property of a server, not of a tool.** Put keyless tools on a server that never challenges, and declare each server's auth in a discovery manifest.++### Applying these patterns in watsonx Orchestrate++Many in this group build for watsonx Orchestrate, so here is what IBM documents that is relevant. Our own position first: three.ws is **not** listed in the watsonx Orchestrate Agent Catalog and has **no** Orchestrate integration today. Nothing below has been run by us against an Orchestrate tenant.++IBM's watsonx Orchestrate ADK documentation describes importing a remote MCP server as a toolkit with `orchestrate toolkits add`, using `--kind mcp`, a `--url`, and a `--transport` of `sse` or `streamable_http`. For this server that would be (flags as IBM documents them, untested by us):++```bash+orchestrate toolkits add --kind mcp \+ --name three-ws-3d-studio \+ --description "Free text-to-3D and 3D asset search" \+ --url https://three.ws/api/mcp-studio \+ --transport streamable_http \+ --tools "*"+```++Three statements on that IBM page connect directly to this post:++- At import the ADK "connects to the remote MCP server to retrieve and validate available tools," and "watsonx Orchestrate waits up to 30 seconds for the server to respond with the tool list." **Keep `tools/list` cheap.** Ours is a static catalog answering in about 130 ms.+- The ADK "checks only the tool schemas for structural correctness" and does not run tools at import. **Your timeout behavior is untested until a real call hits it,** which is why patterns 1 to 4 matter.+- At execution, end-of-file errors "usually happen when the server times out or doesn't respond in time." That is the failure pattern 3 prevents: **return your own pending envelope before the caller's clock runs out,** so what arrives is a handle, not a dropped connection.++The page also lists several authentication methods for remote MCP, including OAuth2 without dynamic client registration, API keys and bearer tokens; a keyless server for anonymous tools sidesteps that configuration entirely.++Separately, IBM's own tutorial on IBM Developer, "Building AI applications with Model Context Protocol (MCP)," builds a watsonx.ai-backed MCP server with Granite models and uses Claude Desktop as its host. If you wrap a slow watsonx.ai workload the same way (a long batch generation, a large embedding job, a forecast over a long series), these patterns apply unchanged.++To reach watsonx.ai itself from any MCP client, three.ws publishes an open-source connector, `@three-ws/ibm-watsonx-mcp` (0.2.2 on npm), that calls the watsonx.ai REST API with **your own** IBM Cloud credentials: six tools (`watsonx_chat`, `watsonx_generate`, `watsonx_embed`, `watsonx_tokenize`, `watsonx_forecast`, `watsonx_list_models`), no intermediary backend, no telemetry. It is community-built, not an IBM product, and not operated or endorsed by IBM.++---++## 14. The pattern checklist++| # | Pattern | The rule | In three.ws |+|---|---|---|---|+| 1 | Job handles | A timed-out wait on accepted work returns success with a signed, pollable handle | `pendingResult()`, `api/_mcp-studio/tools.js` |+| 2 | First-class collector | `check_job`: one probe; done, pending or failed; `retryable` on a failed check; only an unknown handle is final | `handleCheckJob()` |+| 2a | Slow once | First collection post-processes; later reads hit a cached terminal frame | done-frame cache |+| 2b | Free collection | Collecting never spends the generation quota | `GEN_TOOLS`, `handler.js` |+| 3 | Host budgets | One absolute deadline per call, well inside the host's limit, flowing into every wait | `CHATGPT_CALL_BUDGET_MS`, `ctx.deadline` |+| 3a | Backoff | Polling stretches its interval; transient errors never end the loop | `pollJob()` |+| 4 | Submit tickets | The client names the work before it starts, so a lost response never orphans a job | `forge-submit-ticket.js` |+| 5 | Guidance in text | Instructions, descriptions and result text all say when and how to collect | pending narration |+| 5a | Honest numbers | Omit an estimate you no longer have; name a boot as a boot | `pendingTiming()` |+| 6 | Interface collects | Where allowed, the widget polls, rigs and persists; the model does not loop | `component.js` |+| 7 | Two channels | Minimal, identifier-free `structuredContent`; narrating `content`; protocol and tool errors kept apart | `ok()`, `toolError()` |+| 8 | Honest annotations | A tool that writes on any path is not read-only, and a check proves it | `audit:mcp-safety` |+| 9 | Tiers, continuations | Tiers degrade only as described; continuation state rides with the handle | `tier`, `refine` |+| 10 | Shelf first | A fast search before a slow generation | `search_catalog` |+| 11 | Auth per server | Keyless tools on a server that never challenges; auth declared in a manifest | `/api/mcp-studio`, `mcp.json` |+| 12 | Beyond tools | Resources filtered by access; prompts rendered against live tools; real prices | `resources.js`, `prompts.js` |++---++## 15. Build your own: a long-running MCP tool in 150 lines of Node++Here is a complete MCP server, with no dependencies, implementing patterns 1 to 5 around a real, slow computation: counting the primes below a limit with a sieve, in time-sliced chunks so the server can answer status checks while the job runs. Nothing is simulated; a large limit simply takes a while. Save it as `server.mjs` and run it with Node 18 or later.++```js+// A minimal MCP server (Streamable HTTP, JSON-RPC over POST) whose one slow+// tool never fails work that is still running. Node 18+, no dependencies.+import http from 'node:http';+import { randomUUID } from 'node:crypto';++const CALL_BUDGET_MS = Number(process.env.CALL_BUDGET_MS || 5_000);+const jobs = new Map(); // jobId -> { status, startedAt, limit, result }+const byRequest = new Map(); // caller's request_id + limit -> jobId++// Real work: count the primes below `limit` with a sieve, in slices of at+// most ~15 ms, so the event loop stays free to answer check_job meanwhile.+function countPrimes(limit, onDone) {+ const sieve = new Uint8Array(limit + 1);+ let p = 2;+ let m = 0; // next multiple of p to strike out; 0 means p is not started+ let count = 0;+ const slice = () => {+ const until = Date.now() + 15;+ while (p <= limit) {+ if (m === 0 && !sieve[p]) {+ count++;+ m = p * p;+ }+ while (m !== 0 && m <= limit) {+ const stop = Math.min(limit, m + p * 50_000);+ for (; m <= stop; m += p) sieve[m] = 1;+ if (Date.now() >= until) return setImmediate(slice);+ }+ m = 0;+ p++;+ if ((p & 0xffff) === 0 && Date.now() >= until) return setImmediate(slice);+ }+ onDone(count);+ };+ setImmediate(slice);+}++function startJob(limit) {+ const jobId = `job_${randomUUID()}`;+ const job = { status: 'running', startedAt: Date.now(), limit };+ jobs.set(jobId, job);+ countPrimes(limit, (primes) => Object.assign(job, { status: 'done', result: { limit, primes } }));+ return jobId;+}++const sleep = (ms) => new Promise((r) => setTimeout(r, ms));++// Wait for a job, but never past the caller's deadline.+async function waitFor(jobId, deadline) {+ while (Date.now() < deadline && jobs.get(jobId).status === 'running') {+ await sleep(Math.min(250, Math.max(0, deadline - Date.now())));+ }+ return jobs.get(jobId).status !== 'running';+}++function done(job) {+ return {+ content: [{ type: 'text', text: `There are ${job.result.primes} primes below ${job.result.limit}.` }],+ structuredContent: { status: 'done', ...job.result },+ };+}++function pending(jobId) {+ const elapsed = Math.round((Date.now() - jobs.get(jobId).startedAt) / 1000);+ return {+ content: [{ type: 'text', text: `Still running (${elapsed}s so far). Call check_job with job_id ${jobId} in ~5s to collect it.` }],+ structuredContent: { status: 'pending', jobId, elapsedSeconds: elapsed, retryInSeconds: 5 },+ };+}++function toolError(message, extra = {}) {+ return { content: [{ type: 'text', text: message }], structuredContent: { error: true, message, ...extra }, isError: true };+}++const TOOLS = [+ {+ name: 'count_primes',+ description:+ 'Count the primes below a limit. Large limits take longer than one call: the result is then ' ++ 'status "pending" with a jobId. Call check_job with that jobId after retryInSeconds to collect it.',+ inputSchema: {+ type: 'object',+ required: ['limit'],+ properties: {+ limit: { type: 'integer', minimum: 2, maximum: 200_000_000 },+ request_id: { type: 'string', description: 'Optional. Reuse it on a retry to get the same job back instead of a second one.' },+ },+ },+ annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },+ },+ {+ name: 'check_job',+ description: 'Collect a pending count_primes job. One check, no waiting: returns the result, or a fresh pending state.',+ inputSchema: { type: 'object', required: ['job_id'], properties: { job_id: { type: 'string' } } },+ annotations: { readOnlyHint: true, destructiveHint: false, idempotentHint: true, openWorldHint: false },+ },+];++async function callTool(name, args, started) {+ if (name === 'count_primes') {+ const limit = Number(args.limit);+ if (!Number.isInteger(limit) || limit < 2 || limit > 200_000_000) return toolError('limit must be an integer from 2 to 200000000.');+ const key = typeof args.request_id === 'string' && args.request_id ? `${args.request_id}:${limit}` : null;+ const jobId = (key && byRequest.get(key)) || startJob(limit);+ if (key) byRequest.set(key, jobId);+ const finished = await waitFor(jobId, started + CALL_BUDGET_MS);+ return finished ? done(jobs.get(jobId)) : pending(jobId);+ }+ if (name === 'check_job') {+ const job = jobs.get(String(args.job_id || ''));+ if (!job) return toolError('Unknown job_id. Start a new count_primes call.');+ return job.status === 'done' ? done(job) : pending(args.job_id);+ }+ return null;+}++async function rpc(msg) {+ const started = Date.now();+ const { id, method, params = {} } = msg;+ const ok = (result) => ({ jsonrpc: '2.0', id, result });+ const fail = (code, message) => ({ jsonrpc: '2.0', id, error: { code, message } });+ if (method === 'initialize') {+ return ok({+ protocolVersion: '2025-06-18',+ serverInfo: { name: 'primes-demo', version: '1.0.0' },+ capabilities: { tools: { listChanged: false } },+ instructions: 'If a result is status "pending", call check_job after retryInSeconds. Do not call count_primes again for the same request.',+ });+ }+ if (method === 'notifications/initialized') return null;+ if (method === 'tools/list') return ok({ tools: TOOLS });+ if (method === 'tools/call') {+ const result = await callTool(params.name, params.arguments || {}, started);+ return result ? ok(result) : fail(-32602, `unknown tool: ${params.name}`);+ }+ return fail(-32601, `method not found: ${method}`);+}++http+ .createServer(async (req, res) => {+ if (req.method !== 'POST') {+ res.writeHead(405, { allow: 'POST' }).end();+ return;+ }+ let body = '';+ for await (const chunk of req) body += chunk;+ let out;+ try {+ out = await rpc(JSON.parse(body));+ } catch {+ out = { jsonrpc: '2.0', id: null, error: { code: -32700, message: 'parse error' } };+ }+ if (out === null) {+ res.writeHead(202).end();+ return;+ }+ res.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify(out));+ })+ .listen(Number(process.env.PORT || 8787), () => console.log(`primes-demo MCP server on :${process.env.PORT || 8787}`));+```++Run it and exercise both paths:++```bash+node server.mjs &++# A small job finishes inside the 5 s budget and returns inline.+curl -s localhost:8787 -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"count_primes","arguments":{"limit":1000000}}}'++# A large one outlives the budget and comes back pending, with a handle.+curl -s localhost:8787 -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"count_primes","arguments":{"limit":200000000,"request_id":"demo-1"}}}'+```++On our machine the first call returned `{"status":"done","limit":1000000,"primes":78498}` and the second returned pending after five seconds:++```json+{"status":"pending","jobId":"job_fd42dd59-ed96-4def-936e-937faee491a2","elapsedSeconds":5,"retryInSeconds":5}+```++A `check_job` a few seconds later returned `{"status":"done","limit":200000000,"primes":11078937}`; repeating the original call with the same `request_id` returned that same job instead of starting a second sieve; an unknown handle returned an `isError` result telling the caller to start over.++What the toy leaves out, and production needs:++- **A shared job store.** The `Map` lives in one process; behind a load balancer the collect request lands elsewhere. three.ws keeps job state in a shared cache, with per-instance memory only as a degraded fallback.+- **Signed handles.** Ours carry HMAC-signed routing information, so any instance can resolve them and nobody can forge one.+- **Expiry.** Our submit tickets and cached finished frames are kept for six hours, longer than any client's collection loop.+- **Truthful annotations.** This toy only computes, so read-only is honest. The moment your collector persists anything, it is not (section 10).+- **Abuse limits that never block collection** (section 4).++---++## 16. Honest limits++- **ChatGPT's 60-second limit is our observed, designed-for behavior,** documented by us as not configurable from the server. Other hosts' limits vary and change; measure the hosts you target.+- **The interface-side collector only exists where the host lets a widget call tools.** Elsewhere the model collects, and that depends on how well it follows the text.+- **Polling is not push.** The 3D Studio answers every request synchronously over `POST`, with no server-initiated stream. Polling with honest ETAs is the lowest common denominator that works on every host we target today.+- **Free capacity is finite.** The studio allows 4 generations a minute and 30 an hour per caller (on ChatGPT keyed on the anonymized per-user id it sends, with 300 an hour per source IP in total), plus a platform-wide hourly breaker. The caps fail open if the limiter itself is down, by design, because the lanes behind them are free to run.+- **Quality varies by lane and prompt.** The automated gate scored our teapot 0.914, and `look_at_model` lets a model inspect results, but neither replaces a human looking.+- **The watsonx Orchestrate notes in section 13 are untested by us,** and three.ws has no Orchestrate listing or integration.++---++## 17. Try it, and three questions for the group++All of this runs with no account, key, or payment:++```bash+# discover the server and its fourteen tools+curl -s https://three.ws/api/mcp-studio -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | jq '[.result.tools[].name]'++# check the shelf first+curl -s https://three.ws/api/mcp-studio -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"search_catalog","arguments":{"q":"desk lamp","limit":3}}}' \+ | jq -r '.result.content[0].text'++# start a draft generation; on the ChatGPT surface it answers within the 40 s budget+curl -s https://three.ws/api/mcp-chatgpt -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"forge_free","arguments":{"prompt":"a ceramic coffee mug with a chipped rim","tier":"draft"}}}' \+ | jq '.result.structuredContent'++# if it came back pending, collect it with the jobId it returned+curl -s https://three.ws/api/mcp-studio -H 'content-type: application/json' \+ -d '{"jsonrpc":"2.0","id":4,"method":"tools/call","params":{"name":"check_job","arguments":{"job_id":"PASTE_THE_jobId_HERE"}}}' \+ | jq '.result.structuredContent'+```++To use the tools from Claude, ChatGPT, Cursor or VS Code, add `https://three.ws/api/mcp-studio` as a remote MCP server with no authentication, or follow the [connect page](https://three.ws/connect?utm_source=ibm-community). Source is Apache-2.0 at [github.com/nirholas/three.ws](https://github.com/nirholas/three.ws), with the files cited here under `api/_mcp-studio/` and `api/_lib/`.++Three questions I would like this group to argue about:++1. **Should the collector be a tool at all?** We publish `check_job` because hosts render tools. MCP also has progress notifications and resource subscriptions. If you have shipped a long-running tool on a host that honors those, did they replace polling, or did you keep a collector tool as the fallback anyway?+2. **Where should the call budget live?** We hardcode 40 seconds for one host because we measured it. Should hosts advertise their tool-call timeout to servers, in `initialize` say, so every server can budget without guessing? If you build on watsonx Orchestrate, what limits have you observed at tool execution time?+3. **How honest should annotations be about bookkeeping writes?** `check_job` is marked not read-only because its first call persists the model. Strictly true, but a cautious client may then ask the user before checking on a job. Would you want a finer-grained hint in the spec, such as "writes only the server's own state"?++Reply in the comments or start a thread. If you build long-running tools on watsonx.ai or anywhere else, post what broke; this post exists because we wrote ours down.++_three.ws is an IBM Business Partner. The `/api/ibm/*` surfaces and the `@three-ws/ibm-watsonx-mcp` connector are independent developer tools built on IBM's publicly available Granite models; they are not IBM products and not endorsed by IBM. The 3D Studio MCP server described here is a three.ws product._diff --git a/public/x-media/claims-ledger-thread/editorial-bar.png b/public/x-media/claims-ledger-thread/editorial-bar.pngnew file mode 100644index 000000000..29ce30802Binary files /dev/null and b/public/x-media/claims-ledger-thread/editorial-bar.png differdiff --git a/public/x-media/claims-ledger-thread/reel-cut.png b/public/x-media/claims-ledger-thread/reel-cut.pngnew file mode 100644index 000000000..dc19f3122Binary files /dev/null and b/public/x-media/claims-ledger-thread/reel-cut.png differdiff --git a/public/x-media/claims-ledger-thread/reel-stamp.png b/public/x-media/claims-ledger-thread/reel-stamp.pngnew file mode 100644index 000000000..c0e538b9bBinary files /dev/null and b/public/x-media/claims-ledger-thread/reel-stamp.png differdiff --git a/public/x-media/immutable-embed-releases-thread/docs-cdn-versioning.png b/public/x-media/immutable-embed-releases-thread/docs-cdn-versioning.pngnew file mode 100644index 000000000..1fc2d1b87Binary files /dev/null and b/public/x-media/immutable-embed-releases-thread/docs-cdn-versioning.png differdiff --git a/public/x-media/immutable-embed-releases-thread/same-url-two-pins.png b/public/x-media/immutable-embed-releases-thread/same-url-two-pins.pngnew file mode 100644index 000000000..3231ec0ddBinary files /dev/null and b/public/x-media/immutable-embed-releases-thread/same-url-two-pins.png differdiff --git a/public/x-media/open-weight-3d-forge-thread/engine-picker.png b/public/x-media/open-weight-3d-forge-thread/engine-picker.pngnew file mode 100644index 000000000..5f739259dBinary files /dev/null and b/public/x-media/open-weight-3d-forge-thread/engine-picker.png differdiff --git a/public/x-media/open-weight-3d-forge-thread/teapot-model-info.png b/public/x-media/open-weight-3d-forge-thread/teapot-model-info.pngnew file mode 100644index 000000000..04442182bBinary files /dev/null and b/public/x-media/open-weight-3d-forge-thread/teapot-model-info.png differdiff --git a/public/x-media/open-weight-3d-forge-thread/teapot-trellis-draft.png b/public/x-media/open-weight-3d-forge-thread/teapot-trellis-draft.pngnew file mode 100644index 000000000..f9d5d241fBinary files /dev/null and b/public/x-media/open-weight-3d-forge-thread/teapot-trellis-draft.png differdiff --git a/public/x-media/universal-retargeting-thread/rig-doctor-mixamo-report.png b/public/x-media/universal-retargeting-thread/rig-doctor-mixamo-report.pngnew file mode 100644index 000000000..d35b55519Binary files /dev/null and b/public/x-media/universal-retargeting-thread/rig-doctor-mixamo-report.png differdiff --git a/public/x-media/universal-retargeting-thread/rig-doctor-partial-verdict.png b/public/x-media/universal-retargeting-thread/rig-doctor-partial-verdict.pngnew file mode 100644index 000000000..7f8bbe127Binary files /dev/null and b/public/x-media/universal-retargeting-thread/rig-doctor-partial-verdict.png differdiff --git a/public/x-media/universal-retargeting-thread/rig-doctor-walk-preview.png b/public/x-media/universal-retargeting-thread/rig-doctor-walk-preview.pngnew file mode 100644index 000000000..34a94e31cBinary files /dev/null and b/public/x-media/universal-retargeting-thread/rig-doctor-walk-preview.png differ