Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
fix(seo): give tutorials a crawlable body and stop submitting pages that answer noindex
nirholas355235e9b773
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 141b4125d..a68fd48eb 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -6,6 +6,7 @@ Public history for [three.ws](https://three.ws), newest first. New pages come fr## 2026-09-07+- **Docs · SEO and crawler rendering** (`/docs/seo`): How a JavaScript site is served to search engines: the four server-side render paths, the contract that keeps a page and its sitemap entry agreeing, and how to audit the live origin.- **My Creations** (`/mine`): Everything you have made on three.ws in one newest-first grid: the agents you built, the 3D avatars you saved, and every model you forged, including models forged in this browser before you had an account. Open, edit, or copy a link to any of them in one click.- **3D generation and the whole site keep working when our asset storage does not**: Early on 2026-09-07 our object storage stopped accepting the platform's credential, and because every generation parks its reference image there, 3D generation stopped on the website and in ChatGPT at the same time, agent registration could not store its manifest, and images loaded through our CDN stopped resolving. Three things changed so that a storage fault is never again a blank page. Avatars, thumbnails and models now fall back to the bucket's public domain automatically, so the site keeps rendering. When generation genuinely cannot run you get a plain message saying storage is recovering and to try again shortly, instead of a raw error from the storage vendor. And our own health checks now watch the storage credential directly and treat a silent forge as an outage rather than a quiet hour, so we hear about this class of failure before you do. `[fix, infra]`- **A payment we cannot read now comes back with the price list, not a bare error**: When an agent pays one of our x402 endpoints and the payment header arrives mangled (truncated in transit, double-encoded, wrapped by a proxy), the answer used to be a plain 400 error with no prices in it, so the agent had nothing left to work from and no way to retry. Every rejected payment now returns the ordinary 402 with the full list of accepted rails, tokens and amounts attached, exactly as it does when a payment is simply missing. That is what the x402 spec and the OKX seller SDK both do, and it means a paying agent can always read the quote again and try once more. Only a real server fault answers without a quote. (`/docs/okx-marketplace`) `[fix, improvement]`@@ -23,6 +24,7 @@ Public history for [three.ws](https://three.ws), newest first. New pages come fr- **The featured agent is one you can actually open**: The agent we spotlight is picked automatically from real activity, but the pick only checked whether an agent was public, not whether its owner had published it. That let a brand new starter agent, the draft every account is given on signup, take the spot, and every panel that tried to read it came back empty with a live feed error. The pick now requires a published agent, so the spotlight lands on one that every part of the site can open. `[fix]`- **The partner-hosted demo page loads all of its pieces again**: Our partnership page is published once onto a partner's own website, so everything it loads comes from three.ws across origins. Two of its scripts were being refused by the browser there, which quietly cost the page its in-editor map and its iOS handoff. Both are now served with the same cross-origin permission the rest of the page already had, and an audit that loads the page from a foreign origin now passes end to end. `[fix]`- **Turn a photo into a 3D model without leaving your editor**: The Blender tools could already build a model from a written description. They now build one from a picture too: point at an image on your machine or a link to one, add a few words to steer it if you like, and the finished model opens in Blender. It runs on the same free lane as the text version and takes about as long. `[feature, sdk]`+- **Tutorials and walkthroughs are now readable by search engines, and pages we never wanted indexed stopped asking to be**: The 76 tutorial and walkthrough pages loaded their content after the page opened, which meant a search engine reading the raw page found no text at all, and on tutorials found the words 'Tutorial not found' sitting under an empty heading. Those pages now arrive with their full text already in them, so they can finally be found by the questions they answer. Separately, eighteen pages were being submitted to Google while telling Google not to index them: sign-in surfaces, viewers that need a link to work, and one endpoint that returns data rather than a page. They have been withdrawn from the sitemap, and a new check refuses to let a page ask for indexing and refuse it at the same time. Two documentation pages that shared a title with the product page they described have been retitled. (`/tutorials`) `[improvement, docs]`- **World tiles show their artwork again**: The picker that lets you choose a community world was loading each world's artwork straight from a public storage gateway, and browsers now refuse to draw images served that way. Nine tiles on the page failed silently every visit. The artwork is now fetched through our own image service, so tiles paint their real picture, and a world whose artwork has gone missing falls back to its lettered tile instead of an empty square. `[fix]`## 2026-09-05@@ -197,10 +199,10 @@ Public history for [three.ws](https://three.ws), newest first. New pages come fr## 2026-09-01- **3D Drops** (`/drops`): Launch a supply-capped collection of AI-forged 3D characters. Pick a style, define weighted trait layers, and three.ws rolls the whole supply from a published seed: verifiable rarity, rigged animation-ready art, forged on reveal.-- **3D Drops** (`/docs/drops`): How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.- **Agent Spotlight** (`/spotlight`): The three.ws community showcase: agents people actually built something with, written up by the builder and ranked by community upvotes. Filter by category, open any entry in 3D, upvote what you like, and submit your own.- **Docs · Agent Spotlight** (`/docs/spotlight`): How the Agent Spotlight community showcase works: what an entry stores versus what it reads live off the agent, the trending decay formula, curated versus community entries, and the full showcase API.- **Filming three.ws on a Seeker without a Seeker** (`/docs/seeker-video`): How to record a Seeker screen recording of the app with no device: why the Trusted Web Activity makes a browser capture at the real panel geometry a genuine capture, and which moments still have to come from an Android emulator.+- **How 3D Drops collections work** (`/docs/drops`): How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.- **Page performance: the rules a three.ws page plays by** (`/docs/page-performance`): How three.ws keeps real 3D on ordinary pages without making them slow: previews that spin only while someone is looking, models that load when they are wanted, boxes reserved before late content arrives, and images fetched at the size they are shown.- **$THREE tiers page: the ladder now loads for everyone, signed in or not**: Opening /three without a wallet connected or an account signed in showed an error card instead of the tiers, because the tier lookup refused anonymous requests. The page now shows every tier, its perks, and the exact $THREE threshold to any visitor, with a connect-wallet or sign-in prompt to see your own position. Signed-in accounts without a linked wallet get a clear prompt to connect one, and if the ladder genuinely fails to load the page keeps its heading and offers a retry. (`/three`) `[fix]`- **3D previews now wake up when you look at them**: Second half of the performance sweep. Every rotating 3D preview on the site (the Marketplace grid, the Discover directory, the home page's community strip, the Create page's base-avatar card) used to spin for as long as the tab stayed open, which kept the page busy even when nobody was watching. They now turn when they scroll into view, settle once they have turned far enough to read as 3D, and turn again when you hover or focus them, with no motion at all if your system asks for reduced motion. The Create page's card shows a picture of the base avatar straight away and loads the live, rotatable model when you reach for it, so opening the page no longer waits on a 748 KB download. Forge stops downloading the 3D viewer until you actually generate something. Three things that used to appear late and shove the page down (the site header, the Marketplace hero, the weekly theme strip) now hold their space from the first frame. Coin art in the /play world and Marketplace card art are fetched at the size they are displayed instead of at full resolution. (`/marketplace`) `[improvement, fix]`@@ -252,11 +254,11 @@ Public history for [three.ws](https://three.ws), newest first. New pages come fr- **Glance** (`/glance`): Your three.ws agent on every home screen: a live card for the Windows 11 widgets board, a README, Slack, or any page.- **Glance: agent cards for home screens** (`/docs/glance`): Put a three.ws agent on the Windows 11 widgets board, in a README, in Slack, on a page, or in your terminal. Endpoint, element, widget worker and npm client.- **Herald** (`/herald`): Deliver a message in person: a 3D character walks onto your page and tells the user what happened. Open SDK, one line of code, an accessible fallback, and a CLI that announces when your build finishes.+- **How Portal builds a 3D world from a URL** (`/docs/portal`): Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.- **Knock** (`/knock`): Put a price on reaching you. Anyone, a person or an agent, can pay it to get exactly one message through, and your 3D companion walks on screen and delivers it out loud with their name and what they paid. The USDC settles straight to your wallet.- **Model Diff** (`/diff`): Drop two .glb files and see exactly what changed: geometry, materials, textures, skeletons and animation clips, with git-style rename and move detection and a severity you can gate a build on. Overlay, wipe, or set the two models side by side in 3D. Runs in your browser, nothing is uploaded.- **Oracle Lab: the model, published** (`/oracle-lab`): The Oracle's conviction model in full: every bucket weight with the number of labeled launches behind it and what happened to them, the held-out reliability curve for each tier, the version history of every refit including the ones that were refused, and a live scorer that runs the real weights in your browser. Download it and reproduce any score offline.- **Portal** (`/portal`): Paste any web address and walk through it in 3D. Portal reads the page's real structure and builds a place from it: sections become buildings sized by what they say, links become doors you step into, images become billboards. Download the world as a GLB or embed it in your own site.-- **Portal: walk any website in 3D** (`/docs/portal`): Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.- **Progressive Avatars** (`/stream`): A3S packs a GLB into nested layers whose first 50 KB is already a complete, skinned, spec-valid avatar, then refines it in place over plain HTTP range requests. Live side-by-side against a classic GLB load.- **Publishing to the Solana dApp Store** (`/docs/seeker-publishing`): The release runbook for the three.ws Seeker app: Publisher Portal setup, the release keystore, building and verifying a signed APK, and submitting it for review.- **Ship log** (`/ship`): Every release note joined to the commits that produced it, with a confidence score and the reasoning behind every link.diff --git a/data/changelog.json b/data/changelog.jsonindex 51187e78a..d86efefab 100644--- a/data/changelog.json+++ b/data/changelog.json@@ -1,6 +1,13 @@{"$comment": "Curated changelog entries for three.ws, the editorial layer of the public changelog. New PAGE launches are tracked automatically via the `added` field in data/pages.json; this file is for everything else holders care about: improvements, fixes, SDK releases, trading/payments capabilities, security work. scripts/build-page-index.mjs merges both sources into CHANGELOG.md, public/changelog.json, and public/changelog.xml. Newest first. Allowed tags: feature, improvement, fix, sdk, infra, docs, security. `link` is optional and must be a live three.ws path.","entries": [+ {+ "date": "2026-09-07",+ "title": "Tutorials and walkthroughs are now readable by search engines, and pages we never wanted indexed stopped asking to be",+ "summary": "The 76 tutorial and walkthrough pages loaded their content after the page opened, which meant a search engine reading the raw page found no text at all, and on tutorials found the words 'Tutorial not found' sitting under an empty heading. Those pages now arrive with their full text already in them, so they can finally be found by the questions they answer. Separately, eighteen pages were being submitted to Google while telling Google not to index them: sign-in surfaces, viewers that need a link to work, and one endpoint that returns data rather than a page. They have been withdrawn from the sitemap, and a new check refuses to let a page ask for indexing and refuse it at the same time. Two documentation pages that shared a title with the product page they described have been retitled.",+ "tags": ["improvement", "docs"],+ "link": "/tutorials"+ },{"date": "2026-09-07","title": "Reference-image upload now tells you to paste a URL instead of failing with a network error",diff --git a/data/guards.json b/data/guards.jsonindex e7d1d972b..f3f05bf7f 100644--- a/data/guards.json+++ b/data/guards.json@@ -191,8 +191,8 @@"script": "scripts/audit-page-index.mjs","npm": "audit:pages","title": "Route documentation",- "protects": "Every human-facing route in vercel.json is documented in data/pages.json.",- "why": "data/pages.json feeds the sitemap, llms.txt, features.json, and the changelog. A page missing from it is invisible to crawlers and to the changelog, so it ships without ever being announced.",+ "protects": "Every human-facing route in vercel.json is documented in data/pages.json, and every page submitted for indexing is crawlable and index-allowed.",+ "why": "data/pages.json feeds the sitemap, llms.txt, features.json, and the changelog. A page missing from it is invisible to crawlers and to the changelog, so it ships without ever being announced. It also cross-checks indexability, because the manifest decides what gets submitted while the page itself decides what a crawler may do with it: seventeen pages were being submitted while answering noindex, plus a JSON endpoint submitted as a page while robots.txt disallowed its path. Search Console files that difference as an error against us, and neither half looks wrong on its own.","stages": ["prebuild","gate"diff --git a/data/pages.json b/data/pages.jsonindex 9c91c97df..ea3ab9d32 100644--- a/data/pages.json+++ b/data/pages.json@@ -266,7 +266,8 @@"description": "View all the skills and subscriptions you've unlocked on three.ws.","priority": 0.5,"changefreq": "weekly",- "added": "2026-06-17"+ "added": "2026-06-17",+ "indexable": false},{"path": "/portal",@@ -310,7 +311,8 @@"description": "Every payment, sale, remix, follow, quest, and royalty notification on three.ws: all in one place, filterable by category.","priority": 0.5,"changefreq": "daily",- "added": "2026-07-12"+ "added": "2026-07-12",+ "indexable": false},{"path": "/skills",@@ -431,7 +433,8 @@"description": "Live avatars, agents, 3D models, worlds, and launches across three.ws: filter to everyone or just the people you follow.","priority": 0.6,"changefreq": "daily",- "added": "2026-06-21"+ "added": "2026-06-21",+ "indexable": false},{"path": "/partners",@@ -484,7 +487,8 @@"title": "Lab Demos","description": "Index of live demos across the platform: viewers, agents, payments, and experiments.","priority": 0.6,- "changefreq": "weekly"+ "changefreq": "weekly",+ "indexable": false},{"path": "/demos/agents",@@ -754,7 +758,8 @@"description": "Build an embeddable 3D widget: pick an avatar (or your own model URL), pick a widget type, tune it live, and copy the snippet. No account needed for the URL-based embeds.","priority": 0.8,"changefreq": "weekly",- "added": "2026-05-24"+ "added": "2026-05-24",+ "indexable": false},{"path": "/artifact",@@ -782,7 +787,7 @@},{"path": "/docs/portal",- "title": "Portal: walk any website in 3D",+ "title": "How Portal builds a 3D world from a URL","description": "Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.","priority": 0.6,"changefreq": "monthly",@@ -938,7 +943,8 @@"description": "Import any GLB or glTF avatar into three.ws and give it an agent brain.","priority": 0.6,"changefreq": "monthly",- "added": "2026-05-25"+ "added": "2026-05-25",+ "indexable": false},{"path": "/threews/claim",@@ -993,7 +999,8 @@"title": "TRELLIS NIM Forge","description": "Drop in a photo, get a textured 3D model, reconstructed on a self-hosted Microsoft TRELLIS NIM. Single-hop image to GLB.","priority": 0.6,- "changefreq": "weekly"+ "changefreq": "weekly",+ "indexable": false},{"path": "/avatar-sdk",@@ -1016,7 +1023,8 @@"description": "Accept an invitation to somebody's connected home. The link says which home it is, exactly what your role would let you do there and what it would never let you do, before you sign in. It works once and it expires.","priority": 0.4,"changefreq": "yearly",- "added": "2026-09-03"+ "added": "2026-09-03",+ "indexable": false},{"path": "/smart-home/privacy",@@ -1188,7 +1196,8 @@"description": "Preview any public glTF/GLB inline, then take it further: pose and animate it, restyle the material, build a scene, place it in AR, embed it on your site, or launch it as a pump.fun coin.","priority": 0.5,"changefreq": "monthly",- "added": "2026-07-08"+ "added": "2026-07-08",+ "indexable": false},{"path": "/watch",@@ -1196,7 +1205,8 @@"description": "Live progress page for a running text-to-3D generation: real countdown, the painted concept art, and an automatic hand-off into the interactive viewer the moment the model is ready.","priority": 0.4,"changefreq": "monthly",- "added": "2026-07-29"+ "added": "2026-07-29",+ "indexable": false},{"path": "/ar",@@ -1222,7 +1232,8 @@"description": "Place any three.ws GLB in your real space from a plain link: real ARKit Quick Look on iPhone (a genuine on-device USDZ conversion, not a camera overlay), Scene Viewer or WebXR on Android. The shared landing page behind /api/ar and every AR hand-off across the site.","priority": 0.3,"changefreq": "monthly",- "added": "2026-08-19"+ "added": "2026-08-19",+ "indexable": false},{"path": "/daily",@@ -1563,7 +1574,8 @@"description": "Your personal autonomous coin launcher: set your mode (trend, meme, hybrid, random), pick trend sources, tune your cadence, preview what your agents would mint, then go live: real pump.fun launches on your schedule, self-funded from your own agents' wallets under a hard daily SOL cap.","priority": 0.8,"changefreq": "daily",- "added": "2026-06-27"+ "added": "2026-06-27",+ "indexable": false},{"path": "/clash",@@ -1629,7 +1641,8 @@"description": "Live command center for all 4 autonomous agent capabilities: Alpha Hunt smart-money scoring, autonomous Coin Launcher on schedule, Creator Auto-Claim for fee harvesting, and Market Maker with Jito execution.","priority": 0.7,"changefreq": "daily",- "added": "2026-06-27"+ "added": "2026-06-27",+ "indexable": false},{"path": "/arena",@@ -2273,7 +2286,8 @@"description": "An embeddable 3D AI avatar that holds a Solana wallet, chats, and can autonomously send SOL.","priority": 0.5,"changefreq": "weekly",- "added": "2026-06-02"+ "added": "2026-06-02",+ "indexable": false},{"path": "/demo",@@ -2345,7 +2359,8 @@"description": "Rich article reader for crypto news: full story extraction with AI summary, key points, sentiment, detected tickers, and related coverage from across the industry.","priority": 0.5,"changefreq": "hourly",- "added": "2026-07-10"+ "added": "2026-07-10",+ "indexable": false},{"path": "/markets/archive",@@ -2418,7 +2433,8 @@"title": "Agent Wallet","description": "The wallet surface for your agents: balances, funding, and activity.","priority": 0.6,- "changefreq": "weekly"+ "changefreq": "weekly",+ "indexable": false},{"path": "/wallet",@@ -2426,7 +2442,8 @@"description": "One custodial wallet for your account, separate from your agents'. Hold SOL and USDC across Solana and Base, top up any agent you own, send to any Solana address, and read every transaction straight from chain.","priority": 0.6,"changefreq": "weekly",- "added": "2026-07-30"+ "added": "2026-07-30",+ "indexable": false},{"path": "/trading",@@ -3065,7 +3082,7 @@},{"path": "/docs/drops",- "title": "3D Drops",+ "title": "How 3D Drops collections work","description": "How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.","priority": 0.6,"changefreq": "monthly",@@ -3673,7 +3690,7 @@},{"path": "/docs/vscode",- "title": "Docs \u00b7 three.ws in VS Code",+ "title": "Docs · three.ws in VS Code","description": "The VS Code extension that renders .glb and .gltf files in an editor tab, reports on them, generates and rigs new models on the free lane, and writes a pinned <agent-3d> embed.","added": "2026-09-05"},@@ -4494,6 +4511,14 @@"changefreq": "monthly","added": "2026-07-30"},+ {+ "path": "/docs/seo",+ "title": "Docs · SEO and crawler rendering",+ "description": "How a JavaScript site is served to search engines: the four server-side render paths, the contract that keeps a page and its sitemap entry agreeing, and how to audit the live origin.",+ "priority": 0.5,+ "changefreq": "monthly",+ "added": "2026-09-07"+ },{"path": "/tutorials/wallet-sign-in","title": "Tutorial · Add Wallet Sign-In to Your App",@@ -6227,7 +6252,8 @@"description": "A ready-to-paste allowlist telling an AI client which three.ws MCP tools it may run without asking you. Three profiles from strict to open, in Claude, VS Code, or portable format, derived from the tool safety labels that are verified against each handler on every build.","priority": 0.5,"changefreq": "weekly",- "added": "2026-07-31"+ "added": "2026-07-31",+ "indexable": false},{"path": "/llms.txt",diff --git a/docs/guards.md b/docs/guards.mdindex 031c21102..08b8715ab 100644--- a/docs/guards.md+++ b/docs/guards.md@@ -105,7 +105,7 @@ Run `npm run audit:guards` to print the current count and per-stage breakdown. T| Guard | Command | Protects ||---|---|---|-| Route documentation | `npm run audit:pages` | Every human-facing route is documented in `data/pages.json`. |+| Route documentation | `npm run audit:pages` | Every human-facing route is documented in `data/pages.json`, and no page is submitted for indexing while its own HTML answers noindex or robots.txt disallows its path. || Declared pages resolve | `npm run check:pages` | Every page declared in `data/pages.json` is actually reachable. || Routing and 404 model | `npm run audit:routes` | Catalog pages reachable, unknown paths reach the designed 404, no shadowed routes. || Route shadowing | `npm run audit:route-shadowing` | Every handler under `api/` that looks routed can actually be reached through the production router's walk of `vercel.json`. |diff --git a/docs/seo.md b/docs/seo.mdnew file mode 100644index 000000000..00d0d075f--- /dev/null+++ b/docs/seo.md@@ -0,0 +1,60 @@+# How three.ws is served to search engines++three.ws is a JavaScript application, and search engines index HTML. Most of the site's pages are shells whose content arrives after a fetch, so every surface that matters for search has a server-side path that puts real text in the response before any script runs.++This page maps those paths, names the file that owns each one, and describes the two contracts that keep them honest. Read it before changing how any page is rendered, and before adding a route to a sitemap.++## The four render paths++| Surface | What the crawler gets | Owner |+|---|---|---|+| Catalogued static pages (`/`, `/create`, `/markets`, and 800 others) | The page's own `<title>`, description, canonical, Open Graph tags and JSON-LD, rewritten per path into a shared shell | [server/seo-head.mjs](../server/seo-head.mjs) |+| Content pages behind a fetch (`/tutorials/<slug>`, `/walkthroughs/<slug>`) | The head above, plus the rendered markdown or walkthrough steps injected into the shell body | [server/crawler-body.mjs](../server/crawler-body.mjs) |+| Entity pages (`/agents/<id>`, `/avatars/<id>`) | A full server-rendered page: name, description, facts, tags, breadcrumbs and JSON-LD, routed by User-Agent | [api/_lib/crawler-page.js](../api/_lib/crawler-page.js) |+| News permalinks (`/markets/news/<month>/<id>-<slug>`) | A server-rendered story page with `NewsArticle` and `BreadcrumbList` JSON-LD and a bounded excerpt | [api/news/story-page.js](../api/news/story-page.js) |++Sitemaps are generated per entity type by [api/sitemap/[type].js](../api/sitemap/%5Btype%5D.js): `core` reads `data/pages.json`, the entity sitemaps read the database, and `news` reads the archive. `sitemap.xml` is the index over all six.++## Contract 1: a page and its manifest entry must agree++`data/pages.json` decides what is submitted to search engines. The page itself decides what a crawler may do once it arrives. When those disagree the crawler obeys the page, and Search Console records the difference as an error against the site.++Two ways to disagree, both caught by `npm run audit:pages`:++- The manifest submits a page whose HTML answers `<meta name="robots" content="noindex">`. Reported as "Submitted URL marked noindex".+- The manifest submits a path that `robots.txt` disallows. Reported as "Indexed, though blocked by robots.txt", and the crawler never sees the page well enough to drop it.++The fix for either is one line: set `"indexable": false` on the manifest entry. The page stays reachable, stays in the human sitemap with an `internal` badge, and leaves the XML sitemap. Signed-in surfaces, viewers that need a query parameter, and API endpoints all belong in that category.++`robots.txt` deliberately does **not** disallow `/dashboard`, `/settings` or `/my-agents`: those pages carry a `noindex` tag, and a crawler has to be allowed to fetch a page to read the tag that removes it. Disallowing them is what kept them in the index as bare URLs.++## Contract 2: server content is replaced, never merged++Every server-rendered body here is written into a container the client player overwrites wholesale on load (`article.innerHTML = …`, `root.innerHTML = …`). A visitor with JavaScript therefore sees exactly one version of the page, and a crawler without it sees a complete one.++Two consequences worth knowing before you edit a shell:++- The server render and the client render must agree on structure, or the page changes shape as it hydrates. The tutorial shell's hero owns the `<h1>`, so both [server/crawler-body.mjs](../server/crawler-body.mjs) and the player strip the leading `# Title` line out of the markdown before rendering it. Skip that on either side and the document ends up with two competing top-level headings.+- If a shell's markup changes so the injection no longer matches, `crawler-body.mjs` logs and returns the untouched shell rather than serving a half-rendered page. A blank body that still answers 200 is the failure mode to avoid.++## Auditing++```bash+npm run audit:seo # the live site, core pages+npm run audit:seo -- --limit 40 # a quick pass+npm run audit:seo -- --sitemap agents # another URL set+npm run audit:seo -- --base http://localhost:3000+npm run audit:seo -- --strict # exit 1 on any error+```++[scripts/audit-seo.mjs](../scripts/audit-seo.mjs) reads a sitemap, fetches every URL in it as Googlebot, and reports what a search engine would act on: dead or redirecting sitemap entries, pages answering noindex, missing or non-self-referencing canonicals, duplicate titles and descriptions across the corpus, missing `h1`, absent Open Graph tags, and structured data that fails to parse or carries a truncated character.++It reads the live origin on purpose. Only the served response proves what a crawler receives after routing, the shell rewrite and the CDN, so point `--base` at a preview origin to check a change before it ships.++Errors are things a search engine acts on immediately; warnings are quality signals. A title over 65 characters or a description over 165 is truncated in results rather than dropped, so those are reported as warnings, not failures.++## Related++- [docs/guards.md](./guards.md): the full guard registry, including `audit:seo` and `audit:pages`+- [api/_lib/safe-text.js](../api/_lib/safe-text.js): why every string that reaches JSON-LD is truncated on grapheme boundaries+- [STRUCTURE.md](../STRUCTURE.md): where each product surface livesdiff --git a/docs/start-here.md b/docs/start-here.mdindex 611051ff3..097deb715 100644--- a/docs/start-here.md+++ b/docs/start-here.md@@ -180,6 +180,7 @@ Deeper references that don't fit a track above but answer real questions:- [Docs World](./docs-world.md): these same docs as a walkable 3D world, where every section is a pavilion you can stroll up to and read inside the scene- [Agent Monitor](./monitor.md): the ops room at [/monitor](https://three.ws/monitor), twelve live panels covering fleet activity, money, revenue, creations and uptime, plus how to add a panel- [Agent Spotlight](./spotlight.md): the community showcase at [/spotlight](https://three.ws/spotlight), what an entry stores versus what it reads live off the agent, the trending decay formula, and the full showcase API+- [How three.ws is served to search engines](./seo.md): the four server-side render paths behind a JavaScript site, the two contracts that keep a page and its sitemap entry agreeing, and how to audit the live origin- [Design tokens](./DESIGN-TOKENS.md): the canonical design vocabulary (colors, spacing, type, motion) every surface builds from- [NVIDIA models on three.ws](./nvidia-models.md): the free hosted inference layer model by model, one key behind text-to-3D, chat, vision, embeddings, safety, and speech- [Run an inference node](./inference-node-operator.md): join the supply side of the open inference network with your own hardware, from clone to a signed, verified result, on CPU or an NVIDIA GPUdiff --git a/pages/tutorial.html b/pages/tutorial.htmlindex 3a3801932..382f7e444 100644--- a/pages/tutorial.html+++ b/pages/tutorial.html@@ -1387,8 +1387,13 @@// Turn `figure:` directives into inert slots BEFORE parsing. Left as// <img src="figure:…">, the browser fires a request the moment the// markup lands and logs ERR_UNKNOWN_URL_SCHEME for every figure.+ // Drop the markdown's own `# Title`: the hero above already renders+ // it as the page's <h1>, so parsing it again gives the document+ // two competing top-level headings. server/crawler-body.mjs+ // strips the same line, so the crawler and the reader agree.+ const mdBody = md.replace(/^\s*#\s+.+?[ \t]*(?:\r?\n|$)/, '');article.innerHTML = marked.parse(- window.tutorialFigures ? window.tutorialFigures.preprocess(md) : md,+ window.tutorialFigures ? window.tutorialFigures.preprocess(mdBody) : mdBody,);// Freshness: a tutorial is a promise that these steps still work,diff --git a/public/changelog.json b/public/changelog.jsonindex 1949fc382..7021b554f 100644--- a/public/changelog.json+++ b/public/changelog.json@@ -1,11 +1,21 @@{- "generated_at": "2026-09-07T20:01:21.224Z",+ "generated_at": "2026-09-07T20:35:47.680Z","generated_by": "scripts/build-page-index.mjs from data/pages.json + data/changelog.json","site": {"name": "three.ws","url": "https://three.ws"},"entries": [+ {+ "date": "2026-09-07",+ "type": "launch",+ "title": "Docs · SEO and crawler rendering",+ "summary": "How a JavaScript site is served to search engines: the four server-side render paths, the contract that keeps a page and its sitemap entry agreeing, and how to audit the live origin.",+ "link": "/docs/seo",+ "tags": [+ "launch"+ ]+ },{"date": "2026-09-07","type": "launch",@@ -188,6 +198,17 @@"sdk"]},+ {+ "date": "2026-09-07",+ "type": "update",+ "title": "Tutorials and walkthroughs are now readable by search engines, and pages we never wanted indexed stopped asking to be",+ "summary": "The 76 tutorial and walkthrough pages loaded their content after the page opened, which meant a search engine reading the raw page found no text at all, and on tutorials found the words 'Tutorial not found' sitting under an empty heading. Those pages now arrive with their full text already in them, so they can finally be found by the questions they answer. Separately, eighteen pages were being submitted to Google while telling Google not to index them: sign-in surfaces, viewers that need a link to work, and one endpoint that returns data rather than a page. They have been withdrawn from the sitemap, and a new check refuses to let a page ask for indexing and refuse it at the same time. Two documentation pages that shared a title with the product page they described have been retitled.",+ "link": "/tutorials",+ "tags": [+ "improvement",+ "docs"+ ]+ },{"date": "2026-09-07","type": "update",@@ -1874,16 +1895,6 @@"launch"]},- {- "date": "2026-09-01",- "type": "launch",- "title": "3D Drops",- "summary": "How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.",- "link": "/docs/drops",- "tags": [- "launch"- ]- },{"date": "2026-09-01","type": "launch",@@ -1914,6 +1925,16 @@"launch"]},+ {+ "date": "2026-09-01",+ "type": "launch",+ "title": "How 3D Drops collections work",+ "summary": "How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.",+ "link": "/docs/drops",+ "tags": [+ "launch"+ ]+ },{"date": "2026-09-01","type": "launch",@@ -2384,6 +2405,16 @@"launch"]},+ {+ "date": "2026-08-28",+ "type": "launch",+ "title": "How Portal builds a 3D world from a URL",+ "summary": "Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.",+ "link": "/docs/portal",+ "tags": [+ "launch"+ ]+ },{"date": "2026-08-28","type": "launch",@@ -2424,16 +2455,6 @@"launch"]},- {- "date": "2026-08-28",- "type": "launch",- "title": "Portal: walk any website in 3D",- "summary": "Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.",- "link": "/docs/portal",- "tags": [- "launch"- ]- },{"date": "2026-08-28","type": "launch",diff --git a/public/changelog.xml b/public/changelog.xmlindex db4b69397..19512f113 100644--- a/public/changelog.xml+++ b/public/changelog.xml@@ -6,6 +6,14 @@<atom:link href="https://three.ws/changelog.xml" rel="self" type="application/rss+xml"/><description>What's new on three.ws: features, improvements, and releases, newest first.</description><language>en</language>+ <item>+ <title>Docs · SEO and crawler rendering</title>+ <link>https://three.ws/docs/seo</link>+ <guid isPermaLink="false">2026-09-07:Docs · SEO and crawler rendering</guid>+ <pubDate>Mon, 07 Sep 2026 12:00:00 GMT</pubDate>+ <category>launch</category>+ <description>How a JavaScript site is served to search engines: the four server-side render paths, the contract that keeps a page and its sitemap entry agreeing, and how to audit the live origin.</description>+ </item><item><title>My Creations</title><link>https://three.ws/mine</link>@@ -142,6 +150,14 @@<category>feature</category><description>The Blender tools could already build a model from a written description. They now build one from a picture too: point at an image on your machine or a link to one, add a few words to steer it if you like, and the finished model opens in Blender. It runs on the same free lane as the text version and takes about as long.</description></item>+ <item>+ <title>Tutorials and walkthroughs are now readable by search engines, and pages we never wanted indexed stopped asking to be</title>+ <link>https://three.ws/tutorials</link>+ <guid isPermaLink="false">2026-09-07:Tutorials and walkthroughs are now readable by search engines, and pages we never wanted indexed stopped asking to be</guid>+ <pubDate>Mon, 07 Sep 2026 12:00:00 GMT</pubDate>+ <category>improvement</category>+ <description>The 76 tutorial and walkthrough pages loaded their content after the page opened, which meant a search engine reading the raw page found no text at all, and on tutorials found the words 'Tutorial not found' sitting under an empty heading. Those pages now arrive with their full text already in them, so they can finally be found by the questions they answer. Separately, eighteen pages were being submitted to Google while telling Google not to index them: sign-in surfaces, viewers that need a link to work, and one endpoint that returns data rather than a page. They have been withdrawn from the sitemap, and a new check refuses to let a page ask for indexing and refuse it at the same time. Two documentation pages that shared a title with the product page they described have been retitled.</description>+ </item><item><title>World tiles show their artwork again</title><link>https://three.ws/changelog</link>@@ -1414,14 +1430,6 @@<category>launch</category><description>Launch a supply-capped collection of AI-forged 3D characters. Pick a style, define weighted trait layers, and three.ws rolls the whole supply from a published seed: verifiable rarity, rigged animation-ready art, forged on reveal.</description></item>- <item>- <title>3D Drops</title>- <link>https://three.ws/docs/drops</link>- <guid isPermaLink="false">2026-09-01:3D Drops</guid>- <pubDate>Tue, 01 Sep 2026 12:00:00 GMT</pubDate>- <category>launch</category>- <description>How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.</description>- </item><item><title>Agent Spotlight</title><link>https://three.ws/spotlight</link>@@ -1446,6 +1454,14 @@<category>launch</category><description>How to record a Seeker screen recording of the app with no device: why the Trusted Web Activity makes a browser capture at the real panel geometry a genuine capture, and which moments still have to come from an Android emulator.</description></item>+ <item>+ <title>How 3D Drops collections work</title>+ <link>https://three.ws/docs/drops</link>+ <guid isPermaLink="false">2026-09-01:How 3D Drops collections work</guid>+ <pubDate>Tue, 01 Sep 2026 12:00:00 GMT</pubDate>+ <category>launch</category>+ <description>How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.</description>+ </item><item><title>Page performance: the rules a three.ws page plays by</title><link>https://three.ws/docs/page-performance</link>@@ -1806,6 +1822,14 @@<category>launch</category><description>Deliver a message in person: a 3D character walks onto your page and tells the user what happened. Open SDK, one line of code, an accessible fallback, and a CLI that announces when your build finishes.</description></item>+ <item>+ <title>How Portal builds a 3D world from a URL</title>+ <link>https://three.ws/docs/portal</link>+ <guid isPermaLink="false">2026-08-28:How Portal builds a 3D world from a URL</guid>+ <pubDate>Fri, 28 Aug 2026 12:00:00 GMT</pubDate>+ <category>launch</category>+ <description>Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.</description>+ </item><item><title>Knock</title><link>https://three.ws/knock</link>@@ -1838,14 +1862,6 @@<category>launch</category><description>Paste any web address and walk through it in 3D. Portal reads the page's real structure and builds a place from it: sections become buildings sized by what they say, links become doors you step into, images become billboards. Download the world as a GLB or embed it in your own site.</description></item>- <item>- <title>Portal: walk any website in 3D</title>- <link>https://three.ws/docs/portal</link>- <guid isPermaLink="false">2026-08-28:Portal: walk any website in 3D</guid>- <pubDate>Fri, 28 Aug 2026 12:00:00 GMT</pubDate>- <category>launch</category>- <description>Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.</description>- </item><item><title>Progressive Avatars</title><link>https://three.ws/stream</link>diff --git a/public/features.json b/public/features.jsonindex 99909cff0..990d4a5b4 100644--- a/public/features.json+++ b/public/features.json@@ -1,5 +1,5 @@{- "generated_at": "2026-09-07T06:40:26.852Z",+ "generated_at": "2026-09-07T20:35:47.664Z","generated_by": "scripts/build-page-index.mjs from data/pages.json","site": {"name": "three.ws",@@ -312,7 +312,7 @@"description": "View all the skills and subscriptions you've unlocked on three.ws.","added": "2026-06-17","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -362,7 +362,7 @@"description": "Every payment, sale, remix, follow, quest, and royalty notification on three.ws: all in one place, filterable by category.","added": "2026-07-12","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -512,7 +512,7 @@"description": "Live avatars, agents, 3D models, worlds, and launches across three.ws: filter to everyone or just the people you follow.","added": "2026-06-21","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -582,7 +582,7 @@"description": "Index of live demos across the platform: viewers, agents, payments, and experiments.","added": null,"auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -909,7 +909,7 @@"description": "Build an embeddable 3D widget: pick an avatar (or your own model URL), pick a widget type, tune it live, and copy the snippet. No account needed for the URL-based embeds.","added": "2026-05-24","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -945,7 +945,7 @@},{"path": "/docs/portal",- "title": "Portal: walk any website in 3D",+ "title": "How Portal builds a 3D world from a URL","description": "Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.","added": "2026-08-28","auth": null,@@ -1139,7 +1139,7 @@"description": "Import any GLB or glTF avatar into three.ws and give it an agent brain.","added": "2026-05-25","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -1209,7 +1209,7 @@"description": "Drop in a photo, get a textured 3D model, reconstructed on a self-hosted Microsoft TRELLIS NIM. Single-hop image to GLB.","added": null,"auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -1239,7 +1239,7 @@"description": "Accept an invitation to somebody's connected home. The link says which home it is, exactly what your role would let you do there and what it would never let you do, before you sign in. It works once and it expires.","added": "2026-09-03","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -1446,7 +1446,7 @@"description": "Preview any public glTF/GLB inline, then take it further: pose and animate it, restyle the material, build a scene, place it in AR, embed it on your site, or launch it as a pump.fun coin.","added": "2026-07-08","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -1456,7 +1456,7 @@"description": "Live progress page for a running text-to-3D generation: real countdown, the painted concept art, and an automatic hand-off into the interactive viewer the moment the model is ready.","added": "2026-07-29","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -1486,7 +1486,7 @@"description": "Place any three.ws GLB in your real space from a plain link: real ARKit Quick Look on iPhone (a genuine on-device USDZ conversion, not a camera overlay), Scene Viewer or WebXR on Android. The shared landing page behind /api/ar and every AR hand-off across the site.","added": "2026-08-19","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -1903,7 +1903,7 @@"description": "Your personal autonomous coin launcher: set your mode (trend, meme, hybrid, random), pick trend sources, tune your cadence, preview what your agents would mint, then go live: real pump.fun launches on your schedule, self-funded from your own agents' wallets under a hard daily SOL cap.","added": "2026-06-27","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -1983,7 +1983,7 @@"description": "Live command center for all 4 autonomous agent capabilities: Alpha Hunt smart-money scoring, autonomous Coin Launcher on schedule, Creator Auto-Claim for fee harvesting, and Market Maker with Jito execution.","added": "2026-06-27","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -2773,7 +2773,7 @@"description": "An embeddable 3D AI avatar that holds a Solana wallet, chats, and can autonomously send SOL.","added": "2026-06-02","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -2863,7 +2863,7 @@"description": "Rich article reader for crypto news: full story extraction with AI summary, key points, sentiment, detected tickers, and related coverage from across the industry.","added": "2026-07-10","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -2963,7 +2963,7 @@"description": "The wallet surface for your agents: balances, funding, and activity.","added": null,"auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -2973,7 +2973,7 @@"description": "One custodial wallet for your account, separate from your agents'. Hold SOL and USDC across Solana and Base, top up any agent you own, send to any Solana address, and read every transaction straight from chain.","added": "2026-07-30","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},@@ -3750,7 +3750,7 @@},{"path": "/docs/drops",- "title": "3D Drops",+ "title": "How 3D Drops collections work","description": "How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.","added": "2026-09-01","auth": null,@@ -5748,6 +5748,16 @@"tags": [],"showcase": false},+ {+ "path": "/docs/seo",+ "title": "Docs · SEO and crawler rendering",+ "description": "How a JavaScript site is served to search engines: the four server-side render paths, the contract that keeps a page and its sitemap entry agreeing, and how to audit the live origin.",+ "added": "2026-09-07",+ "auth": null,+ "indexable": true,+ "tags": [],+ "showcase": false+ },{"path": "/tutorials/wallet-sign-in","title": "Tutorial · Add Wallet Sign-In to Your App",@@ -7985,7 +7995,7 @@"description": "A ready-to-paste allowlist telling an AI client which three.ws MCP tools it may run without asking you. Three profiles from strict to open, in Claude, VS Code, or portable format, derived from the tool safety labels that are verified against each handler on every build.","added": "2026-07-31","auth": null,- "indexable": true,+ "indexable": false,"tags": [],"showcase": false},diff --git a/public/guards.json b/public/guards.jsonindex e7d1d972b..f3f05bf7f 100644--- a/public/guards.json+++ b/public/guards.json@@ -191,8 +191,8 @@"script": "scripts/audit-page-index.mjs","npm": "audit:pages","title": "Route documentation",- "protects": "Every human-facing route in vercel.json is documented in data/pages.json.",- "why": "data/pages.json feeds the sitemap, llms.txt, features.json, and the changelog. A page missing from it is invisible to crawlers and to the changelog, so it ships without ever being announced.",+ "protects": "Every human-facing route in vercel.json is documented in data/pages.json, and every page submitted for indexing is crawlable and index-allowed.",+ "why": "data/pages.json feeds the sitemap, llms.txt, features.json, and the changelog. A page missing from it is invisible to crawlers and to the changelog, so it ships without ever being announced. It also cross-checks indexability, because the manifest decides what gets submitted while the page itself decides what a crawler may do with it: seventeen pages were being submitted while answering noindex, plus a JSON endpoint submitted as a page while robots.txt disallowed its path. Search Console files that difference as an error against us, and neither half looks wrong on its own.","stages": ["prebuild","gate"diff --git a/public/llms-full.txt b/public/llms-full.txtindex a0a4494f2..5f14c80fe 100644--- a/public/llms-full.txt+++ b/public/llms-full.txt@@ -191,6 +191,7 @@ Both sides of the moment a metered free trial runs out: the trials you are holdi### My CollectionURL: https://three.ws/collection+Indexable: no (excluded from sitemap)View all the skills and subscriptions you've unlocked on three.ws.@@ -221,6 +222,7 @@ Deliver a message in person: a 3D character walks onto your page and tells the u### NotificationsURL: https://three.ws/notifications+Indexable: no (excluded from sitemap)Every payment, sale, remix, follow, quest, and royalty notification on three.ws: all in one place, filterable by category.@@ -312,6 +314,7 @@ Give any website an AI concierge with a talking 3D avatar: one-tag embed, stream### FeedURL: https://three.ws/feed+Indexable: no (excluded from sitemap)Live avatars, agents, 3D models, worlds, and launches across three.ws: filter to everyone or just the people you follow.@@ -354,6 +357,7 @@ Browse every ERC-8004 agent that carries a 3D glTF body, indexed from canonical### Lab DemosURL: https://three.ws/demos+Indexable: no (excluded from sitemap)Index of live demos across the platform: viewers, agents, payments, and experiments.@@ -556,6 +560,7 @@ Khronos-spec validator for glTF/GLB files. Paste a URL or upload a model.### Widget StudioURL: https://three.ws/studio+Indexable: no (excluded from sitemap)Build an embeddable 3D widget: pick an avatar (or your own model URL), pick a widget type, tune it live, and copy the snippet. No account needed for the URL-based embeds.@@ -577,7 +582,7 @@ URL: https://three.ws/docs/brownoutEvery response says where its data came from and how fresh it is, and every fallback the API claims has been executed for real against a genuinely refusing provider, inside the request path a user hits.-### Portal: walk any website in 3D+### How Portal builds a 3D world from a URLURL: https://three.ws/docs/portal@@ -694,6 +699,7 @@ The home screen of the three.ws app on Solana Seeker: one-tap Seed Vault sign-in### Import an avatar URLURL: https://three.ws/import/rpm+Indexable: no (excluded from sitemap)Import any GLB or glTF avatar into three.ws and give it an agent brain.@@ -736,6 +742,7 @@ Build a custom 3D avatar from scratch. Sculpt the face and body, dress it from t### TRELLIS NIM ForgeURL: https://three.ws/nim-forge+Indexable: no (excluded from sitemap)Drop in a photo, get a textured 3D model, reconstructed on a self-hosted Microsoft TRELLIS NIM. Single-hop image to GLB.@@ -754,6 +761,7 @@ Turn any 3D model into a real printed object. Live printability analysis and aut### Join a homeURL: https://three.ws/smart-home/join+Indexable: no (excluded from sitemap)Accept an invitation to somebody's connected home. The link says which home it is, exactly what your role would let you do there and what it would never let you do, before you sign in. It works once and it expires.@@ -878,12 +886,14 @@ Standalone Three.js viewer for avatar artifacts: embeddable and shareable withou### glTF / GLB ViewerURL: https://three.ws/viewer+Indexable: no (excluded from sitemap)Preview any public glTF/GLB inline, then take it further: pose and animate it, restyle the material, build a scene, place it in AR, embed it on your site, or launch it as a pump.fun coin.### Generation WatchURL: https://three.ws/watch+Indexable: no (excluded from sitemap)Live progress page for a running text-to-3D generation: real countdown, the painted concept art, and an automatic hand-off into the interactive viewer the moment the model is ready.@@ -902,6 +912,7 @@ A live multi-model AR scene through your camera: place any number of 3D models a### View in ARURL: https://three.ws/ar/view+Indexable: no (excluded from sitemap)Place any three.ws GLB in your real space from a plain link: real ARKit Quick Look on iPhone (a genuine on-device USDZ conversion, not a camera overlay), Scene Viewer or WebXR on Android. The shared landing page behind /api/ar and every AR hand-off across the site.@@ -1152,6 +1163,7 @@ Check any Solana or Ethereum wallet against tracked airdrop criteria: a real on-### Memetic LauncherURL: https://three.ws/launcher+Indexable: no (excluded from sitemap)Your personal autonomous coin launcher: set your mode (trend, meme, hybrid, random), pick trend sources, tune your cadence, preview what your agents would mint, then go live: real pump.fun launches on your schedule, self-funded from your own agents' wallets under a hard daily SOL cap.@@ -1202,6 +1214,7 @@ Verified traders publish their live entry/exit signals as x402-metered feeds. Yo### Capabilities: Alpha Hunt, Launcher, Auto-Claim & Market MakerURL: https://three.ws/dashboard/capabilities+Indexable: no (excluded from sitemap)Live command center for all 4 autonomous agent capabilities: Alpha Hunt smart-money scoring, autonomous Coin Launcher on schedule, Creator Auto-Claim for fee harvesting, and Market Maker with Jito execution.@@ -1677,6 +1690,7 @@ Every autonomous action your three.ws agent took: each grounded in the memory th### Avatar WalletURL: https://three.ws/avatar-wallet-chat+Indexable: no (excluded from sitemap)An embeddable 3D AI avatar that holds a Solana wallet, chats, and can autonomously send SOL.@@ -1731,6 +1745,7 @@ Live crypto news aggregated natively from 192 publisher feeds in 18 languages: f### Crypto News ReaderURL: https://three.ws/markets/news/article+Indexable: no (excluded from sitemap)Rich article reader for crypto news: full story extraction with AI summary, key points, sentiment, detected tickers, and related coverage from across the industry.@@ -1791,12 +1806,14 @@ Coordinated groups of trading agents working a shared strategy, with live positi### Agent WalletURL: https://three.ws/agent-wallet+Indexable: no (excluded from sitemap)The wallet surface for your agents: balances, funding, and activity.### Your master walletURL: https://three.ws/wallet+Indexable: no (excluded from sitemap)One custodial wallet for your account, separate from your agents'. Hold SOL and USDC across Solana and Base, top up any agent you own, send to any Solana address, and read every transaction straight from chain.@@ -2279,7 +2296,7 @@ URL: https://three.ws/docs/news-rightsWhat three.ws shows from a publisher's article and what it never shows: the enforced lead-excerpt limit, how a withdrawn story is removed (410, deindex, purge), and how publishers can ask us to take something down.-### 3D Drops+### How 3D Drops collections workURL: https://three.ws/docs/drops@@ -3479,6 +3496,12 @@ URL: https://three.ws/docs/guardsThe local guards that protect this codebase in place of CI: what each one blocks, which stage runs it, and how to add a new one.+### Docs · SEO and crawler rendering++URL: https://three.ws/docs/seo++How a JavaScript site is served to search engines: the four server-side render paths, the contract that keeps a page and its sitemap entry agreeing, and how to audit the live origin.+### Tutorial · Add Wallet Sign-In to Your AppURL: https://three.ws/tutorials/wallet-sign-in@@ -4819,6 +4842,7 @@ XML sitemap for search-engine crawlers.### MCP trust policyURL: https://three.ws/api/mcp-policy+Indexable: no (excluded from sitemap)A ready-to-paste allowlist telling an AI client which three.ws MCP tools it may run without asking you. Three profiles from strict to open, in Claude, VS Code, or portable format, derived from the tool safety labels that are verified against each handler on every build.diff --git a/public/llms.txt b/public/llms.txtindex c8d94a7fc..54ea55c01 100644--- a/public/llms.txt+++ b/public/llms.txt@@ -60,12 +60,10 @@ Entry points into the product.- [Marketplace](https://three.ws/marketplace): Buy access to agents, skills, and avatars from other creators.- [Marketplace Analytics](https://three.ws/marketplace/analytics): Real-time stats on the three.ws skill marketplace: top skills, top agents, and sales volume.- [Trial Conversions](https://three.ws/conversions): Both sides of the moment a metered free trial runs out: the trials you are holding with the runs you have left and the price to keep them, and the trials running on skills you sell with the revenue waiting in the queue.-- [My Collection](https://three.ws/collection): View all the skills and subscriptions you've unlocked on three.ws.- [Portal](https://three.ws/portal): Paste any web address and walk through it in 3D. Portal reads the page's real structure and builds a place from it: sections become buildings sized by what they say, links become doors you step into, images become billboards. Download the world as a GLB or embed it in your own site.- [Companion](https://three.ws/companion): A 3D companion that delivers your important messages in person. Connect your own Telegram bot, calendar, inbox, or phone, give each contact their own avatar and voice, and hear only what clears your bar.- [Knock](https://three.ws/knock): Put a price on reaching you. Anyone, a person or an agent, can pay it to get exactly one message through, and your 3D companion walks on screen and delivers it out loud with their name and what they paid. The USDC settles straight to your wallet.- [Herald](https://three.ws/herald): Deliver a message in person: a 3D character walks onto your page and tells the user what happened. Open SDK, one line of code, an accessible fallback, and a CLI that announces when your build finishes.-- [Notifications](https://three.ws/notifications): Every payment, sale, remix, follow, quest, and royalty notification on three.ws: all in one place, filterable by category.- [Community](https://three.ws/community): Live platform activity plus where to follow, contribute, and connect with three.ws creators.- [Crew HQ](https://three.ws/crews): Found a crew, invite your people, and watch the whole roster stand in one 3D headquarters. Every member appears as their own agent, lit when they are live in the world.- [Search](https://three.ws/search): One search across every creation type on three.ws: avatars, agents, 3D models, worlds, and coins. Remix a model straight from the results.@@ -79,14 +77,12 @@ Entry points into the product.- [Tour Atlas](https://three.ws/tour/atlas): Every stop on the guided tour as a real screenshot of the live site, with the feature circled by the tour's own spotlight. Search 263 stops, filter by chapter, read what the guide says about each one, and drop the tour onto any feature in a single click.- [Walk Anywhere on the Web](https://three.ws/walk): Your 3D avatar walks anywhere on the web. Live demos across six environments, a Chrome extension, one-tag embeds, and a global distance leaderboard.- [Concierge: AI Chat Widget](https://three.ws/concierge): Give any website an AI concierge with a talking 3D avatar: one-tag embed, streaming answers grounded in the live page, voice in and out, themes, and an avatar picker. Free and open source.-- [Feed](https://three.ws/feed): Live avatars, agents, 3D models, worlds, and launches across three.ws: filter to everyone or just the people you follow.- [Partner Ecosystem](https://three.ws/partners): three.ws partner ecosystem: OpenAI, IBM, AWS, Google Cloud, Alibaba Cloud, NVIDIA, Quicknode, HackerNoon. The AI, cloud, and infrastructure programs powering 3D AI agents at enterprise scale.- [three.ws is an OpenAI Select Partner](https://three.ws/openai): three.ws is an OpenAI Select Partner in the OpenAI Partner Network. Connect the free three.ws 3D Studio MCP server to ChatGPT and turn a prompt into an interactive, riggable, AR-ready 3D model inline in the conversation.- [three.ws on NVIDIA](https://three.ws/nvidia): three.ws is a member of the NVIDIA Inception program. Every 3D model, avatar, and voice on the platform runs on NVIDIA: a free hosted lane behind one API key covering text-to-3D, chat, vision, embeddings, safety and speech, plus a self-hosted fleet of Cloud Run GPU workers on L4 and RTX PRO 6000.- [Press kit](https://three.ws/press): Logos, announcement graphics, boilerplate, and usage rules for three.ws. Every asset is downloadable and current, and editorial use needs no permission.- [Pricing](https://three.ws/pricing): Plans and pricing for three.ws: free forge quota, premium passes, and pay-per-use API surfaces.- [Showcase](https://three.ws/showcase): Browse every ERC-8004 agent that carries a 3D glTF body, indexed from canonical registries across 22 chains, and open any of them in the live viewer.-- [Lab Demos](https://three.ws/demos): Index of live demos across the platform: viewers, agents, payments, and experiments.## Build@@ -119,11 +115,10 @@ Create and customize an agent.- [Scene Composer](https://three.ws/compose): Real-time 3D scene composer: forge items from text, attach them to your avatar's skeleton bones, arrange and scale freely, then export your creation or save it as an outfit.- [Animation Studio](https://three.ws/pose): Pose any three.ws avatar or the built-in mannequin with FK/IK, keyframe a timeline, and preview the motion live. Export an animated GLB or clip JSON, save animations to your account, play them back across the platform, and sell them for USDC.- [glTF Validator](https://three.ws/validation): Khronos-spec validator for glTF/GLB files. Paste a URL or upload a model.-- [Widget Studio](https://three.ws/studio): Build an embeddable 3D widget: pick an avatar (or your own model URL), pick a widget type, tune it live, and copy the snippet. No account needed for the URL-based embeds.- [Artifact Viewer](https://three.ws/artifact): Renders Claude artifact bundles as standalone embeddable apps.- [create-agent: one command, one 3D character](https://three.ws/docs/create-agent): npm create @three-ws/agent turns a sentence into a rigged, downloadable 3D character with a working demo page. No account, no API key.- [Brownout: proven fallbacks, not promised ones](https://three.ws/docs/brownout): Every response says where its data came from and how fresh it is, and every fallback the API claims has been executed for real against a genuinely refusing provider, inside the request path a user hits.-- [Portal: walk any website in 3D](https://three.ws/docs/portal): Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.+- [How Portal builds a 3D world from a URL](https://three.ws/docs/portal): Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.- [Glance: agent cards for home screens](https://three.ws/docs/glance): Put a three.ws agent on the Windows 11 widgets board, in a README, in Slack, on a page, or in your terminal. Endpoint, element, widget worker and npm client.- [Native widgets: your agent on the home screen](https://three.ws/docs/native-widgets): Put your three.ws agent on an Android home screen (and the Windows 11 widgets board): what the widget shows, how to link and revoke it, every state it can be in, and the endpoints behind it.- [Glance](https://three.ws/glance): Your three.ws agent on every home screen: a live card for the Windows 11 widgets board, a README, Slack, or any page.@@ -142,17 +137,14 @@ Create and customize an agent.- [Deploy: On-Chain Agent Identity](https://three.ws/features/deploy): Register your AI agent on Solana via ERC-8004 and Metaplex Core. Permanent, verifiable identity: discoverable by any wallet or other agent.- [Selfie to Avatar](https://three.ws/create/selfie): One selfie. About a minute. A rigged 3D avatar that works everywhere.- [three.ws for Seeker](https://three.ws/seeker): The home screen of the three.ws app on Solana Seeker: one-tap Seed Vault sign-in, selfie to avatar, your agents, the marketplace, and Seeker Genesis Token verification.-- [Import an avatar URL](https://three.ws/import/rpm): Import any GLB or glTF avatar into three.ws and give it an agent brain.- [Claim threews.sol Subdomain](https://three.ws/threews/claim): Claim your own <name>.threews.sol subdomain and get a Brave-resolvable personal showcase page.- [Hands-free voice for your home](https://three.ws/voice/home): Talk to your house without finding a phone: a wake word matched on your own device, real barge-in mid-sentence, and a spoken confirmation that a background yes can never satisfy.- [Voice Lab](https://three.ws/voice): Browse hundreds of voices across Microsoft Edge, Gemini, NVIDIA, OpenAI and ElevenLabs, preview any of them, or clone your own from a short recording, then give it to your agent.- [Make Dad a 3D Avatar](https://three.ws/dad): Turn one photo of your dad into a recognizable, rigged, animated 3D avatar. It idles, walks, and waves, and you can share a permalink.- [CA → x402](https://three.ws/ca2x402): Paste any token contract address and get a live, agent-payable x402 endpoint for its market intel (price, momentum, and a bullish/bearish signal) for $0.01 USDC, discoverable in the x402 bazaar.- [Avatar Studio](https://three.ws/avatar-studio): Build a custom 3D avatar from scratch. Sculpt the face and body, dress it from the closet, add accessories, and walk it around the stage. No selfie required.-- [TRELLIS NIM Forge](https://three.ws/nim-forge): Drop in a photo, get a textured 3D model, reconstructed on a self-hosted Microsoft TRELLIS NIM. Single-hop image to GLB.- [Avatar SDK](https://three.ws/avatar-sdk): The @three-ws/avatar package: embed the platform 3D avatars in any site with a few lines of JavaScript.- [Materialize](https://three.ws/materialize): Turn any 3D model into a real printed object. Live printability analysis and automatic repair, a true-scale AR preview on your own desk, an itemized price from the mesh's own measured volume, and USDC checkout on Solana.-- [Join a home](https://three.ws/smart-home/join): Accept an invitation to somebody's connected home. The link says which home it is, exactly what your role would let you do there and what it would never let you do, before you sign in. It works once and it expires.- [Your home data](https://three.ws/smart-home/privacy): Everything three.ws stores about the homes you connected, with why it exists and how long it stays. Set how long your action log lives and watch the old entries go, download all of it as JSON, or delete one home or every trace of all of them. Room names, device states and voice audio are never stored at all.- [Home plan](https://three.ws/smart-home/plan): What your plan covers for connected homes: homes, household seats, live screens, voice minutes and agent turns, each with what you have used and when it resets. Pause and resume which homes are live without disconnecting any of them. Locking up, closing a garage and arming an alarm are never limited.- [Smart home](https://three.ws/smart-home): Connect a Home Assistant house to your three.ws agent. Your agent reads every room and device, runs the scenes you already built, and stops to ask before it unlocks, opens or disarms anything.@@ -177,11 +169,8 @@ Experimental features actively shipping. Expect fast iteration.- [Lipsync (TTS)](https://three.ws/lipsync): Type text, stream it through the avatar's voice, and the mouth animates in real time via wawa-lipsync viseme detection.- [Lipsync (Mic)](https://three.ws/lipsync/mic): Feed live mic audio through the viseme analyser and watch the avatar's mouth track your voice in real time.- [Avatar Artifact](https://three.ws/avatar-artifact): Standalone Three.js viewer for avatar artifacts: embeddable and shareable without any wrapper page.-- [glTF / GLB Viewer](https://three.ws/viewer): Preview any public glTF/GLB inline, then take it further: pose and animate it, restyle the material, build a scene, place it in AR, embed it on your site, or launch it as a pump.fun coin.-- [Generation Watch](https://three.ws/watch): Live progress page for a running text-to-3D generation: real countdown, the painted concept art, and an automatic hand-off into the interactive viewer the moment the model is ready.- [AR Forge](https://three.ws/ar): Type a prompt, get a real 3D model, and place it in your room: free text-to-3D with one-tap AR on any phone (WebXR, Scene Viewer, Quick Look) and a QR handoff from desktop.- [AR Studio](https://three.ws/ar/studio): A live multi-model AR scene through your camera: place any number of 3D models and avatars in your real space, pull in your forge creations, community models, the CC0 object library, or any GLB link, and generate new models by prompt without leaving the view.-- [View in AR](https://three.ws/ar/view): Place any three.ws GLB in your real space from a plain link: real ARKit Quick Look on iPhone (a genuine on-device USDZ conversion, not a camera overlay), Scene Viewer or WebXR on Android. The shared landing page behind /api/ar and every AR hand-off across the site.- [Daily Forge](https://three.ws/daily): A new 3D creative challenge every day. Forge today's theme into a real 3D model, free and no sign-up, build a daily streak, and place your creation in AR on your phone.- [Spatial MCP: reference renderer](https://three.ws/spatial-mcp): The open standard for returning a live, interactive 3D scene as a first-class MCP tool result, with a framework-free reference renderer that displays any conformant artifact: including ones transformed from other tools.- [Playground](https://three.ws/playground): Sandbox for experimenting with agents, prompts, and 3D scenes.@@ -227,14 +216,12 @@ On-chain features: pay-per-call, token launches, vanity wallets.- [Deploy On-chain](https://three.ws/deploy-onchain): Mint your agent into the Metaplex Agent Registry on Solana, in the exact shape the Genesis 333 shipped: a Metaplex Core asset with its own wallet, EIP-8004 identity, verified creator, royalties, and immutable metadata. Phantom, Solflare, or any Solana wallet signs; keys never leave the browser, and devnet rehearsal is free.- [Wallet Portfolio](https://three.ws/portfolio): A live portfolio view for any Solana or Ethereum wallet: total value, 24h move with honest coverage, stablecoin/major/token breakdown, top-asset allocation, and every holding priced in real time. Free, keyless, deep-linkable, no account.- [Airdrop Checker](https://three.ws/airdrops): Check any Solana or Ethereum wallet against tracked airdrop criteria: a real on-chain activity scan, a 0-100 eligibility score per program, exactly what is missing with next steps, and the manual protocol actions a scan cannot see. Free, keyless, deep-linkable.-- [Memetic Launcher](https://three.ws/launcher): Your personal autonomous coin launcher: set your mode (trend, meme, hybrid, random), pick trend sources, tune your cadence, preview what your agents would mint, then go live: real pump.fun launches on your schedule, self-funded from your own agents' wallets under a hard daily SOL cap.- [Coin Clash](https://three.ws/clash): Token-gated community warfare. Every coin community is an army: hold the coin, enlist, and rally for your faction in timed battles against other communities.- [Trader Leaderboard](https://three.ws/leaderboard): Pump.fun traders ranked by a provable, on-chain track record: realized P&L, win rate, and drawdown, with every number traceable to its transaction.- [KOL Tracker](https://three.ws/tracker): Solana crypto KOLs ranked by realized P&L computed from their own wallet's on-chain trade history, with win rate, volume, trade count, and X follower counts for verified handles.- [Agent Labor Market](https://three.ws/labor-market): A live machine economy: agents post bounties, bid on each other's work, and settle in $THREE on-chain: escrowed, verified, and royalty-routed, with no human in the loop.- [Back-an-Agent Vaults](https://three.ws/vaults): Copy-trade a 3D agent with a verifiable on-chain reputation: stake into a verified trader, share its real P&L pro-rata, and watch it trade your capital live: segregated custody, hard spend limits, and a drawdown circuit breaker.- [Signal Marketplace: paid alpha feeds, ranked by proven edge](https://three.ws/signals): Verified traders publish their live entry/exit signals as x402-metered feeds. Your agent pays per signal or per epoch in USDC and auto-mirrors the trade through the firewall + MEV engine: every spend guarded, every fill audited. Each signal binds to a real on-chain fill and accrues a realized outcome, so feeds rank by proven accuracy, never follower count.-- [Capabilities: Alpha Hunt, Launcher, Auto-Claim & Market Maker](https://three.ws/dashboard/capabilities): Live command center for all 4 autonomous agent capabilities: Alpha Hunt smart-money scoring, autonomous Coin Launcher on schedule, Creator Auto-Claim for fee harvesting, and Market Maker with Jito execution.- [The Arena: live PvP trading tournaments](https://three.ws/arena): Time-boxed PvP tournaments where AI agents compete on real, verified pump.fun P&L. Live rankings, on-chain attested standings, and $THREE prizes for the winners.- [Trending: Top Agents & Coins on three.ws](https://three.ws/trending): What’s hot on three.ws right now. Top 3D AI agents ranked by real chat activity and top Oracle conviction coins by score, with 24h/7d/all-time windows.- [Live Trade Feed: Notable pump.fun Exits](https://three.ws/trades): Real-time feed of every notable pump.fun exit from three.ws AI agents: realized PnL, hold time, exit multiple, conviction tier, and a one-click path to copy the trader.@@ -313,7 +300,6 @@ On-chain features: pay-per-call, token launches, vanity wallets.- [Agent Economy Volume](https://three.ws/agent-economy-volume): Live dashboard of total agent-to-agent economy volume: real USDC settled between AI agents hiring each other for paid skills over x402, with top earners, top spenders, and a 90-day volume chart.- [Agent Commerce](https://three.ws/agent-trade): Watch two AI agents pay each other on-chain using x402: a live Solana transaction, rendered in 3D.- [Autopilot Activity](https://three.ws/autopilot-activity): Every autonomous action your three.ws agent took: each grounded in the memory that motivated it, signed, and reversible. Explainable autonomy you can audit.-- [Avatar Wallet](https://three.ws/avatar-wallet-chat): An embeddable 3D AI avatar that holds a Solana wallet, chats, and can autonomously send SOL.- [Agent Economy: NOVA & ORACLE](https://three.ws/demo): Two three.ws AI agents with real Solana wallets discover services on the x402 bazaar, pay each other in SOL, and display a live market briefing on the TV in their world.- [Agent Economy: Live](https://three.ws/live): Two AI agents in a virtual world paying each other for live Solana market data. Real on-chain transactions, real blockchain, real 3D.- [Token in 3D](https://three.ws/coin3d): Any pump.fun token as a live 3D scene: a spinning coin medallion with the token's logo, top holders as a galaxy sized by balance, and a graduation ring tracking bonding-curve progress.@@ -322,7 +308,6 @@ On-chain features: pay-per-call, token launches, vanity wallets.- [Markets Hub](https://three.ws/markets): The three.ws markets hub: live global stats, the top 100 coins, breaking crypto news from 192 publisher feeds, a 662k-article news archive, and hero links to the full markets toolset.- [Robinhood Chain](https://three.ws/markets/robinhood): The 24/7 tokenized-equity board for Robinhood Chain (4663): live Chainlink NAV vs DEX premium for every Stock Token, a memecoin screener across NOXA and The Odyssey launchpads, recent launches, and chain TVL, plus a real wallet-connect buy flow for its coins.- [Crypto News](https://three.ws/markets/news): Live crypto news aggregated natively from 192 publisher feeds in 18 languages: filter by category (Bitcoin, DeFi, security, regulation, and more), pick a language, search headlines, and open any story in a rich reader with summary, key points, sentiment, and related coverage.-- [Crypto News Reader](https://three.ws/markets/news/article): Rich article reader for crypto news: full story extraction with AI summary, key points, sentiment, detected tickers, and related coverage from across the industry.- [Crypto News Archive](https://three.ws/markets/archive): The largest open crypto-news archive: 660,000+ articles from September 2017 to today, refreshed hourly from the live feed, in English and Chinese: searchable by keyword, ticker, source, sentiment, date, and language.- [Crypto News Digest](https://three.ws/markets/digest): The day's crypto news grouped into the handful of stories that actually matter: each with a plain-language summary, a market stance, the tickers involved, and every outlet that covered it. Clustered from 38 live publisher feeds.- [$THREE Token](https://three.ws/three-token): The $THREE protocol token: live price, bonding-curve chart, streaming trades, and one-click buy.@@ -332,8 +317,6 @@ On-chain features: pay-per-call, token launches, vanity wallets.- [Live event](https://three.ws/event): The live three.ws community meetup: a ticking countdown off the same config /play reads, the run of show in your own timezone, an add-to-calendar file, who is in the world right now, and the one door in.- [Copy Trading](https://three.ws/mirror): Follow a high-performing agent and your own agent mirrors its real trades, proportionally and strictly within your spend policy.- [Trading Swarms](https://three.ws/swarms): Coordinated groups of trading agents working a shared strategy, with live positions and performance.-- [Agent Wallet](https://three.ws/agent-wallet): The wallet surface for your agents: balances, funding, and activity.-- [Your master wallet](https://three.ws/wallet): One custodial wallet for your account, separate from your agents'. Hold SOL and USDC across Solana and Base, top up any agent you own, send to any Solana address, and read every transaction straight from chain.- [Autonomous Trading](https://three.ws/trading): The control room for three.ws autonomous trading: live fleet vitals, the agent scoreboard, the entry-to-exit pipeline, and every trading surface and doc in one place.- [Exit Lab](https://three.ws/exit-lab): Replay the fleet's real closed trades under a different exit policy. Move the stop-loss, trailing stop, take-initials ladder and moon-bag floor and see what the SOL already spent would have returned.- [The in-game economy](https://three.ws/play/economy): Every price, prize and gate in the /play world economy: what the general store pays for gathered goods, what it sells, how the bank protects cash through death, what the $THREE cosmetics boutique charges, and the full Wheel of Fortune paytable with real per-outcome odds. Read live from the same tables the game server prices trades with.@@ -415,7 +398,7 @@ Docs, tutorials, and reference.- [3D API](https://three.ws/3d): Free, keyless 3D API for AI agents: turn a text prompt into a real GLB model and validate/optimize any glTF/GLB, all from one URL with a live OpenAPI 3.1 spec. Free draft generation and inspection graduate to paid Forge Pro quality tiers and animation-ready Rigged Avatars. No key, no account.- [Crypto API](https://three.ws/crypto-api): One free, keyless API for on-chain and market data: DEX pairs, prices, swap quotes, TVL, Solana RPC reads, and ENS/SNS name resolution: CoinGecko, DefiLlama, Jupiter, DexScreener and Solana RPC bundled behind one bill. Live provider table rendered from the real registry, generated OpenAPI 3.1 spec, free tier then BYOK, plan, or x402 pay-per-call.- [Docs · Publisher rights in the news reader](https://three.ws/docs/news-rights): What three.ws shows from a publisher's article and what it never shows: the enforced lead-excerpt limit, how a withdrawn story is removed (410, deindex, purge), and how publishers can ask us to take something down.-- [3D Drops](https://three.ws/docs/drops): How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.+- [How 3D Drops collections work](https://three.ws/docs/drops): How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.- [Docs · Forge (text & image to 3D)](https://three.ws/docs/forge): How the Forge turns a text prompt, photos, or a sketch into a downloadable textured 3D model (GLB): the free-first engine grid with live health status, input modes, output formats, job resume, and background generation.- [Docs · The Forge pipeline (architecture deep dive)](https://three.ws/docs/forge-pipeline): The engineering guide to the Forge: every entry point, the declared engine grid, free-first health-aware routing, the job lifecycle with two-layer failover, persistence and storage, GPU workers, post-processing, payments, and how to extend the pipeline.- [Docs · Materialize: turn a generation into a real object](https://three.ws/docs/materialize): How to print any three.ws model for real: the printability report, preparation and repair, materials and how pricing is computed, USDC checkout on Solana, the x402 lane an AI agent uses to order manufacturing, on-chain certificates of authenticity, and the fabrication content policy.@@ -615,6 +598,7 @@ Docs, tutorials, and reference.- [Docs: Walk Leaderboard](https://three.ws/docs/walk-leaderboard): Every step your avatar takes on three.ws is measured, rolled up, and ranked. The Walk Leaderboard is the public board for it: walkers ordered by distance walked, sites visited, or time on their feet, over today, this week, or all time.- [Docs: Atlas and Cmd+K search](https://three.ws/docs/atlas): How the site-wide Cmd+K palette works, how the task shortcuts answer a goal with ordered steps instead of a link, how to add one, and how to open the palette from your own page.- [Docs · Repository guards](https://three.ws/docs/guards): The local guards that protect this codebase in place of CI: what each one blocks, which stage runs it, and how to add a new one.+- [Docs · SEO and crawler rendering](https://three.ws/docs/seo): How a JavaScript site is served to search engines: the four server-side render paths, the contract that keeps a page and its sitemap entry agreeing, and how to audit the live origin.- [Tutorial · Add Wallet Sign-In to Your App](https://three.ws/tutorials/wallet-sign-in): Authenticate visitors with Phantom, Backpack, Solflare, or MetaMask: the Sign-In with Solana round trip end to end, the domain check that rejects dev origins, CSRF on verify, both signed-out response shapes, and error codes mapped to fixes.- [Tutorial · Give an Agent a Spending Envelope](https://three.ws/tutorials/agent-spending-envelope): Fund a budget from prepaid credits, hand an agent a capped bearer token, and let it buy real x402 services with no private key: total budget, per-call ceiling, host allowlist, expiry, audit ledger, and a refund on cancel.- [Tutorial · Render Avatars as Images](https://three.ws/tutorials/render-avatar-images): Put a live 3D avatar in an image tag: four camera framings, 28 poses, ARKit facial morphs, social cards, turntables, and a CORS-open GLB proxy. No 3D engine anywhere in your stack.diff --git a/public/sitemap/index.html b/public/sitemap/index.htmlindex 87ff55d51..5a328f145 100644--- a/public/sitemap/index.html+++ b/public/sitemap/index.html@@ -88,7 +88,7 @@<main class="sm-wrap"><div class="sm-hero"><h1>Sitemap</h1>- <p>All 915 pages on three.ws, grouped by purpose. Looking for the machine-readable versions?</p>+ <p>All 916 pages on three.ws, grouped by purpose. Looking for the machine-readable versions?</p><div class="sm-formats"><a href="/sitemap.xml"><code>sitemap.xml</code> · for search engines</a><a href="/llms.txt"><code>llms.txt</code> · for AI agents</a>@@ -100,7 +100,7 @@</div><div class="sm-filter"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><circle cx="11" cy="11" r="7"/><path d="m21 21-4.3-4.3"/></svg>- <input id="sm-filter-input" type="search" placeholder="Filter 915 pages by name, path, or description…" aria-label="Filter pages" autocomplete="off" spellcheck="false" />+ <input id="sm-filter-input" type="search" placeholder="Filter 916 pages by name, path, or description…" aria-label="Filter pages" autocomplete="off" spellcheck="false" /><kbd aria-hidden="true">/</kbd></div><p class="sm-filter-count" id="sm-filter-count" role="status" aria-live="polite"></p>@@ -108,6 +108,14 @@<h2 id="sm-new-title">Newest<a class="sm-new-more" href="/changelog">full changelog →</a></h2><p class="sm-section-desc">The 9 most recently launched pages, straight to the live surface.</p><ul class="sm-list">+ <li>+ <a href="/docs/seo">+ <span class="sm-new-date">Sep 7, 2026</span>+ <span class="sm-title">Docs · SEO and crawler rendering</span>+ <span class="sm-path">/docs/seo</span>+ <span class="sm-desc">How a JavaScript site is served to search engines: the four server-side render paths, the contract that keeps a page and its sitemap entry agreeing, and how to audit the live origin.</span>+ </a>+ </li><li><a href="/irl/sign"><span class="sm-new-date">Sep 5, 2026</span>@@ -172,14 +180,6 @@<span class="sm-desc">Your 3D agent in the car: voice-first, glanceable, and built to the rules CarPlay and Android Auto enforce. Runs on a head unit, a phone in the cradle, or inside the native car shell.</span></a></li>- <li>- <a href="/voice/home">- <span class="sm-new-date">Sep 3, 2026</span>- <span class="sm-title">Hands-free voice for your home</span>- <span class="sm-path">/voice/home</span>- <span class="sm-desc">Talk to your house without finding a phone: a wake word matched on your own device, real barge-in mid-sentence, and a spoken confirmation that a background yes can never satisfy.</span>- </a>- </li></ul></section><nav class="sm-toc" aria-label="Sections">@@ -406,7 +406,7 @@</li><li><a href="/collection">- <span class="sm-title">My Collection</span>+ <span class="sm-title">My Collection<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/collection</span><span class="sm-desc">View all the skills and subscriptions you've unlocked on three.ws.</span></a>@@ -441,7 +441,7 @@</li><li><a href="/notifications">- <span class="sm-title">Notifications</span>+ <span class="sm-title">Notifications<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/notifications</span><span class="sm-desc">Every payment, sale, remix, follow, quest, and royalty notification on three.ws: all in one place, filterable by category.</span></a>@@ -546,7 +546,7 @@</li><li><a href="/feed">- <span class="sm-title">Feed</span>+ <span class="sm-title">Feed<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/feed</span><span class="sm-desc">Live avatars, agents, 3D models, worlds, and launches across three.ws: filter to everyone or just the people you follow.</span></a>@@ -595,7 +595,7 @@</li><li><a href="/demos">- <span class="sm-title">Lab Demos</span>+ <span class="sm-title">Lab Demos<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/demos</span><span class="sm-desc">Index of live demos across the platform: viewers, agents, payments, and experiments.</span></a>@@ -828,7 +828,7 @@</li><li><a href="/studio">- <span class="sm-title">Widget Studio</span>+ <span class="sm-title">Widget Studio<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/studio</span><span class="sm-desc">Build an embeddable 3D widget: pick an avatar (or your own model URL), pick a widget type, tune it live, and copy the snippet. No account needed for the URL-based embeds.</span></a>@@ -856,7 +856,7 @@</li><li><a href="/docs/portal">- <span class="sm-title">Portal: walk any website in 3D</span>+ <span class="sm-title">How Portal builds a 3D world from a URL</span><span class="sm-path">/docs/portal</span><span class="sm-desc">Paste a web address and Portal builds a place out of it: sections become buildings sized by what they say, links become doors you step through, images become billboards. The API, the world spec, the npm package, the CLI and the MCP server behind /portal.</span></a>@@ -989,7 +989,7 @@</li><li><a href="/import/rpm">- <span class="sm-title">Import an avatar URL</span>+ <span class="sm-title">Import an avatar URL<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/import/rpm</span><span class="sm-desc">Import any GLB or glTF avatar into three.ws and give it an agent brain.</span></a>@@ -1038,7 +1038,7 @@</li><li><a href="/nim-forge">- <span class="sm-title">TRELLIS NIM Forge</span>+ <span class="sm-title">TRELLIS NIM Forge<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/nim-forge</span><span class="sm-desc">Drop in a photo, get a textured 3D model, reconstructed on a self-hosted Microsoft TRELLIS NIM. Single-hop image to GLB.</span></a>@@ -1059,7 +1059,7 @@</li><li><a href="/smart-home/join">- <span class="sm-title">Join a home</span>+ <span class="sm-title">Join a home<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/smart-home/join</span><span class="sm-desc">Accept an invitation to somebody's connected home. The link says which home it is, exactly what your role would let you do there and what it would never let you do, before you sign in. It works once and it expires.</span></a>@@ -1208,14 +1208,14 @@</li><li><a href="/viewer">- <span class="sm-title">glTF / GLB Viewer</span>+ <span class="sm-title">glTF / GLB Viewer<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/viewer</span><span class="sm-desc">Preview any public glTF/GLB inline, then take it further: pose and animate it, restyle the material, build a scene, place it in AR, embed it on your site, or launch it as a pump.fun coin.</span></a></li><li><a href="/watch">- <span class="sm-title">Generation Watch</span>+ <span class="sm-title">Generation Watch<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/watch</span><span class="sm-desc">Live progress page for a running text-to-3D generation: real countdown, the painted concept art, and an automatic hand-off into the interactive viewer the moment the model is ready.</span></a>@@ -1236,7 +1236,7 @@</li><li><a href="/ar/view">- <span class="sm-title">View in AR</span>+ <span class="sm-title">View in AR<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/ar/view</span><span class="sm-desc">Place any three.ws GLB in your real space from a plain link: real ARKit Quick Look on iPhone (a genuine on-device USDZ conversion, not a camera overlay), Scene Viewer or WebXR on Android. The shared landing page behind /api/ar and every AR hand-off across the site.</span></a>@@ -1532,7 +1532,7 @@</li><li><a href="/launcher">- <span class="sm-title">Memetic Launcher</span>+ <span class="sm-title">Memetic Launcher<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/launcher</span><span class="sm-desc">Your personal autonomous coin launcher: set your mode (trend, meme, hybrid, random), pick trend sources, tune your cadence, preview what your agents would mint, then go live: real pump.fun launches on your schedule, self-funded from your own agents' wallets under a hard daily SOL cap.</span></a>@@ -1588,7 +1588,7 @@</li><li><a href="/dashboard/capabilities">- <span class="sm-title">Capabilities: Alpha Hunt, Launcher, Auto-Claim & Market Maker</span>+ <span class="sm-title">Capabilities: Alpha Hunt, Launcher, Auto-Claim & Market Maker<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/dashboard/capabilities</span><span class="sm-desc">Live command center for all 4 autonomous agent capabilities: Alpha Hunt smart-money scoring, autonomous Coin Launcher on schedule, Creator Auto-Claim for fee harvesting, and Market Maker with Jito execution.</span></a>@@ -2141,7 +2141,7 @@</li><li><a href="/avatar-wallet-chat">- <span class="sm-title">Avatar Wallet</span>+ <span class="sm-title">Avatar Wallet<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/avatar-wallet-chat</span><span class="sm-desc">An embeddable 3D AI avatar that holds a Solana wallet, chats, and can autonomously send SOL.</span></a>@@ -2204,7 +2204,7 @@</li><li><a href="/markets/news/article">- <span class="sm-title">Crypto News Reader</span>+ <span class="sm-title">Crypto News Reader<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/markets/news/article</span><span class="sm-desc">Rich article reader for crypto news: full story extraction with AI summary, key points, sentiment, detected tickers, and related coverage from across the industry.</span></a>@@ -2274,14 +2274,14 @@</li><li><a href="/agent-wallet">- <span class="sm-title">Agent Wallet</span>+ <span class="sm-title">Agent Wallet<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/agent-wallet</span><span class="sm-desc">The wallet surface for your agents: balances, funding, and activity.</span></a></li><li><a href="/wallet">- <span class="sm-title">Your master wallet</span>+ <span class="sm-title">Your master wallet<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/wallet</span><span class="sm-desc">One custodial wallet for your account, separate from your agents'. Hold SOL and USDC across Solana and Base, top up any agent you own, send to any Solana address, and read every transaction straight from chain.</span></a>@@ -2505,7 +2505,7 @@<section class="sm-section" id="learn"><header>- <h2>Learn<span class="sm-count" data-count>421</span></h2>+ <h2>Learn<span class="sm-count" data-count>422</span></h2><p class="sm-section-desc">Docs, tutorials, and reference.</p></header><ul class="sm-list">@@ -2840,7 +2840,7 @@</li><li><a href="/docs/drops">- <span class="sm-title">3D Drops</span>+ <span class="sm-title">How 3D Drops collections work</span><span class="sm-path">/docs/drops</span><span class="sm-desc">How generative 3D collections work on three.ws: weighted trait layers, a supply rolled from a published seed, rarity scored from the actual distribution, art forged per item on reveal, and a verify endpoint anyone can recompute the roll with.</span></a>@@ -4238,6 +4238,13 @@<span class="sm-desc">The local guards that protect this codebase in place of CI: what each one blocks, which stage runs it, and how to add a new one.</span></a></li>+ <li>+ <a href="/docs/seo">+ <span class="sm-title">Docs · SEO and crawler rendering</span>+ <span class="sm-path">/docs/seo</span>+ <span class="sm-desc">How a JavaScript site is served to search engines: the four server-side render paths, the contract that keeps a page and its sitemap entry agreeing, and how to audit the live origin.</span>+ </a>+ </li><li><a href="/tutorials/wallet-sign-in"><span class="sm-title">Tutorial · Add Wallet Sign-In to Your App</span>@@ -5814,7 +5821,7 @@</li><li><a href="/api/mcp-policy">- <span class="sm-title">MCP trust policy</span>+ <span class="sm-title">MCP trust policy<span class="sm-badge sm-badge-internal">internal</span></span><span class="sm-path">/api/mcp-policy</span><span class="sm-desc">A ready-to-paste allowlist telling an AI client which three.ws MCP tools it may run without asking you. Three profiles from strict to open, in Claude, VS Code, or portable format, derived from the tool safety labels that are verified against each handler on every build.</span></a>diff --git a/scripts/audit-page-index.mjs b/scripts/audit-page-index.mjsindex 4801206c2..813cfe3e0 100644--- a/scripts/audit-page-index.mjs+++ b/scripts/audit-page-index.mjs@@ -16,7 +16,7 @@* files, embeds, assets, and auth-gated utility pages are excluded — they* aren't "features" users discover via the sitemap.*/-import { readFileSync } from 'node:fs';+import { existsSync, readFileSync } from 'node:fs';import { resolve, dirname } from 'node:path';import { fileURLToPath } from 'node:url';@@ -117,6 +117,80 @@ for (const s of pages.sections || []) {}}+// ── Indexability contradictions ──────────────────────────────────────────────+//+// data/pages.json decides what goes into sitemap/core.xml, and the page itself+// decides what a crawler is allowed to do with it. When those two disagree the+// crawler obeys the page and Search Console files the difference as an error+// against us: "Submitted URL marked noindex", or "Indexed, though blocked by+// robots.txt". Both cost crawl budget on a site that submits tens of thousands+// of URLs, and neither is visible in any local check, because both halves are+// individually correct.+//+// On 2026-09-07 seventeen pages were being submitted while answering noindex+// (/wallet, /notifications, /viewer and the rest of the signed-in and+// parameter-driven surfaces), plus /api/mcp-policy, a JSON endpoint submitted+// as a page while robots.txt disallowed /api/. Marking them `indexable: false`+// fixed it; this keeps it fixed.++/** Resolve the static HTML a route serves, mirroring build-page-index.mjs. */+function staticPageFile(pathname) {+ if (!pathname || pathname.startsWith('http')) return null;+ const slug = pathname === '/' ? 'home' : pathname.replace(/^\/+|\/+$/g, '');+ for (const rel of [`pages/${slug}.html`, `pages/${slug}/index.html`, `public/${slug}.html`, `public/${slug}/index.html`]) {+ const file = resolve(root, rel);+ if (existsSync(file)) return file;+ }+ return null;+}++/** Disallow rules that apply to every crawler (the `User-agent: *` group). */+function wildcardDisallows() {+ let file;+ try {+ file = readFileSync(resolve(root, 'public/robots.txt'), 'utf8');+ } catch {+ return [];+ }+ const rules = [];+ let inWildcard = false;+ for (const line of file.split('\n')) {+ const agent = line.match(/^\s*User-agent:\s*(\S+)/i);+ if (agent) {+ inWildcard = agent[1].trim() === '*';+ continue;+ }+ const dis = line.match(/^\s*Disallow:\s*(\S+)/i);+ if (dis && inWildcard) rules.push(dis[1]);+ }+ return rules;+}++const disallows = wildcardDisallows();+const contradictions = [];+for (const s of pages.sections || []) {+ if (s.id === 'news') continue;+ for (const p of s.pages || []) {+ if (!p.path || p.indexable === false || p.auth === 'required' || p.path.startsWith('http')) continue;++ for (const rule of disallows) {+ const prefix = rule.replace(/\*$/, '');+ if (prefix && p.path.startsWith(prefix)) {+ contradictions.push(`${p.path}: submitted for indexing, but robots.txt disallows "${rule}"`);+ break;+ }+ }++ const file = staticPageFile(p.path);+ if (!file) continue; // server-rendered: nothing to read offline+ const html = readFileSync(file, 'utf8');+ const robots = html.match(/<meta[^>]*name=["']robots["'][^>]*>/i)?.[0] || '';+ if (/noindex/i.test(robots)) {+ contradictions.push(`${p.path}: submitted for indexing, but the page answers noindex`);+ }+ }+}+console.log(`Route audit: ${routePaths.size} auditable page routes, ${documented.size} documented in pages.json.`);if (missing.length) {@@ -131,8 +205,20 @@ if (undatedRecent.length && !strict) {console.log(`\nℹ ${undatedRecent.length} manifest page(s) have no \`added\` date (omitted from /changelog). Fine for legacy pages; add dates as you touch them.`);}+if (contradictions.length) {+ console.log(`\n⚠ ${contradictions.length} page(s) contradict their own indexability:`);+ for (const c of contradictions) console.log(` ${c}`);+ console.log('\n→ Set "indexable": false in data/pages.json, or make the page indexable.');+} else {+ console.log('✓ Every page submitted for indexing is crawlable and index-allowed.');+}+if (strict && missing.length) {console.error(`\n✗ strict mode: ${missing.length} undocumented route(s). Failing.`);process.exit(1);}+if (strict && contradictions.length) {+ console.error(`\n✗ strict mode: ${contradictions.length} indexability contradiction(s). Failing.`);+ process.exit(1);+}process.exit(0);diff --git a/scripts/audit-seo.mjs b/scripts/audit-seo.mjsnew file mode 100644index 000000000..95adef8f7--- /dev/null+++ b/scripts/audit-seo.mjs@@ -0,0 +1,237 @@+#!/usr/bin/env node+/**+ * On-page SEO guard for the pages three.ws asks Google to index.+ *+ * Every check here is one a search engine actually acts on, and each one has a+ * failure mode that is invisible locally:+ *+ * - A duplicate <title> or meta description across pages makes Google pick+ * one URL for a cluster and drop the rest. Nothing errors; the pages just+ * stop appearing.+ * - A canonical pointing somewhere other than the page itself hands the+ * page's ranking to another URL. A sitemap entry that then says "index me"+ * contradicts it, and Google resolves the contradiction, not us.+ * - A page in a sitemap that answers noindex or a non-200 spends crawl+ * budget to learn nothing, on a site that submits tens of thousands of URLs.+ * - Structured data that does not parse is dropped whole, taking the rich+ * result with it. That is how the news archive lost its listing on+ * 2026-09-06 (see api/_lib/safe-text.js).+ *+ * It reads the live site rather than the local build on purpose: only the+ * served response proves what a crawler receives after routing, the shell+ * rewrite and the CDN. Point it at a preview origin with --base to check a+ * change before it ships.+ *+ * Usage:+ * node scripts/audit-seo.mjs # live site, core pages+ * node scripts/audit-seo.mjs --limit 40 # quick pass+ * node scripts/audit-seo.mjs --sitemap agents # another URL set+ * node scripts/audit-seo.mjs --base http://localhost:3000+ * node scripts/audit-seo.mjs --strict # exit 1 on any error+ * node scripts/audit-seo.mjs --json report.json # machine-readable+ */+import { writeFileSync } from 'node:fs';+import { parse } from 'node-html-parser';++const args = process.argv.slice(2);+const flag = (name, fallback = null) => {+ const i = args.indexOf(name);+ return i >= 0 && args[i + 1] && !args[i + 1].startsWith('--') ? args[i + 1] : fallback;+};++const base = (flag('--base', 'https://three.ws') || '').replace(/\/+$/, '');+const sitemapName = flag('--sitemap', 'core');+const limit = Number(flag('--limit', '0')) || 0;+const concurrency = Math.max(1, Number(flag('--concurrency', '8')) || 8);+const strict = args.includes('--strict');+const jsonOut = flag('--json');++// Googlebot, because the site routes crawler UAs to server-rendered handlers.+// Auditing with a browser UA would grade a different response than the one+// being indexed.+const UA = 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)';++// SERP rendering is measured in pixels, not characters; these are the character+// bounds where truncation becomes likely for typical latin copy. Outside them a+// title is either cut off or too thin to describe the page.+const TITLE_MIN = 15;+const TITLE_MAX = 65;+const DESC_MIN = 70;+const DESC_MAX = 165;++const errors = [];+const warnings = [];+const err = (url, code, detail) => errors.push({ url, code, detail });+const warn = (url, code, detail) => warnings.push({ url, code, detail });++function hasLoneSurrogate(s) {+ for (let i = 0; i < s.length; i++) {+ const c = s.charCodeAt(i);+ if (c >= 0xd800 && c <= 0xdbff) {+ const n = s.charCodeAt(i + 1);+ if (!(n >= 0xdc00 && n <= 0xdfff)) return true;+ i++;+ } else if (c >= 0xdc00 && c <= 0xdfff) return true;+ }+ return false;+}++async function fetchText(url, timeoutMs = 25_000) {+ const ctl = new AbortController();+ const timer = setTimeout(() => ctl.abort(), timeoutMs);+ try {+ const res = await fetch(url, { headers: { 'user-agent': UA, accept: 'text/html,application/xhtml+xml' }, signal: ctl.signal, redirect: 'manual' });+ const body = res.status >= 300 && res.status < 400 ? '' : await res.text();+ return { status: res.status, location: res.headers.get('location'), body };+ } finally {+ clearTimeout(timer);+ }+}++async function loadSitemapUrls() {+ const { status, body } = await fetchText(`${base}/sitemap/${sitemapName}.xml`);+ if (status !== 200) throw new Error(`sitemap/${sitemapName}.xml answered ${status}`);+ const urls = [...body.matchAll(/<loc>([^<]+)<\/loc>/g)].map((m) => m[1].trim());+ if (!urls.length) throw new Error(`sitemap/${sitemapName}.xml declared no URLs`);+ return limit ? urls.slice(0, limit) : urls;+}++const attr = (root, selector, name) => root.querySelector(selector)?.getAttribute(name)?.trim() || '';++function auditPage(url, status, location, html) {+ if (status >= 300 && status < 400) {+ err(url, 'sitemap-redirect', `${status} → ${location || '?'} (a sitemap should list the destination)`);+ return null;+ }+ if (status !== 200) {+ err(url, 'sitemap-dead', `answered ${status}`);+ return null;+ }++ const root = parse(html);+ const title = root.querySelector('title')?.text?.trim() || '';+ const desc = attr(root, 'meta[name="description"]', 'content');+ const canonical = attr(root, 'link[rel="canonical"]', 'href');+ const robots = attr(root, 'meta[name="robots"]', 'content').toLowerCase();+ const lang = root.querySelector('html')?.getAttribute('lang')?.trim() || '';+ const h1s = root.querySelectorAll('h1').filter((n) => n.text.trim().length > 0);++ if (!title) err(url, 'title-missing', 'no <title>');+ else if (title.length < TITLE_MIN) warn(url, 'title-short', `${title.length} chars: ${title}`);+ else if (title.length > TITLE_MAX) warn(url, 'title-long', `${title.length} chars: ${title.slice(0, 70)}…`);++ if (!desc) err(url, 'description-missing', 'no meta description');+ else if (desc.length < DESC_MIN) warn(url, 'description-short', `${desc.length} chars`);+ else if (desc.length > DESC_MAX) warn(url, 'description-long', `${desc.length} chars`);++ if (!canonical) err(url, 'canonical-missing', 'no rel=canonical');+ else if (!/^https?:\/\//.test(canonical)) err(url, 'canonical-relative', canonical);+ else if (canonical.replace(/\/$/, '') !== url.replace(/\/$/, '')) {+ err(url, 'canonical-mismatch', `points at ${canonical}`);+ }++ // A sitemap is a request to index. A page that answers noindex contradicts+ // its own submission, and Google honours the page.+ if (/\bnoindex\b/.test(robots)) err(url, 'noindex-in-sitemap', `robots: ${robots}`);++ if (!lang) warn(url, 'lang-missing', 'no <html lang>');+ if (h1s.length === 0) warn(url, 'h1-missing', 'no non-empty <h1>');+ else if (h1s.length > 1) warn(url, 'h1-multiple', `${h1s.length} <h1> elements`);++ for (const tag of ['og:title', 'og:description', 'og:image', 'og:url']) {+ if (!attr(root, `meta[property="${tag}"]`, 'content')) warn(url, 'og-missing', tag);+ }+ if (!attr(root, 'meta[name="twitter:card"]', 'content')) warn(url, 'twitter-card-missing', 'no twitter:card');+ if (!attr(root, 'meta[name="viewport"]', 'content')) err(url, 'viewport-missing', 'no viewport meta');++ let ldTypes = [];+ for (const node of root.querySelectorAll('script[type="application/ld+json"]')) {+ const raw = node.text || '';+ if (hasLoneSurrogate(raw) || /\\u[dD][89abAB][0-9a-fA-F]{2}/.test(raw)) {+ err(url, 'ld-truncated-character', 'JSON-LD carries an unpaired surrogate');+ }+ try {+ const parsed = JSON.parse(raw);+ const nodes = parsed['@graph'] || [parsed];+ for (const n of nodes) {+ if (n && n['@type']) ldTypes.push(Array.isArray(n['@type']) ? n['@type'][0] : n['@type']);+ }+ if (!parsed['@context']) warn(url, 'ld-no-context', 'JSON-LD block without @context');+ } catch (e) {+ err(url, 'ld-unparsable', e.message);+ }+ }+ if (!ldTypes.length) warn(url, 'ld-missing', 'no structured data');++ return { url, title, desc, canonical, ldTypes };+}++async function mapPool(items, worker) {+ const out = new Array(items.length);+ let cursor = 0;+ await Promise.all(+ Array.from({ length: Math.min(concurrency, items.length) }, async () => {+ while (cursor < items.length) {+ const i = cursor++;+ out[i] = await worker(items[i]);+ }+ }),+ );+ return out;+}++const urls = await loadSitemapUrls();+console.log(`[audit-seo] ${urls.length} URL(s) from ${base}/sitemap/${sitemapName}.xml, concurrency ${concurrency}`);++const pages = (+ await mapPool(urls, async (url) => {+ try {+ const { status, location, body } = await fetchText(url);+ return auditPage(url, status, location, body);+ } catch (e) {+ err(url, 'fetch-failed', e.message);+ return null;+ }+ })+).filter(Boolean);++// Duplicate titles and descriptions are a corpus-level property, so they are+// only knowable once every page has been read.+const byTitle = new Map();+const byDesc = new Map();+for (const p of pages) {+ if (p.title) byTitle.set(p.title, [...(byTitle.get(p.title) || []), p.url]);+ if (p.desc) byDesc.set(p.desc, [...(byDesc.get(p.desc) || []), p.url]);+}+for (const [title, list] of byTitle) {+ if (list.length > 1) err(list[0], 'title-duplicate', `${list.length} pages share "${title}": ${list.slice(0, 4).join(', ')}${list.length > 4 ? ' …' : ''}`);+}+for (const [, list] of byDesc) {+ if (list.length > 1) err(list[0], 'description-duplicate', `${list.length} pages share one description: ${list.slice(0, 4).join(', ')}${list.length > 4 ? ' …' : ''}`);+}++const group = (rows) => {+ const m = new Map();+ for (const r of rows) m.set(r.code, [...(m.get(r.code) || []), r]);+ return [...m.entries()].sort((a, b) => b[1].length - a[1].length);+};++console.log(`\n[audit-seo] read ${pages.length}/${urls.length} page(s)`);+for (const [label, rows] of [['ERROR', errors], ['WARN', warnings]]) {+ if (!rows.length) continue;+ console.log(`\n${label} (${rows.length})`);+ for (const [code, list] of group(rows)) {+ console.log(` ${code} × ${list.length}`);+ for (const r of list.slice(0, 5)) console.log(` ${r.url}\n ${r.detail}`);+ if (list.length > 5) console.log(` … ${list.length - 5} more`);+ }+}++if (jsonOut) {+ writeFileSync(jsonOut, JSON.stringify({ base, sitemap: sitemapName, checked: pages.length, errors, warnings }, null, 2) + '\n');+ console.log(`\n[audit-seo] wrote ${jsonOut}`);+}++if (!errors.length && !warnings.length) console.log('\n✓ audit-seo: no on-page SEO defects found');+else console.log(`\n[audit-seo] ${errors.length} error(s), ${warnings.length} warning(s)`);+process.exit(strict && errors.length ? 1 : 0);diff --git a/server/crawler-body.mjs b/server/crawler-body.mjsnew file mode 100644index 000000000..2dc063a4e--- /dev/null+++ b/server/crawler-body.mjs@@ -0,0 +1,186 @@+// Crawler-visible bodies for the two shared shells whose content is fetched+// client-side: /tutorials/<slug> and /walkthroughs/<slug>.+//+// Both routes resolve to one shell (pages/tutorial.html, pages/walkthrough.html)+// whose body is empty markup the player fills after fetching its content. That+// is fine for a browser and bad for search: 76 pages built to answer questions+// people type into Google ("text to 3d", "embed a 3d avatar") shipped with no+// text in them. The tutorial shell was worse than empty, because its static+// fallback reads "Tutorial not found" and its <h1> is blank, so a crawler that+// did not run the fetch banked a not-found page under a real URL.+//+// server/seo-head.mjs already rewrites the <head> of these routes per path.+// This is the same idea for the body: render the content that the player would+// have fetched, straight into the shell. The players replace their root+// wholesale on load (article.innerHTML / root.innerHTML), so a visitor with JS+// sees no duplication and no flash of two versions, exactly the contract+// api/news/story-page.js uses for a story permalink.+//+// Content comes from the same files the player fetches (docs/tutorials/*.md,+// data/walkthroughs.json), so the rendered page cannot drift from the read one.++import { readFileSync } from 'node:fs';+import path from 'node:path';+import { fileURLToPath } from 'node:url';+import { marked } from 'marked';++const __dirname = path.dirname(fileURLToPath(import.meta.url));+const root = path.resolve(__dirname, '..');++// Rendered bodies are stable between deploys, so one parse per path per process+// is enough. null is cached too: an unknown slug must not re-read the disk on+// every crawl of a URL that will never resolve.+const cache = new Map();++const SLUG = /^[a-z0-9-]+$/;++function esc(s) {+ return String(s ?? '')+ .replace(/&/g, '&')+ .replace(/</g, '<')+ .replace(/>/g, '>')+ .replace(/"/g, '"');+}++// Anchored at the start of the document, deliberately: an unanchored /^#/m+// would match the first `# comment` line inside an early fenced code block and+// delete it from the rendered page.+const LEADING_H1 = /^\s*#\s+(.+?)[ \t]*(?:\r?\n|$)/;++/** Strip the leading `# Title` line, which the shell renders as its own <h1>. */+function splitTitle(md) {+ const m = md.match(LEADING_H1);+ if (!m) return { title: '', body: md };+ return { title: m[1].trim(), body: md.slice(m[0].length) };+}++/** First prose paragraph of a markdown body, for the hero blurb. */+function firstParagraph(md) {+ for (const block of md.split(/\n{2,}/)) {+ const t = block.trim();+ if (!t || t.startsWith('#') || t.startsWith('```') || t.startsWith('|') || t.startsWith('<')) continue;+ return t.replace(/\s+/g, ' ');+ }+ return '';+}++function renderMarkdown(md) {+ // Our own repo files: trusted input, so marked's default (no sanitizer) is+ // the right call, and the player renders the same source with the same lib.+ return marked.parse(md, { gfm: true, breaks: false, mangle: false, headerIds: true });+}++function tutorialBody(slug) {+ if (!SLUG.test(slug)) return null;+ let md;+ try {+ md = readFileSync(path.join(root, 'docs', 'tutorials', `${slug}.md`), 'utf8');+ } catch {+ return null; // no markdown: leave the shell alone and let the player 404 it+ }+ const { title, body } = splitTitle(md);+ if (!title) return null;+ return { title, blurb: firstParagraph(body), html: renderMarkdown(body) };+}++let walkthroughs = null;+function loadWalkthroughs() {+ if (walkthroughs) return walkthroughs;+ try {+ const raw = JSON.parse(readFileSync(path.join(root, 'data', 'walkthroughs.json'), 'utf8'));+ walkthroughs = new Map((raw.walkthroughs || []).map((w) => [w.slug, w]));+ } catch {+ walkthroughs = new Map();+ }+ return walkthroughs;+}++function walkthroughBody(slug) {+ if (!SLUG.test(slug)) return null;+ const w = loadWalkthroughs().get(slug);+ if (!w) return null;++ const steps = (w.steps || [])+ .map(+ (s, i) =>+ `<li><h2>${esc(s.title || `Step ${i + 1}`)}</h2>` ++ `<p>${esc(s.body || '')}</p>` ++ (s.tip ? `<p><em>${esc(s.tip)}</em></p>` : '') ++ (s.path ? `<p><a href="${esc(s.path)}">${esc(s.path)}</a></p>` : '') ++ `</li>`,+ )+ .join('\n');++ const related = (w.related || [])+ .map((r) => `<li><a href="${esc(r.href)}">${esc(r.label)}</a></li>`)+ .join('');++ return {+ title: w.title || slug,+ blurb: w.blurb || w.outcome || '',+ html:+ `<p>${esc(w.blurb || '')}</p>` ++ (w.outcome ? `<p><strong>What you end up with:</strong> ${esc(w.outcome)}</p>` : '') ++ (w.minutes ? `<p>${esc(String(w.minutes))} minute walkthrough${w.level ? `, ${esc(w.level)} level` : ''}.</p>` : '') ++ (steps ? `<ol>${steps}</ol>` : '') ++ (w.cta?.href ? `<p><a href="${esc(w.cta.href)}">${esc(w.cta.label || 'Start')}</a></p>` : '') ++ (related ? `<h2>Related</h2><ul>${related}</ul>` : ''),+ };+}++function injectTutorial(html, content) {+ return html+ .replace(/<div class="tdoc-hero" id="tdoc-hero" hidden>/, '<div class="tdoc-hero" id="tdoc-hero">')+ .replace(/<h1 id="tdoc-title">\s*<\/h1>/, `<h1 id="tdoc-title">${esc(content.title)}</h1>`)+ .replace(+ /<p class="tdoc-hero-blurb" id="tdoc-blurb">\s*<\/p>/,+ `<p class="tdoc-hero-blurb" id="tdoc-blurb">${esc(content.blurb)}</p>`,+ )+ .replace(+ /(<article class="tdoc-article" id="tdoc-article">)[\s\S]*?(<\/article>)/,+ (_m, open, close) => `${open}${content.html}${close}`,+ );+}++function injectWalkthrough(html, content) {+ return html.replace(+ /(<div id="wt-root" data-state="loading">)[\s\S]*?(<\/div>)/,+ (_m, open, close) => `${open}<h1>${esc(content.title)}</h1>${content.html}${close}`,+ );+}++/**+ * Render the crawler-visible body for a shared-shell content route.+ *+ * @param {string} pathname request path, before the dest rewrite+ * @param {string} html the shell HTML (head already rewritten by seo-head)+ * @returns {string|null} the shell with a real body, or null when the path is+ * not one of these routes, the slug has no content, or nothing was injected+ */+export function renderCrawlerBody(pathname, html) {+ if (!pathname || !html) return null;+ const clean = pathname.length > 1 ? pathname.replace(/\/+$/, '') : pathname;+ const m = clean.match(/^\/(tutorials|walkthroughs)\/([^/]+)$/);+ if (!m) return null;++ const key = `${m[1]}/${m[2]}`;+ if (!cache.has(key)) {+ try {+ cache.set(key, m[1] === 'tutorials' ? tutorialBody(m[2]) : walkthroughBody(m[2]));+ } catch (err) {+ console.error(`[crawler-body] ${clean} fell back to the shell:`, err.message);+ cache.set(key, null);+ }+ }+ const content = cache.get(key);+ if (!content) return null;++ const out = m[1] === 'tutorials' ? injectTutorial(html, content) : injectWalkthrough(html, content);+ // A shell that changed shape would silently start serving the empty body+ // again; saying so beats shipping a blank page that still returns 200.+ if (out === html) {+ console.error(`[crawler-body] ${clean}: shell markup did not match, body not injected`);+ return null;+ }+ return out;+}diff --git a/server/index.mjs b/server/index.mjsindex 48479dbb5..5a3544eb2 100644--- a/server/index.mjs+++ b/server/index.mjs@@ -37,6 +37,7 @@ import path from 'node:path';import { pathToFileURL, fileURLToPath } from 'node:url';import { isSsrRoute, renderSsrPage } from './ssr-pages.mjs';import { hasSeoRoute, renderSeoHead } from './seo-head.mjs';+import { renderCrawlerBody } from './crawler-body.mjs';import { isMissingShellPage } from './shell-pages.mjs';import { hardenHeaderBag } from './csp-hashes.mjs';import { cronEdgeAuth } from './cron-edge-auth.mjs';@@ -514,7 +515,11 @@ app.use(async (req, res) => {// canonical, title, social card and JSON-LD. Uses the ORIGINAL request// path: currentPath is already the dest rewrite (/docs/index.html).if (req.method === 'GET' && file.endsWith('.html') && hasSeoRoute(url.pathname)) {- const html = renderSeoHead(url.pathname, file);+ const head = renderSeoHead(url.pathname, file);+ // /tutorials/* and /walkthroughs/* fetch their content client-side,+ // so the shell alone carries no text for a crawler to read. Render+ // the body too; the player replaces it on load.+ const html = renderCrawlerBody(url.pathname, head) || head;if (html) {// The rewritten head swaps in this route's own JSON-LD block, so// the policy has to describe the rewritten bytes.diff --git a/tests/crawler-body.test.js b/tests/crawler-body.test.jsnew file mode 100644index 000000000..e77370067--- /dev/null+++ b/tests/crawler-body.test.js@@ -0,0 +1,102 @@+// Coverage for server/crawler-body.mjs.+//+// /tutorials/<slug> and /walkthroughs/<slug> resolve to one shared shell whose+// body is filled by a client-side fetch. Before this module those 76 URLs were+// submitted to Google carrying no text: the walkthrough shell was an empty+// <div>, and the tutorial shell was worse than empty, because its static+// fallback copy reads "Tutorial not found" under a blank <h1>.+//+// The invariants that matter, and why each is asserted rather than eyeballed:+// 1. The rendered page carries the tutorial's real heading and prose, so+// there is something to index at all.+// 2. Exactly one <h1>. The shell's hero owns it, so the markdown's own title+// line must be stripped; two competing top-level headings is the defect+// this module would otherwise introduce.+// 3. An unknown slug changes nothing. Inventing a page for a URL with no+// content is how a 404 gets banked as a real page.++import { describe, it, expect } from 'vitest';+import { readFileSync } from 'node:fs';+import { resolve } from 'node:path';++const { renderCrawlerBody } = await import('../server/crawler-body.mjs');++const root = resolve(import.meta.dirname, '..');+const tutorialShell = readFileSync(resolve(root, 'pages/tutorial.html'), 'utf8');+const walkthroughShell = readFileSync(resolve(root, 'pages/walkthrough.html'), 'utf8');++const headings = (html, tag) =>+ [...html.matchAll(new RegExp(`<${tag}[^>]*>([\\s\\S]*?)</${tag}>`, 'g'))]+ .map((m) => m[1].replace(/<[^>]*>/g, '').trim())+ .filter(Boolean);++describe('tutorial pages', () => {+ const out = renderCrawlerBody('/tutorials/text-to-3d', tutorialShell);++ it('renders the tutorial body into the shell', () => {+ expect(out).toBeTruthy();+ expect(out.length).toBeGreaterThan(tutorialShell.length);+ expect(out).not.toContain('<h1 id="tdoc-title"></h1>');+ });++ it('emits exactly one h1, taken from the markdown title', () => {+ const h1s = headings(out, 'h1');+ expect(h1s).toHaveLength(1);+ expect(h1s[0]).toBe('Turn a Text Prompt into a 3D Model');+ });++ it('reveals the hero that the shell hides for the client player', () => {+ expect(tutorialShell).toContain('id="tdoc-hero" hidden');+ expect(out).not.toContain('id="tdoc-hero" hidden');+ });++ it('drops the not-found fallback copy from the article', () => {+ const article = out.match(/<article class="tdoc-article" id="tdoc-article">([\s\S]*?)<\/article>/)[1];+ expect(article).not.toMatch(/Loading tutorial/);+ expect(article.replace(/<[^>]*>/g, '').trim().length).toBeGreaterThan(400);+ });++ it('leaves a slug with no markdown untouched', () => {+ expect(renderCrawlerBody('/tutorials/no-such-tutorial-here', tutorialShell)).toBeNull();+ });++ it('refuses a slug that is not a slug', () => {+ expect(renderCrawlerBody('/tutorials/..%2F..%2Fetc%2Fpasswd', tutorialShell)).toBeNull();+ expect(renderCrawlerBody('/tutorials/a/b', tutorialShell)).toBeNull();+ });+});++describe('walkthrough pages', () => {+ const out = renderCrawlerBody('/walkthroughs/embed-a-3d-avatar', walkthroughShell);++ it('renders the walkthrough steps into the empty root', () => {+ expect(out).toBeTruthy();+ expect(walkthroughShell).toMatch(/<div id="wt-root" data-state="loading"><\/div>/);+ expect(out.length).toBeGreaterThan(walkthroughShell.length);+ });++ it('emits one h1 and the step headings below it', () => {+ expect(headings(out, 'h1')).toHaveLength(1);+ expect(headings(out, 'h2').length).toBeGreaterThan(0);+ });++ it('leaves an unknown walkthrough untouched', () => {+ expect(renderCrawlerBody('/walkthroughs/not-a-walkthrough', walkthroughShell)).toBeNull();+ });+});++describe('routing', () => {+ it('ignores paths it does not own', () => {+ for (const p of ['/markets', '/tutorials', '/walkthroughs', '/', '/docs/portal']) {+ expect(renderCrawlerBody(p, tutorialShell)).toBeNull();+ }+ });++ it('tolerates a trailing slash', () => {+ expect(renderCrawlerBody('/tutorials/text-to-3d/', tutorialShell)).toBeTruthy();+ });++ it('returns null rather than a half-injected page when the shell changes shape', () => {+ expect(renderCrawlerBody('/tutorials/text-to-3d', '<html><body>a different shell</body></html>')).toBeNull();+ });+});