Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
AR in ChatGPT
How a sentence typed into ChatGPT becomes a 3D model standing in your room. This document covers the whole pipeline: the free generation lane ChatGPT calls, the place-in-your-room link (arUrl) every generation carries, the device-aware launcher behind that link, the living-avatar handoff for rigged bodies, and how a shared link unfurls with a real render of the model.
Nothing here is ChatGPT-exclusive. ChatGPT is the first surface that ships the full flow, but every contract below is public and keyless, so any agent, MCP client, or plain HTTP caller gets the identical capability.
If you just want to try it, start with the tutorial: Your first prompt to 3D in ChatGPT.
The one-link design
ChatGPT cannot run WebGL, ARKit, or ARCore. It can only print links. So the entire AR capability is compressed into a single URL that any chat surface can hand to the user:
https://three.ws/api/ar?src=<glbUrl>&title=<name>
https://three.ws/api/ar?src=<glbUrl>&title=<name>&kind=avatar (rigged avatar)
src(required): a publichttps.glb/.gltfURL, URL-encoded.title(optional): a human label. The launcher accepts up to 120 characters (api/ar.js); the generation pipeline puts the user's prompt here, trimmed to 80, so the AR page arrives labeled (api/_mcp-studio/gpt-forge-client.js).kind=avatar(optional): marks the model as a rigged agent body and unlocks the living-agent lane (below).
Every free generation endpoint returns this link as the arUrl field, built by one shared constructor (buildArLaunchUrl in api/_lib/ar-launch.js, mirrored by arLaunchUrl in api/_mcp-studio/gpt-forge-client.js, the ChatGPT surfaces' client over /api/gpt-forge, the ChatGPT-dedicated clone of the forge pipeline). ChatGPT's only job is to print it. Everything hard (device detection, format conversion, the AR session itself) happens on the three.ws side after the tap, which is why one URL works identically in ChatGPT, Claude, an email, or a text message.
The two ChatGPT surfaces
The same free lane ships into ChatGPT twice, because ChatGPT has two integration models with different strengths.
1. The ChatGPT app (Apps SDK / MCP connector)
Three front doors serve the same free tools over one handler, all with no authentication. https://three.ws/api/mcp-chatgpt is the one ChatGPT lists: the eight 3D tools and the model viewer, with no catalog or persona tools and no frameDomains, because OpenAI's app guidelines reserve frame domains for embedding an essential third-party experience. https://three.ws/api/mcp-studio is the full surface for Claude and every other MCP host, and https://three.ws/api/mcp-grok serves the same full set to Grok and the xAI Responses API. SURFACES in api/_mcp-studio/dispatch.js is the single definition of what each door advertises. The full surface carries sixteen free tools: six generation tools (forge_free, text_to_avatar, mesh_forge, rig_mesh, forge_avatar, refine_model), the check_job collector that picks up a generation which outran the tool call (a pending result names its stage, mesh or rig, so the collector knows which leg it is waiting on), get_job (full and Grok surfaces only, not the ChatGPT listing), a status reader for the same jobs that reports progress, eta_seconds and elapsed_seconds while one runs and never starts or repeats a generation, the read-only look_at_model tool that renders any public GLB from several angles and returns the frames as images plus a plain reading of its geometry, so the assistant can judge its own output before printing a link, four read-only catalog and docs tools (search_catalog, get_catalog_item, get_item_source, search_docs, in api/_mcp-studio/catalog-tools.js), and three persona/embodiment tools (create_agent_persona, get_agent_persona, persona_say, in api/_mcp-studio/persona-tools.js). The first eight are the ChatGPT listing. Every generation tool also accepts an optional idempotency_key, so a retried call returns the first call's job instead of generating twice, and every generation, rig and refinement result carries a signed run receipt (see Run receipts). Full server documentation: 3D Studio MCP.
ChatGPT drops a tool call still open at 60 seconds, while a generation takes one to four minutes. So on /api/mcp-chatgpt every call answers inside a 40 second budget (CHATGPT_CALL_BUDGET_MS) with either the finished model or a pending job, and the inline viewer widget collects that job itself (running the rig leg when an avatar's mesh lands first) and shows the model when it is ready. The assistant does not have to loop on check_job. Generation quotas on this door are keyed per ChatGPT user rather than on OpenAI's shared egress IP.
Every successful tool result carries arUrl in its structuredContent (next to the four plain links every model result now leads with for clients that render no widget: viewer_url, glb_url, poster_png_url, and embed_html), and the inline 3D widget renders it as a View in your space button next to the rotatable model. When the result is a rigged avatar the response additionally carries irlUrl and the button becomes Bring it to life. Refined versions (refine_model) each carry their own arUrl, so any point in the version lineage can be placed in the room.
2. The "three.ws 3D Studio" custom GPT (Actions)
Not every ChatGPT plan supports connectors, so the same lane also ships as a plain REST Actions surface for the custom GPT in the GPT Store. The GPT's builder settings and instruction text live in chatgpt-3d-studio-gpt.md; the REST contract it calls:
curl -s -X POST https://three.ws/api/3d/studio \
-H 'content-type: application/json' \
-d '{"prompt":"a small ceramic robot figurine"}'
Submit returns either the finished model or a job to poll (the response shaping lives in api/_mcp-studio/studio-shape.js, shared with the tests):
{ "status": "pending", "job": "…", "poll": "/api/3d/studio?job=…&title=a%20small%20ceramic%20robot%20figurine", "watchUrl": "https://three.ws/watch?job=…&title=…", "etaSeconds": 45, "format": "glb" }
{
"status": "done",
"glbUrl": "https://three.ws/cdn/creations/…/model.glb",
"viewerUrl": "https://three.ws/viewer?src=…",
"arUrl": "https://three.ws/api/ar?src=…&title=a%20small%20ceramic%20robot%20figurine",
"format": "glb"
}
Design decisions on this endpoint, all deliberate:
- Store-clean responses. The payload carries model URLs and job state only: no pricing, no upgrade hints, no internal identifiers, no wallet or token surface. That is what makes it publishable as a GPT Store action. (The agent-facing twin,
POST /api/3d/generate, returns the same core fields plusfree/upgrade; see the free 3D API.) - The full forge experience. The request accepts an optional
tier(draft|standard|high; High runs the deepest sampler and texture budget on the self-host GPU lanes, operator-funded, and simply takes minutes through the poll loop). Routing is the same health-aware free-first router the /forge page uses, with no pinned backend. And because the forge paints a concept image before sculpting the mesh, pending and done states carrypreviewImageUrl, so ChatGPT shows the painted view the moment it exists while the 3D model is still generating. - The title-carrying poll. ChatGPT Actions never resend context, so the submit response embeds the prompt in the poll URL as
&title=. When the poll finally returnsdone, thearUrlandviewerUrlare labeled with the original prompt without the GPT doing anything except calling the poll path verbatim. - Timeout honesty. Actions calls time out around 45 seconds; generation takes 20 to 60. The endpoint holds the request as long as it safely can and falls back to
pending+ poll rather than dying mid-request. - Safety before GPU. Every prompt passes an age-13+ appropriateness gate (api/_mcp-studio/safety.js) before any provider work; refusals return
400 prompt_rejectedwith a clear message. - An outage is an answer, not a 5xx. When the poll finds the generation lane unconfigured upstream (
503/501withunconfiguredorbackend_unconfigured), it answers200 { "status": "error", "job": "…", "error": "3D generation is temporarily unavailable…" }, so the GPT can tell the user plainly instead of surfacing a transport failure.
The GPT's instructions tell it to present three links on every finished model, in this order: See it in your space (arUrl), Preview in your browser (viewerUrl), Download (glbUrl), and to never fabricate a URL an action did not return. The OpenAPI schema the GPT imports is served at https://three.ws/.well-known/3d-studio-openapi.yaml and describes /api/ar alongside the generation action, so the AR launch is part of the app's published contract rather than an undocumented link. The full GPT build sheet lives at prompts/store-submissions/_generated/ (openai-gpt-config.md, plus openai-actions.yaml, which is regenerated from the served schema by npm run sync:studio-openapi).
What happens when the link is tapped
GET /api/ar (api/ar.js) reads the request's User-Agent server-side (no client round-trip, no JS sniffing) and branches:
| Device | What the user gets |
|---|---|
| iPhone / iPad | A tiny interstitial carrying the model's real og:image/og:title (so a pasted link still unfurls; crawlers do not follow a bare redirect's JS) that refreshes straight into /ar/view?src=…&title=… (pages/ar-view.html, src/ar-view.js). That Vite-bundled page generates a real USDZ from the GLB on the device with three's USDZExporter and then offers Apple Quick Look with a genuine ios-src. No server-side USD tooling exists or is needed; <model-viewer> alone does not do the conversion, which is why the AR page is a bundled page rather than HTML the handler writes inline. |
| Android (static model) | An HTTP 302 straight into Google Scene Viewer via an ARCore intent:// URL, with a browser_fallback_url back to the WebGL viewer for devices without ARCore. |
| Desktop | The same interstitial into /ar/view, which falls back to the interactive WebGL viewer. There is no dead end: the link is safe to open anywhere. |
Responses set vary: user-agent so CDN caching stays correct per device class. Bad input (non-https, non-GLB, missing src) returns a designed error page, never a crash.
The pure routing logic (detectArTarget, buildSceneViewerUrl, buildArViewUrl, planArLaunch) lives dependency-free in api/_lib/ar-launch.js and is covered by tests/ar-export.test.js. planArLaunch now resolves every branch to a redirect target: Scene Viewer for a static model on Android, /ar/view for everything else, with viewerUrl and sceneViewerUrl still returned for callers such as the export_ar tool that want the raw URLs alongside the launch link. The deeper AR reference (WebXR sessions, USDZ pipeline details, optimization limits) is docs/ar.md.
Living avatars: kind=avatar
AR on three.ws is not a prop viewer; it is how agents cross into physical space. When the generated model is a rigged avatar (an agent's body), the pipeline appends kind=avatar to the arUrl, and the launcher changes behavior:
/ar/viewreceives the handoff as&irl=<irlUrl>and leads with a Bring it to life button into/irl?avatar=<glbUrl>, where the avatar walks, animates, and talks with the user through the camera, standing in their real room. See IRL.- Android is NOT blind-redirected into Scene Viewer, because that would hide the living path. It lands on
/ar/viewtoo, with static placement still one tap away.
In ChatGPT this happens automatically: the avatar-producing tools (text_to_avatar, rig_mesh, forge_avatar) set the flag and return irlUrl alongside arUrl, and the inline widget swaps its button label accordingly.
Sharing: links that unfurl with the model
A place-in-your-room link is built to be forwarded. When it is pasted into a chat app or social surface, the launch page's Open Graph tags make it unfurl with a real render of that exact model, not a bare URL:
og:title: "Place<title>in your room"og:image:https://three.ws/api/render/glb?glbUrl=<encoded GLB>&width=1200&height=630
GET /api/render/glb (api/render/glb.js) is a public, URL-addressable GLB-to-PNG renderer: point any image tag at it with a public GLB URL and get a PNG back (the shared renderGlbToPng in api/_lib/render-glb.js: an in-process software rasterizer first, headless Chromium as the failover for Draco/KTX2 models, the same pipeline as the platform's OG cards). It clamps dimensions to 64-2048, HEAD-checks the GLB at 10 MB max, accepts a background of transparent or any CSS color (markup or anything else that is not a color is rejected with 400 bad_request), is SSRF-guarded and per-IP rate-limited, and is CDN-cached for a day. The launch page itself ends with a "Create your own" link, so a shared model converts its recipient into a creator.
Using the same contract outside ChatGPT
Everything above is public plumbing:
- Plain HTTP, free:
POST /api/3d/generateis the keyless agent-facing twin of the studio endpoint, same lane, samearUrl. Contract, states, and per-IP limits: the free 3D API. - Any MCP client: the full connector at
/api/mcp-studioworks in Claude and every other MCP host, not just ChatGPT. Every generation it returns already carriesarUrl(andirlUrlfor rigged avatars) plus a Spatial MCP artifact, so no extra tool call is needed to get the AR link. To build that link set for a GLB the connector did not generate, use the read-onlyexport_artool on the separate/api/mcp-3dserver, which returnsarLaunchUrl,sceneViewerUrl,viewerUrl, andirlUrl. That server is account- or payment-gated and is not part of the free connector. - By hand: the
arUrlformat is stable. If you have a public GLB, you can constructhttps://three.ws/api/ar?src=<encoded GLB>&title=<name>yourself and it will route correctly on every device.
The generation lane is operator-funded, so the end user pays nothing and needs no account. No backend is pinned: the health-aware router in api/gpt-forge.js walks the free lanes first (the self-hosted TRELLIS and Hunyuan3D GPU workers, then the Hugging Face Spaces image→3D lane and the free NVIDIA NIM text→3D lane; freeLaneCandidates in api/_lib/forge-tiers.js is the ordering) and keeps the platform-keyed Replicate lane as a failover rung, so a cold or throttled free worker degrades to the next healthy engine instead of failing the request. Either way the cost lands on the platform. Real per-IP rate limits apply and are documented per endpoint (see Funding & limits).
Where it lives
| Piece | File |
|---|---|
| Custom GPT Actions endpoint | api/3d/studio.js |
| Free agent endpoint | api/3d/generate.js |
AR launcher (/api/ar) + unfurl tags |
api/ar.js |
AR routing core (buildArLaunchUrl, device branch, IRL handoff) |
api/_lib/ar-launch.js |
| MCP studio tools + response envelope | api/_mcp-studio/tools.js |
Studio Actions response shaping (shapeSubmit, shapePoll) |
api/_mcp-studio/studio-shape.js |
The on-device AR page (/ar/view) |
pages/ar-view.html, src/ar-view.js |
| Inline widget (View in your space / Bring it to life) | api/_mcp-studio/component.js |
| GLB-to-PNG renderer (og:image) | api/render/glb.js |
| GPT OpenAPI schema (served, source of truth) | public/.well-known/3d-studio-openapi.yaml |
| GPT build sheet + submission kit | prompts/store-submissions/_generated/ |
| Tests | tests/api/3d-studio.test.js, tests/ar-export.test.js |
See also
- Your first prompt to 3D in ChatGPT: the user-facing walkthrough of this whole flow
- Place your 3D model in AR: the on-site AR tutorial
- 3D Studio MCP: the free MCP server behind the ChatGPT app
- The free 3D API: the keyless HTTP contract with
arUrl - AR and WebXR reference: Quick Look, Scene Viewer, WebXR, and the USDZ pipeline in depth
- IRL: the living-agent experience behind Bring it to life