Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

feat(vscode-3d): make the workbench reachable: Animate, Bake clip, Refine, Optimize, Compare, Check quality, turntable GIFs, and <agent-3d> linting

nirholas50b9e3ecf6ac
diff --git a/docs/vscode.md b/docs/vscode.md
index 439425f08..38cc3c82f 100644
--- a/docs/vscode.md
+++ b/docs/vscode.md
@@ -1,30 +1,35 @@
# three.ws in VS Code
Two extensions put the platform inside the editor. **three.ws 3D** makes model
-files and 3D generation native to VS Code. **three.ws x402** does the same for
-paid endpoints. This page covers the first; see [x402](./x402.md#open-source-packages) for
-the second.
+files, 3D generation, animation, and embedding native to VS Code. **three.ws
+x402** does the same for paid endpoints. This page covers the first; see
+[x402](./x402.md#open-source-packages) for the second.
| | |
|---|---|
-| **Extension** | three.ws 3D (`threews.vscode-3d`) |
+| **Extension** | three.ws 3D (`threews.vscode-3d`), version 0.2.0 |
| **Source** | [`packages/vscode-3d`](../packages/vscode-3d) |
| **Requires** | VS Code 1.85 or newer |
-| **Auth** | None. Generation runs on the free lane. |
+| **Auth** | None. Generation, refinement, the motion library, and the quality check all run on free lanes. |
## What it does
VS Code has no idea what a `.glb` is. Every 3D asset in a repository is an opaque
binary you have to leave the editor to look at: upload it somewhere, open a
-browser viewer, come back. This extension closes that loop.
+browser viewer, come back. And once you can see it, the next questions (does it
+animate, is it too heavy, what changed since the last commit, is the embed on
+my page correct) each need another tool. This extension closes the whole loop.
+
+### See it
**Model files render in an editor tab.** Double-click any `.glb` or `.gltf` and
it opens in a three.js viewer instead of the "binary file not shown" screen.
Orbit with the left mouse button, pan with the right, zoom with the wheel.
Animation clips get a picker, a play/pause button, and a scrub bar. There are
-toggles for a wireframe, the skeleton, the ground grid, and a slow auto-orbit,
-plus a **Snapshot** button that writes the current camera view as a PNG next to
-the model.
+toggles for a wireframe, the skeleton, the ground grid, and a slow auto-orbit.
+**Snapshot** writes the current camera view as a PNG next to the model;
+**Turntable** writes a looping GIF of one full orbit, with the playing animation
+stepped in sync, sized by `threews3d.turntableFrames` and `turntableSize`.
Draco, KTX2/Basis, and meshopt assets decode locally: the decoders ship inside
the extension rather than loading from a CDN, so a meshopt-compressed three.ws
@@ -32,17 +37,106 @@ avatar (which is most of them) renders on a plane with no connection.
**Every model carries a report.** The panel behind the **Report** button lists
triangles, vertices, meshes, materials, textures and their weight, animations,
-rig size, and the glTF extensions in use, followed by optimization notes:
-oversized textures, non-indexed primitives, a mesh that would benefit from
-compression. It is the same inspector that powers the
-[`/validation`](https://three.ws/validation) page, running in the extension host,
-so your model is never uploaded anywhere to be measured.
-
-**Models can be generated in place.** `3D: Generate a Model from Text` describes
-an object; `3D: Generate an Avatar from Text` describes a character. Both call
-the free [3D Studio](./mcp-studio.md) lane, save the finished GLB into your
-workspace, and open it in the viewer. `3D: Rig a Model for Animation` sends a
-static model to the auto-rigger and brings back one with a humanoid skeleton.
+rig size, and the glTF extensions in use, followed by optimization notes. It is
+the same inspector that powers the [`/validation`](https://three.ws/validation)
+page, running in the extension host, so your model is never uploaded anywhere
+to be measured. The status bar repeats the headline numbers (triangles, size,
+bones, clips) while a model is open; click it to toggle the report.
+
+### Move it
+
+**Any rig plays the library.** Press **Animate** (or run `3D: Try a Library
+Animation on this Model`) and a searchable picker lists the three.ws motion
+library: 2,800 clips, the same ones the `/animations` gallery and the
+`<agent-3d>` embed play. Pick one and it plays on the open model immediately.
+Behind that, the clip (authored on the platform's canonical humanoid skeleton)
+is retargeted onto the model's own bones inside the viewer by
+[`src/animation-retarget.js`](../src/animation-retarget.js), the retargeter
+that runs on three.ws itself, so Mixamo, Avaturn, VRM/VRoid, Unreal, Daz,
+MakeHuman, Blender `.L/.R`, and simple `shoulderL` rigs all work. A rig it
+cannot drive gets a message with how many bones matched and an offer to rig it.
+
+**Bake it into the file.** While a library clip is playing, **Bake clip**
+writes it into a copy of the model (`astronaut-waving.glb`) as real glTF
+animation samplers and channels on the model's nodes, so the clip plays in
+Unity, Unreal, Blender, `<model-viewer>`, or `<agent-3d>`, not only in this
+viewer. Meshopt-compressed models are re-encoded rather than inflated.
+
+**Or describe the motion.** `3D: Animate this Model from a Text Prompt` sends
+"waving confidently with the right hand" or "a slow tai-chi sweep" to the
+text-to-motion worker on the three.ws GPU fleet, which samples a clip in the
+same format, and retargets it the same way. The picker offers this as its
+first entry, so a search that finds nothing turns into a generation.
+
+### Make it
+
+**Models can be generated in place.** `3D: Generate a Model from Text`
+describes an object; `3D: Generate an Avatar from Text` describes a character.
+Both call the free [3D Studio](./mcp-studio.md) lane, save the finished GLB
+into your workspace, and open it in the viewer.
+
+**Refine by describing a change.** **Refine** (or `3D: Refine this Model by
+Describing a Change`) asks for the change in words: "make it metallic", "bigger
+helmet", "add a cape". The studio generates a new version anchored to the
+current one, folding the original prompt into the new one so form and subject
+carry forward, and records it in a version lineage. The new file saves as
+`name-v2.glb`, then `-v3`, and the extension remembers the lineage per file, so
+refining a refinement extends one history instead of forking a new one. After
+each refinement you can jump straight to a structural comparison with the
+previous version.
+
+**Rig a static model.** `3D: Rig a Model for Animation` sends it to the
+auto-rigger and brings back one with a humanoid skeleton, then offers to try an
+animation on it.
+
+**Ask a vision model what is wrong.** **Check quality** renders the current
+view, sends the PNG to the quality gate that scores every three.ws generation,
+and opens the verdict as a Markdown review: realism and completeness scores,
+the subject it recognised, the defects it can see ("fused fingers", "back is
+unfinished"), and a concrete fix. One click feeds that fix into **Refine**.
+Because the render is what travels, this works on a file that has never left
+your disk.
+
+### Ship it
+
+**Optimize for the web, locally.** `3D: Optimize a Copy for the Web` runs the
+asset pipeline's passes in the extension host: dedup, prune, weld, resample
+animations, and meshopt compression, with a **Compact** preset that adds vertex
+quantization. It writes `name.web.glb` next to the original and reports the
+savings ("12.0 MB → 3.1 MB (−74%), 210k → 98k vertices"). Nothing is uploaded.
+Textures are left as they are; texture resizing stays in the server pipeline.
+
+**Compare with the committed version.** `3D: Compare with the Committed
+Version` reads the model at `HEAD` straight out of git (no checkout), opens it
+in a second viewer beside your working copy, and writes a structural diff as a
+Markdown review using [`@three-ws/glb-diff`](../packages/glb-diff): what changed
+in geometry, materials, textures, skeleton, and animations, with a severity that
+says whether a consumer of the model breaks. It is the review a binary blob in a
+pull request cannot give you. The same comparison is offered after a refinement
+(against the previous version) and after an optimization (against the original).
+
+**`<agent-3d>` embeds are checked as you type.** In HTML, JSX/TSX, Vue, Svelte,
+Astro, Markdown, and PHP files the extension runs the rules from the
+[Embedding guide](./embedding.md) as diagnostics:
+
+| Finding | Severity | Quick fix |
+|---|---|---|
+| Element has no `src`, `agent-id`, `avatar-id`, `manifest`, or `body` | error | |
+| Element has no width and height (it collapses to 0×0) | warning | Add an inline 400×500 size |
+| Library `<script>` on the `latest` channel | warning | Pin to the current release with its integrity hash |
+| Library `<script>` on a minor channel (`1`, `1.5`) | hint | Pin to the current release |
+| Pinned `<script>` with no `integrity` | hint | Add the published hash |
+| Pinned `<script>` whose `integrity` does not match the release (the browser refuses it) | error | Use the published hash |
+| A newer release exists than the one pinned | hint | Upgrade and re-pin |
+| Unknown or misspelt attribute (`modee`) | information | Rename to the closest attribute |
+| Invalid value for an enumerated attribute (`mode="popup"`) | warning | |
+| An `api-key` written into HTML | warning | |
+
+Hover any attribute for its documentation, get completions for attribute names
+(sources first) and enumerated values, and use the CodeLens above an embed to
+preview the referenced model in the viewer. The release and hash come from the
+live manifest at `/agent-3d/versions.json`, cached for ten minutes; offline,
+the rules that need it are skipped rather than guessed.
**The embed snippet is generated correctly.** `3D: Insert <agent-3d> Embed
Snippet` reads the live release manifest, pins the current library version, adds
@@ -62,14 +156,15 @@ its subresource-integrity hash, and writes the whole thing at your cursor:
></agent-3d>
```
-That is the snippet [Embedding](./embedding.md) documents, with the pin already
-resolved. Set `threews3d.embedChannel` to `latest`, `1`, or `1.5` if you would
-rather follow a moving channel than pin exact bytes.
+Set `threews3d.embedChannel` to `latest`, `1`, or `1.5` if you would rather
+follow a moving channel than pin exact bytes (the diagnostics will then remind
+you what that costs).
**The sidebar has both halves of your library.** The three.ws icon in the
activity bar lists every model in the workspace (grouped by folder, with file
-sizes) and, below it, recent public creations from the forge. Preview any of
-those in the viewer, and import the ones you want with one click.
+sizes; right-click for Animate, Refine, Optimize, Compare) and, below it,
+recent public creations from the forge. Preview any of those in the viewer, and
+import the ones you want with one click.
## Install
@@ -80,7 +175,7 @@ repository:
cd packages/vscode-3d
npm install
npm run package
-code --install-extension vscode-3d-0.1.0.vsix
+code --install-extension vscode-3d-0.2.0.vsix
```
For development, open `packages/vscode-3d` in VS Code and press <kbd>F5</kbd> to
@@ -93,54 +188,74 @@ launch an Extension Development Host with the extension loaded, or run
|---|---|
| `3D: Generate a Model from Text` | Free text to 3D. Saves the GLB and opens it. |
| `3D: Generate an Avatar from Text` | Same, tuned for characters. |
+| `3D: Refine this Model by Describing a Change` | New version anchored to this one; lineage kept. |
+| `3D: Try a Library Animation on this Model` | Search 2,800 clips; retarget and play in the viewer. |
+| `3D: Animate this Model from a Text Prompt` | Text to motion, retargeted onto the rig. |
| `3D: Rig a Model for Animation` | Adds a humanoid skeleton to a static model. |
+| `3D: Optimize a Copy for the Web` | Local dedup, weld, resample, meshopt (and quantize). |
+| `3D: Compare with the Committed Version` | Structural diff against `HEAD`, viewers side by side. |
+| `3D: Check Quality with a Vision Model` | Realism and completeness score, defects, fix hint. |
| `3D: Insert <agent-3d> Embed Snippet` | Writes a pinned, SRI-checked embed at the cursor. |
| `3D: Preview a Model URL` | Opens any hosted `.glb` in the viewer. |
| `3D: Open in the three.ws Viewer` | Opens the model in the browser viewer. |
| `3D: Copy the Model's three.ws URL` | Copies the hosted URL to the clipboard. |
| `3D: Save a PNG Snapshot of the View` | Writes the current view beside the model. |
+| `3D: Save a Turntable GIF of the Model` | Writes a looping orbit beside the model. |
+| `3D: Toggle the Model Report` | Shows or hides the report panel. |
## Settings
| Setting | Default | Purpose |
|---|---|---|
-| `threews3d.origin` | `https://three.ws` | Host of the studio API and the gallery. |
+| `threews3d.origin` | `https://three.ws` | Host of the studio API, gallery, motion library, and quality check. |
| `threews3d.downloadFolder` | `models` | Workspace folder generated models are saved into. |
| `threews3d.tier` | `draft` | Detail level for text to 3D: `draft`, `standard`, or `high`. All free. |
| `threews3d.environment` | `studio` | Image-based lighting: `studio`, `neutral`, or `none`. |
| `threews3d.showGrid` | `true` | Show the ground grid when a model opens. |
| `threews3d.autoRotate` | `false` | Orbit the camera automatically when a model opens. |
| `threews3d.embedChannel` | `pinned` | `pinned` resolves the current release plus its SRI hash; or name a channel. |
+| `threews3d.turntableFrames` | `36` | Frames in a turntable GIF (one full orbit). |
+| `threews3d.turntableSize` | `480` | Pixel size of each square turntable frame. |
+| `threews3d.embedDiagnostics` | `true` | Check `<agent-3d>` and library `<script>` tags in supported files. |
## What talks to the network, and what does not
-Three public, unauthenticated endpoints, and nothing else:
+Public, unauthenticated endpoints, and nothing else:
| Call | Endpoint | When |
|---|---|---|
-| Generate, rig | `POST /api/mcp-studio` | Only when you run a generate or rig command. |
+| Generate, rig, refine, collect a slow job | `POST /api/mcp-studio` | Only when you run one of those commands. |
+| Motion library | `GET /api/animations/library`, then the clip's CDN URL | The first time you open the Animate picker; one clip per pick. |
+| Text to motion | `POST /api/forge-motion`, `GET ?job=` | Only when you animate from a prompt. |
+| Quality check | `POST /api/forge-quality-check` | Only when you run the check; carries the render, never the model. |
| Gallery feed | `GET /api/forge-gallery` | The first time you expand the gallery view. |
-| Release pin | `GET /agent-3d/versions.json` | When you insert an embed snippet. |
+| Release pin | `GET /agent-3d/versions.json` | When you insert an embed snippet, and every ten minutes while a file with an embed is open. |
The viewer itself is local. Its webview runs under a content security policy that
-allows a single nonce-gated script and no remote origin, so the only thing that
-crosses the network while you look at a model is the model file. Opening,
-inspecting, animating, and snapshotting a `.glb` makes no requests at all.
+allows a single nonce-gated script and no remote origin, so the only things that
+cross the network while you look at a model are the model file and, when you
+pick one, a clip. Retargeting runs in the webview. Baking, optimizing,
+inspecting, and the structural diff run in the extension host with
+glTF-Transform. Opening, inspecting, animating with the file's own clips,
+optimizing, comparing, and snapshotting a `.glb` make no requests at all.
-## Embedding or rigging a local file
+## Embedding, rigging, or refining a local file
-Both need a URL that a browser or the rigger can fetch. Models the extension
-generated or imported remember where they came from, so those commands work
-without asking. For a `.glb` that has only ever existed on your disk, the
-extension asks for the https URL where it is hosted rather than handing you a
-snippet that renders nothing. Upload it (any static host works, or generate it
-through the extension in the first place) and paste the URL.
+Those three need a URL that a browser or the studio can fetch. Models the
+extension generated or imported remember where they came from, so those
+commands work without asking. For a `.glb` that has only ever existed on your
+disk, the extension asks for the https URL where it is hosted rather than
+handing you a snippet that renders nothing. Animating, baking, optimizing,
+comparing, and the quality check work on a purely local file.
## Related
-- [Embedding](./embedding.md) documents every `<agent-3d>` attribute.
+- [Embedding](./embedding.md) documents every `<agent-3d>` attribute the
+ diagnostics and completions know.
- [3D Studio MCP](./mcp-studio.md) is the free generation lane the extension
calls, and the same tools are available to any MCP client.
+- [Animations](./animations.md) covers the motion library the Animate picker
+ searches.
- [x402](./x402.md) covers the sibling extension for paid endpoints.
- [The 3D asset pipeline](./3d-asset-pipeline.md) explains what the optimization
- notes in the report are asking for.
+ notes in the report are asking for, and what Optimize does about them.
diff --git a/packages/README.md b/packages/README.md
index a655904f4..019cbf2f2 100644
--- a/packages/README.md
+++ b/packages/README.md
@@ -87,7 +87,7 @@ Publishable npm packages for the three.ws platform: client SDKs, agent tooling,
| [viewer-presets](viewer-presets/README.md) | Tuned visual presets for avatar viewers: light rigs, floor reflection, bloom, and PBR materials. |
| [vision-mcp](vision-mcp/README.md) | MCP server that analyzes and describes images through the three.ws vision pipeline. |
| [voice](voice/README.md) | Speech for avatars: ASR, TTS, and Audio2Face lipsync visemes in one import. |
-| [vscode-3d](vscode-3d/README.md) | VS Code extension that renders .glb and .gltf files in an editor tab, reports on them, and generates, rigs, and embeds models from the editor. |
+| [vscode-3d](vscode-3d/README.md) | VS Code extension that renders .glb and .gltf files in an editor tab, retargets library or text-described animations onto them and bakes the result, generates and refines models, optimizes and git-diffs them, AI quality-checks a render, and lint-checks `<agent-3d>` embeds with quick fixes. |
| [vscode-x402](vscode-x402/README.md) | VS Code extension to browse the x402 bazaar, decode 402 challenges, and pay per call from the editor. |
| [witness](witness/README.md) | Records a real user session as intent, not pixels, and compiles it into a Playwright spec that stays red until the reported bug is fixed. |
| [x402-fetch](x402-fetch/README.md) | Drop-in fetch wrapper that automatically pays x402 payment challenges. |
diff --git a/packages/vscode-3d/CHANGELOG.md b/packages/vscode-3d/CHANGELOG.md
index f7c381d1f..1963e787b 100644
--- a/packages/vscode-3d/CHANGELOG.md
+++ b/packages/vscode-3d/CHANGELOG.md
@@ -1,5 +1,47 @@
# Changelog
+## 0.2.0
+
+The extension stops being a viewer with a generate button and becomes a
+workbench for the whole life of a model.
+
+- **Animate any rig from the library.** `3D: Try a Library Animation` searches
+ the 2,800-clip three.ws motion library and plays the pick on the open model,
+ retargeted onto its skeleton in the viewer with the platform's own retargeter
+ (Mixamo, Avaturn, VRM, Unreal, Daz and simple rigs all map). **Bake clip**
+ then writes the retargeted clip into a copy of the file as real glTF
+ animation channels, so it plays in Unity, Unreal, Blender, or `<agent-3d>`.
+- **Animate from words.** `3D: Animate this Model from a Text Prompt` samples a
+ motion model on the three.ws GPU fleet ("waving confidently", "a slow tai-chi
+ sweep") and retargets the result the same way.
+- **Refine by describing a change.** `3D: Refine this Model` generates a new
+ version anchored to the current one ("make it metallic", "bigger helmet"),
+ saves it next to the original as `-v2`, `-v3`, and keeps the version lineage
+ so every further refinement extends the same history.
+- **Optimize for the web, locally.** `3D: Optimize a Copy for the Web` runs the
+ asset pipeline's passes (dedup, prune, weld, resample, meshopt, optionally
+ quantize) in the extension host and reports the savings. Nothing is uploaded.
+- **Compare with git.** `3D: Compare with the Committed Version` reads the
+ model at `HEAD` straight out of git, opens it beside the working copy, and
+ writes a structural diff (geometry, materials, textures, skeleton,
+ animations, severity) as a Markdown review. Also offered after a refinement
+ or an optimization.
+- **AI quality check.** `3D: Check Quality with a Vision Model` scores the
+ current render for realism and completeness, names visible defects, and
+ offers to feed the judge's fix hint straight into a refinement. Works for a
+ file that exists only on your disk, because the render is what travels.
+- **Turntable GIF.** One click writes a looping orbit of the model, with its
+ animation stepped in sync, next to the file. Made for READMEs and PRs.
+- **`<agent-3d>` language support.** In HTML, JSX/TSX, Vue, Svelte, Astro,
+ Markdown and PHP: diagnostics for an embed with no source or no size, a
+ library `<script>` on the `latest` channel, a pinned script with a missing or
+ stale integrity hash (an error: the browser refuses it), unknown or misspelt
+ attributes and bad enum values; quick fixes for each, including pinning to
+ the current release with its hash; hovers and completions for every
+ attribute; CodeLenses to preview the referenced model.
+- **Status bar.** Triangles, file size, bones and clips of the active model,
+ with a click to the report.
+
## 0.1.0
First release.
diff --git a/packages/vscode-3d/README.md b/packages/vscode-3d/README.md
index b42c4dbcb..64f69f9bc 100644
--- a/packages/vscode-3d/README.md
+++ b/packages/vscode-3d/README.md
@@ -1,40 +1,97 @@
# three.ws 3D for VS Code
-Preview 3D models where you already are. This extension makes `.glb` and `.gltf`
-files first-class in VS Code: double-click one and it renders, animates, and
-reports on itself in an editor tab, next to the code that ships it. When you need
-a model that does not exist yet, generate it from a text prompt on the free
-[three.ws](https://three.ws) lane without leaving the window.
-
-Nothing here is a preview stub. The viewer is three.js with the same lighting and
-decoders the three.ws site uses, the report is the platform's own inspector, and
-generation calls the live studio API.
+Preview, animate, refine, optimize, and review 3D models where you already are.
+This extension makes `.glb` and `.gltf` files first-class in VS Code: double-click
+one and it renders, animates, and reports on itself in an editor tab, next to the
+code that ships it. Try any of 2,800 library animations on its rig and bake the
+one you like into the file. Describe a change and get a new version. Optimize a
+copy for the web without uploading it. Diff it against the committed version
+before you open the pull request. And when you embed it, the editor checks the
+`<agent-3d>` tag the way it checks your TypeScript.
+
+Nothing here is a preview stub. The viewer is three.js with the same lighting,
+decoders and retargeter the three.ws site uses, the report and the diff are the
+platform's own inspectors, and generation calls the live studio API.
## Features
+### See it
+
- **A real viewer, as the default editor for model files.** Orbit, pan, and zoom
with the mouse. Animation clips get a picker, a play/pause button, and a scrub
bar. Toggle a wireframe, the skeleton, the ground grid, or a slow auto-orbit.
Draco, KTX2/Basis, and meshopt assets decode locally from decoders shipped
inside the extension, so a compressed three.ws avatar renders offline.
- **Model report.** Triangles, vertices, meshes, materials, texture count and
- weight, animations, rig size, and glTF extensions, plus optimization notes
- ("3 textures at 4K", "re-indexing typically halves the vertex count"). It is
- the same inspector that powers the three.ws `/validation` page, running in the
- extension host: your model is never uploaded anywhere to be measured.
+ weight, animations, rig size, and glTF extensions, plus optimization notes.
+ It is the same inspector that powers the three.ws `/validation` page, running
+ in the extension host: your model is never uploaded anywhere to be measured.
+ The status bar shows triangles, size, bones and clips while a model is open.
+- **Snapshots and turntables.** `Snapshot` writes the current view as a PNG
+ beside the model. `Turntable` writes a looping GIF of a full orbit, with the
+ playing animation stepped in sync, for a README or a pull request.
+
+### Move it
+
+- **Try a library animation.** `Animate` in the toolbar (or `3D: Try a Library
+ Animation`) searches the three.ws motion library, 2,800 clips baked on the
+ canonical humanoid skeleton, and plays your pick on the open model. The clip is
+ retargeted onto the model's own bones in the viewer by the platform's
+ retargeter, which knows Mixamo, Avaturn, VRM/VRoid, Unreal, Daz, MakeHuman,
+ Blender `.L/.R` and simple `shoulderL` conventions. If a rig cannot be driven,
+ the message says how many bones matched and offers to rig it.
+- **Bake it in.** `Bake clip` writes the retargeted clip into a copy of the file
+ as real glTF animation samplers and channels, so it plays in Unity, Unreal,
+ Blender, `<model-viewer>`, or `<agent-3d>`. Meshopt-compressed models are
+ re-encoded, not inflated.
+- **Animate from words.** `3D: Animate this Model from a Text Prompt` sends
+ "waving confidently with the right hand" to the text-to-motion worker on the
+ three.ws GPU fleet and retargets the sampled clip the same way.
+
+### Make it
+
- **Text to 3D.** `3D: Generate a Model from Text` for an object or prop,
`3D: Generate an Avatar from Text` for a character. Both run on the free lane
(no key, no account), save the GLB into your workspace, and open it.
+- **Refine by describing a change.** `Refine` (or `3D: Refine this Model`) takes
+ "make it metallic" or "bigger helmet" and generates a new version anchored to
+ the current one, carrying its form and materials forward. Versions save as
+ `name-v2.glb`, `name-v3.glb`, and the lineage is remembered, so refining a
+ refinement extends the same history.
- **Auto-rigging.** `3D: Rig a Model for Animation` sends a static model to the
three.ws rigger and brings back one with a humanoid skeleton and skin weights.
+- **AI quality check.** `Check quality` renders the current view and asks a
+ vision model to score realism and completeness against the subject, list the
+ defects it can see, and suggest a fix. One click feeds that fix into a
+ refinement. Because the render is what travels, it works on a model that has
+ never left your disk.
+
+### Ship it
+
+- **Optimize for the web, locally.** `3D: Optimize a Copy for the Web` runs the
+ asset pipeline's passes in the extension host (dedup, prune, weld, resample,
+ meshopt; the Compact preset adds vertex quantization) and writes `name.web.glb`
+ with a before/after summary. Textures are left as they are.
+- **Compare with the committed version.** `3D: Compare with the Committed
+ Version` reads the file at `HEAD` straight out of git, opens it in a second
+ viewer beside yours, and writes a structural diff as Markdown: what changed in
+ geometry, materials, textures, skeleton and animations, with a severity that
+ says whether a consumer of the model breaks. The same comparison is offered
+ after a refinement or an optimization.
+- **`<agent-3d>` embeds, checked as you type.** In HTML, JSX/TSX, Vue, Svelte,
+ Astro, Markdown and PHP files the extension reports an embed with no source
+ (error), no size (warning: it collapses to 0×0), a library `<script>` on the
+ `latest` channel (warning), a pinned script with no `integrity` (hint) or a
+ stale one (error: the browser refuses the file), unknown or misspelt
+ attributes, and invalid enum values. Every finding has a quick fix, including
+ "pin to the current release with its hash". Hover any attribute for its docs,
+ get completions for attribute names and values, and use the CodeLens to
+ preview the referenced model in the viewer.
- **Embed snippet, correctly pinned.** `3D: Insert <agent-3d> Embed Snippet`
resolves the current library version and its subresource-integrity hash from
the live release manifest and writes a ready-to-paste snippet at your cursor.
- **Forge gallery.** Browse recent public three.ws creations in the sidebar,
preview any of them in the viewer, and import the ones you want.
-- **PNG snapshots.** `Snapshot` in the viewer toolbar writes the current camera
- view as a PNG beside the model, sized to the panel. Useful for a README, a PR,
- or a design review.
## Install
@@ -51,7 +108,7 @@ Extension Development Host, or build a `.vsix` and install it:
```bash
npm run package
-code --install-extension vscode-3d-0.1.0.vsix
+code --install-extension vscode-3d-0.2.0.vsix
```
## Use it
@@ -60,11 +117,11 @@ Open any `.glb` or `.gltf` in the explorer. That is the whole setup: the viewer
is the default editor for those files, and the **three.ws 3D** icon in the
activity bar lists every model in the workspace alongside the public gallery.
-To make something new, run **3D: Generate a Model from Text** from the command
-palette, describe one subject ("a friendly round robot mascot, glossy white
-plastic, big blue eyes"), and the finished GLB lands in your `models/` folder and
-opens. Then either **Rig for animation** to make it posable, or **Insert embed**
-to put it on a page:
+A typical loop: **3D: Generate an Avatar from Text**, describe a character, and
+the GLB lands in `models/` and opens. Press **Animate**, type `wave`, and watch
+it wave. Press **Bake clip** to save `astronaut-waving.glb`. Press **Refine**,
+type `add a glowing visor`, and get `astronaut-v2.glb`. Press **Optimize** for
+`astronaut-v2.web.glb`, then **Insert embed** to put it on a page:
```html
<script
@@ -80,55 +137,79 @@ to put it on a page:
></agent-3d>
```
+If that snippet later drifts (someone swaps the URL to `latest`, or the hash goes
+stale after a release), the editor underlines it and the quick fix repairs it.
+
## Commands
| Command | What it does |
|---|---|
| `3D: Generate a Model from Text` | Free text to 3D. Saves the GLB and opens it. |
| `3D: Generate an Avatar from Text` | Same, tuned for characters. |
+| `3D: Refine this Model by Describing a Change` | New version anchored to this one; lineage kept. |
+| `3D: Try a Library Animation on this Model` | Search 2,800 clips; retarget and play in the viewer. |
+| `3D: Animate this Model from a Text Prompt` | Text to motion on the GPU fleet, retargeted onto the rig. |
| `3D: Rig a Model for Animation` | Adds a humanoid skeleton to a static model. |
+| `3D: Optimize a Copy for the Web` | Local dedup, weld, resample, meshopt (and quantize). |
+| `3D: Compare with the Committed Version` | Structural diff against `HEAD`, side-by-side viewers. |
+| `3D: Check Quality with a Vision Model` | Realism and completeness score, defects, fix hint. |
| `3D: Insert <agent-3d> Embed Snippet` | Writes a pinned, SRI-checked embed at the cursor. |
| `3D: Preview a Model URL` | Opens any hosted `.glb` in the viewer. |
| `3D: Open in the three.ws Viewer` | Opens the model in the browser viewer. |
| `3D: Copy the Model's three.ws URL` | Copies the hosted URL to the clipboard. |
| `3D: Save a PNG Snapshot of the View` | Writes the current view beside the model. |
+| `3D: Save a Turntable GIF of the Model` | Writes a looping orbit beside the model. |
+| `3D: Toggle the Model Report` | Shows or hides the report panel. |
## Settings
| Setting | Default | Purpose |
|---|---|---|
-| `threews3d.origin` | `https://three.ws` | Host of the studio API and the gallery. |
+| `threews3d.origin` | `https://three.ws` | Host of the studio API, the gallery, the motion library, and the quality check. |
| `threews3d.downloadFolder` | `models` | Workspace folder generated models are saved into. |
| `threews3d.tier` | `draft` | Detail level for text to 3D: `draft`, `standard`, or `high`. All free. |
| `threews3d.environment` | `studio` | Image-based lighting: `studio`, `neutral`, or `none`. |
| `threews3d.showGrid` | `true` | Show the ground grid when a model opens. |
| `threews3d.autoRotate` | `false` | Orbit the camera automatically when a model opens. |
| `threews3d.embedChannel` | `pinned` | `pinned` resolves the current release plus its SRI hash; or name a channel such as `latest`, `1`, or `1.5`. |
+| `threews3d.turntableFrames` | `36` | Frames in a turntable GIF (one full orbit). |
+| `threews3d.turntableSize` | `480` | Pixel size of each square turntable frame. |
+| `threews3d.embedDiagnostics` | `true` | Check `<agent-3d>` and library `<script>` tags in supported files. |
## How it works
-Generation, rigging, and the gallery go through public, unauthenticated three.ws
-endpoints:
+Generation, refinement, rigging, the library, and the gallery go through public,
+unauthenticated three.ws endpoints:
- `POST /api/mcp-studio` is the JSON-RPC studio. The extension calls the
- `forge_free`, `text_to_avatar`, and `rig_mesh` tools. No key, no account, no
- payment: the platform's server-side keys cover provider cost.
+ `forge_free`, `text_to_avatar`, `rig_mesh`, `refine_model`, and `check_job`
+ tools. No key, no account, no payment: the platform's server-side keys cover
+ provider cost.
+- `GET /api/animations/library` lists the motion library; each clip is fetched
+ from the CDN as a three.js `AnimationClip` JSON when you pick it.
+- `POST /api/forge-motion` starts a text-to-motion job; `GET ?job=` polls it.
+- `POST /api/forge-quality-check` scores a render you send as a PNG.
- `GET /api/forge-gallery` is the public feed of recent creations.
-- `GET /agent-3d/versions.json` is the library release manifest the embed snippet
- pins against.
+- `GET /agent-3d/versions.json` is the library release manifest the embed
+ snippet and the embed diagnostics pin against.
Everything else is local. The viewer runs three.js inside a webview whose content
security policy allows exactly one nonce-gated script and no remote origin, so
-the only thing that crosses the network is the model file itself. The Draco and
+the only things that cross the network while you look at a model are the model
+file and, when you pick one, a clip. Retargeting runs in the webview with the
+platform's `animation-retarget.js`; baking, optimization, inspection and the
+structural diff run in the extension host with glTF-Transform. The Draco and
Basis decoders are copied out of three.js at build time and served from the
extension, never from a CDN.
-## Which model can be embedded or rigged
+## Which model can be embedded, rigged, or refined
-Both need a URL the browser (or the rigger) can fetch. Models this extension
-generated or imported carry their three.ws URL, so those commands just work. For
-a local file that has never been uploaded, the extension asks for the https URL
-where it is hosted rather than handing you a snippet that renders nothing.
+Those three need a URL the browser (or the studio) can fetch. Models this
+extension generated or imported carry their three.ws URL, so the commands just
+work. For a local file that has never been uploaded, the extension asks for the
+https URL where it is hosted rather than handing you a snippet that renders
+nothing. Animating, baking, optimizing, comparing, and the quality check all
+work on a purely local file.
## Develop
@@ -138,10 +219,13 @@ npm test # builds, then runs the suite
```
`npm test` activates the bundled extension against a stand-in extension host and
-asserts that every command, view, and custom editor the manifest promises really
-registers, that the viewer webview is nonce-gated and CDN-free, and that the
-model reaches it. The studio, gallery, embed, and naming modules are unit tested
-against local HTTP servers.
+asserts that every command, view, language provider, and custom editor the
+manifest promises really registers, that the viewer webview is nonce-gated and
+CDN-free, that an HTML file with a broken embed yields the right diagnostics and
+quick fixes, and that a canonical clip retargets onto the committed reference
+avatar and bakes into it. The studio, library, motion, refine, quality, embed,
+optimize, compare and naming modules are unit tested against local HTTP servers
+and real committed GLBs.
## Related
@@ -149,8 +233,9 @@ against local HTTP servers.
- [`@three-ws/vscode-x402`](../vscode-x402) is the sibling extension for paying
x402 endpoints from the editor.
- [`docs/embedding.md`](../../docs/embedding.md) documents every `<agent-3d>`
- attribute the inserted snippet can take.
-- [`@three-ws/glb-tools`](../glb-tools) does inspection and baking from the shell
- and CI.
+ attribute the diagnostics and completions know.
+- [`@three-ws/glb-diff`](../glb-diff) is the structural differ behind the
+ compare command, also available as a CLI for CI.
+- [`@three-ws/glb-tools`](../glb-tools) does inspection and baking from the shell.
Apache-2.0. Built by [three.ws](https://three.ws).
diff --git a/packages/vscode-3d/build.mjs b/packages/vscode-3d/build.mjs
index e3aceeafb..916fdf281 100644
--- a/packages/vscode-3d/build.mjs
+++ b/packages/vscode-3d/build.mjs
@@ -49,6 +49,10 @@ const host = {
format: 'cjs',
target: 'node18',
external: ['vscode'],
+ // ndarray-pixels (a texture dependency of @gltf-transform/functions) imports
+ // the native `sharp` binary on Node. Nothing here resizes textures, and a
+ // native module cannot ship in a .vsix, so it is aliased to a stub.
+ alias: { sharp: './src/shims/sharp.js' },
sourcemap: !production,
minify: production,
logLevel: 'info',
diff --git a/packages/vscode-3d/media/viewer.css b/packages/vscode-3d/media/viewer.css
index f2fed6f3b..9101d9dcc 100644
--- a/packages/vscode-3d/media/viewer.css
+++ b/packages/vscode-3d/media/viewer.css
@@ -225,3 +225,37 @@ body {
#error[hidden] {
display: none;
}
+
+#toolbar .sep {
+ width: 1px;
+ align-self: stretch;
+ margin: 2px 2px;
+ background: var(--vscode-editorWidget-border, rgba(128, 128, 128, 0.35));
+}
+
+#busy {
+ position: absolute;
+ top: var(--pad);
+ left: 50%;
+ transform: translateX(-50%);
+ display: flex;
+ align-items: center;
+ gap: 8px;
+ padding: 6px 12px;
+ background: var(--vscode-editorWidget-background);
+ border: 1px solid var(--vscode-editorWidget-border, transparent);
+ border-radius: var(--radius);
+ box-shadow: 0 2px 8px rgba(0, 0, 0, 0.25);
+ font-size: 0.92em;
+ transition: opacity 0.15s ease;
+}
+
+#busy .spinner {
+ width: 14px;
+ height: 14px;
+}
+
+#busy[hidden],
+#bake[hidden] {
+ display: none;
+}
diff --git a/packages/vscode-3d/package.json b/packages/vscode-3d/package.json
index d38a69ed6..8b6ca72fa 100644
--- a/packages/vscode-3d/package.json
+++ b/packages/vscode-3d/package.json
@@ -1,8 +1,8 @@
{
"name": "vscode-3d",
"displayName": "three.ws 3D",
- "description": "Preview GLB and glTF models inside VS Code, generate new ones from a text prompt on the free three.ws lane, rig them for animation, and paste a working <agent-3d> embed. Powered by three.ws.",
- "version": "0.1.0",
+ "description": "Preview GLB and glTF models inside VS Code, try 2,800 library animations or a text-described motion on any rig and bake the result, generate and refine models from a prompt on the free three.ws lane, optimize for the web, diff a model against git, AI quality-check a render, and get diagnostics and completions for <agent-3d> embeds. Powered by three.ws.",
+ "version": "0.2.0",
"publisher": "threews",
"license": "Apache-2.0",
"homepage": "https://three.ws",
@@ -32,7 +32,14 @@
"text to 3d",
"preview",
"three",
- "webgl"
+ "webgl",
+ "animation",
+ "retarget",
+ "mixamo",
+ "optimize",
+ "meshopt",
+ "diff",
+ "embed"
],
"type": "module",
"main": "./dist/extension.cjs",
@@ -41,7 +48,16 @@
"color": "#0E1116",
"theme": "dark"
},
- "activationEvents": [],
+ "activationEvents": [
+ "onLanguage:html",
+ "onLanguage:javascriptreact",
+ "onLanguage:typescriptreact",
+ "onLanguage:vue",
+ "onLanguage:svelte",
+ "onLanguage:astro",
+ "onLanguage:markdown",
+ "onLanguage:php"
+ ],
"contributes": {
"customEditors": [
{
@@ -85,7 +101,7 @@
"viewsWelcome": [
{
"view": "threews3d.models",
- "contents": "No .glb or .gltf files here yet.\n\nGenerate one from a text prompt on the free three.ws lane. No key, no account.\n[Generate a 3D model](command:threews3d.generateModel)\n[Generate an avatar](command:threews3d.generateAvatar)\n\nAlready have a model on a URL?\n[Preview a model URL](command:threews3d.previewUrl)"
+ "contents": "No .glb or .gltf files here yet.\n\nGenerate one from a text prompt on the free three.ws lane. No key, no account.\n[Generate a 3D model](command:threews3d.generateModel)\n[Generate an avatar](command:threews3d.generateAvatar)\n\nAlready have a model on a URL?\n[Preview a model URL](command:threews3d.previewUrl)\n\nEmbedding one? Open an HTML file with an <agent-3d> tag: the extension checks it and completes its attributes."
}
],
"commands": [
@@ -103,6 +119,42 @@
"command": "threews3d.rigModel",
"title": "3D: Rig a Model for Animation"
},
+ {
+ "command": "threews3d.refineModel",
+ "title": "3D: Refine this Model by Describing a Change",
+ "icon": "$(wand)"
+ },
+ {
+ "command": "threews3d.animate",
+ "title": "3D: Try a Library Animation on this Model",
+ "icon": "$(play)"
+ },
+ {
+ "command": "threews3d.animateFromText",
+ "title": "3D: Animate this Model from a Text Prompt"
+ },
+ {
+ "command": "threews3d.optimizeModel",
+ "title": "3D: Optimize a Copy for the Web",
+ "icon": "$(zap)"
+ },
+ {
+ "command": "threews3d.compareWithHead",
+ "title": "3D: Compare with the Committed Version",
+ "icon": "$(git-compare)"
+ },
+ {
+ "command": "threews3d.checkQuality",
+ "title": "3D: Check Quality with a Vision Model"
+ },
+ {
+ "command": "threews3d.exportTurntable",
+ "title": "3D: Save a Turntable GIF of the Model"
+ },
+ {
+ "command": "threews3d.toggleReport",
+ "title": "3D: Toggle the Model Report"
+ },
{
"command": "threews3d.insertEmbed",
"title": "3D: Insert <agent-3d> Embed Snippet"
@@ -174,18 +226,58 @@
{
"command": "threews3d.rigModel",
"when": "view == threews3d.models && viewItem == threews3dModel"
+ },
+ {
+ "command": "threews3d.animate",
+ "when": "view == threews3d.models && viewItem == threews3dModel",
+ "group": "threews3d@1"
+ },
+ {
+ "command": "threews3d.refineModel",
+ "when": "view == threews3d.models && viewItem == threews3dModel",
+ "group": "threews3d@2"
+ },
+ {
+ "command": "threews3d.optimizeModel",
+ "when": "view == threews3d.models && viewItem == threews3dModel",
+ "group": "threews3d@3"
+ },
+ {
+ "command": "threews3d.compareWithHead",
+ "when": "view == threews3d.models && viewItem == threews3dModel",
+ "group": "threews3d@4"
}
],
"explorer/context": [
+ {
+ "command": "threews3d.animate",
+ "when": "resourceExtname == .glb || resourceExtname == .gltf",
+ "group": "threews3d@1"
+ },
+ {
+ "command": "threews3d.refineModel",
+ "when": "resourceExtname == .glb",
+ "group": "threews3d@2"
+ },
{
"command": "threews3d.rigModel",
"when": "resourceExtname == .glb",
- "group": "threews3d@1"
+ "group": "threews3d@3"
+ },
+ {
+ "command": "threews3d.optimizeModel",
+ "when": "resourceExtname == .glb || resourceExtname == .gltf",
+ "group": "threews3d@4"
+ },
+ {
+ "command": "threews3d.compareWithHead",
+ "when": "resourceExtname == .glb || resourceExtname == .gltf",
+ "group": "threews3d@5"
},
{
"command": "threews3d.insertEmbed",
"when": "resourceExtname == .glb",
- "group": "threews3d@2"
+ "group": "threews3d@6"
}
],
"commandPalette": [
@@ -204,6 +296,18 @@
{
"command": "threews3d.refreshGallery",
"when": "false"
+ },
+ {
+ "command": "threews3d.toggleReport",
+ "when": "activeCustomEditorId == threews3d.modelViewer"
+ },
+ {
+ "command": "threews3d.exportTurntable",
+ "when": "activeCustomEditorId == threews3d.modelViewer"
+ },
+ {
+ "command": "threews3d.saveSnapshot",
+ "when": "activeCustomEditorId == threews3d.modelViewer"
}
]
},
@@ -254,6 +358,25 @@
"type": "string",
"default": "pinned",
"description": "Version channel used by the inserted <agent-3d> snippet: 'pinned' resolves the current release and adds its SRI hash, or name a channel such as latest, 1, or 1.5."
+ },
+ "threews3d.turntableFrames": {
+ "type": "integer",
+ "default": 36,
+ "minimum": 8,
+ "maximum": 120,
+ "description": "Frames in a turntable GIF (one full orbit). More frames are smoother and larger."
+ },
+ "threews3d.turntableSize": {
+ "type": "integer",
+ "default": 480,
+ "minimum": 128,
+ "maximum": 1024,
+ "description": "Pixel size of a turntable GIF frame (square)."
+ },
+ "threews3d.embedDiagnostics": {
+ "type": "boolean",
+ "default": true,
+ "description": "Check <agent-3d> tags and library script tags in HTML, JSX, Vue, Svelte, Astro, Markdown and PHP files: missing source or size, unpinned or stale library versions, unknown attributes."
}
}
}
diff --git a/packages/vscode-3d/src/animations.js b/packages/vscode-3d/src/animations.js
index 7a0a92686..35a8cfa48 100644
--- a/packages/vscode-3d/src/animations.js
+++ b/packages/vscode-3d/src/animations.js
@@ -94,10 +94,10 @@ export function clipSlug(label) {
*
* @param {string} origin
* @param {string} prompt
- * @param {{ signal?: AbortSignal, durationSeconds?: number, onStatus?: (message: string) => void }} [opts]
+ * @param {{ signal?: AbortSignal, durationSeconds?: number, onStatus?: (message: string) => void, pollMs?: number }} [opts]
* @returns {Promise<{ clipUrl: string, clip: object }>}
*/
-export async function generateMotion(origin, prompt, { signal, durationSeconds = 4, onStatus } = {}) {
+export async function generateMotion(origin, prompt, { signal, durationSeconds = 4, onStatus, pollMs = MOTION_POLL_MS } = {}) {
const base = normalizeOrigin(origin);
const start = await fetch(new URL('/api/forge-motion', base).href, {
method: 'POST',
@@ -119,7 +119,7 @@ export async function generateMotion(origin, prompt, { signal, durationSeconds =
const poll = new URL('/api/forge-motion', base);
poll.searchParams.set('job', started.job_id);
while (Date.now() < deadline) {
- await sleep(MOTION_POLL_MS, signal);
+ await sleep(pollMs, signal);
const status = await getJson(poll.href, { signal, what: 'the motion job' });
if (status?.status === 'failed') {
throw new Error(status.error || 'the motion model could not animate that prompt');
diff --git a/packages/vscode-3d/src/embed-language-vscode.js b/packages/vscode-3d/src/embed-language-vscode.js
new file mode 100644
index 000000000..5db1d31d3
--- /dev/null
+++ b/packages/vscode-3d/src/embed-language-vscode.js
@@ -0,0 +1,246 @@
+// Wires the pure embed rules (embed-language.js) into VS Code: diagnostics with
+// quick fixes, hovers, completions, and CodeLenses for <agent-3d> in HTML and
+// the frameworks people embed from.
+
+import * as vscode from 'vscode';
+import {
+ ATTRIBUTES,
+ attributeAt,
+ completionContext,
+ diagnose,
+ findEmbeds,
+ findLibraryScripts,
+ tagHover,
+} from './embed-language.js';
+import { resolveEmbedRelease } from './embed.js';
+
+export const LANGUAGES = Object.freeze([
+ 'html',
+ 'javascriptreact',
+ 'typescriptreact',
+ 'javascript',
+ 'typescript',
+ 'vue',
+ 'svelte',
+ 'astro',
+ 'markdown',
+ 'php',
+ 'handlebars',
+ 'erb',
+ 'twig',
+ 'liquid',
+]);
+
+const SELECTOR = LANGUAGES.map((language) => ({ language }));
+const SEVERITY = {
+ error: vscode.DiagnosticSeverity.Error,
+ warning: vscode.DiagnosticSeverity.Warning,
+ information: vscode.DiagnosticSeverity.Information,
+ hint: vscode.DiagnosticSeverity.Hint,
+};
+const RELEASE_TTL_MS = 10 * 60_000;
+const DEBOUNCE_MS = 250;
+
+/**
+ * @param {vscode.ExtensionContext} context
+ * @param {{ origin: () => string, output: vscode.OutputChannel }} deps
+ */
+export function registerEmbedLanguage(context, { origin, output }) {
+ const diagnostics = vscode.languages.createDiagnosticCollection('three.ws 3D');
+ // VS Code hands code-action providers copies of the diagnostics, so the fix
+ // for each one is kept here, keyed by document, code, and range.
+ const fixes = new Map();
+ const timers = new Map();
+ let release = null;
+ let releaseAt = 0;
+ let releasePromise = null;
+
+ /** The current library release, refreshed every ten minutes; null offline. */
+ const currentRelease = () => {
+ if (release && Date.now() - releaseAt < RELEASE_TTL_MS) return Promise.resolve(release);
+ if (!releasePromise) {
+ releasePromise = resolveEmbedRelease(origin(), 'pinned')
+ .then((r) => {
+ release = r.integrity ? r : null;
+ releaseAt = Date.now();
+ return release;
+ })
+ .catch(() => null)
+ .finally(() => {
+ releasePromise = null;
+ });
+ }
+ return releasePromise;
+ };
+
+ /** Cheap pre-check so documents with no embed never run the rules. */
+ const mentions = (doc) => /agent-3d/i.test(doc.getText());
+
+ const refresh = async (doc) => {
+ if (!LANGUAGES.includes(doc.languageId)) return;
+ if (!vscode.workspace.getConfiguration('threews3d').get('embedDiagnostics', true) || !mentions(doc)) {
+ diagnostics.delete(doc.uri);
+ return;
+ }
+ const text = doc.getText();
+ const rel = await currentRelease();
+ const findings = diagnose(text, { release: rel, origin: origin() });
+ const docFixes = new Map();
+ diagnostics.set(
+ doc.uri,
+ findings.map((f) => {
+ const r = range(doc, f.start, f.end);
+ const d = new vscode.Diagnostic(r, f.message, SEVERITY[f.severity]);
+ d.source = 'three.ws';
+ d.code = f.code;
+ if (f.fix) docFixes.set(fixKey(f.code, r), f.fix);
+ return d;
+ }),
+ );
+ fixes.set(doc.uri.toString(), docFixes);
+ };
+
+ const schedule = (doc) => {
+ clearTimeout(timers.get(doc.uri.toString()));
+ timers.set(
+ doc.uri.toString(),
+ setTimeout(() => {
+ timers.delete(doc.uri.toString());
+ refresh(doc).catch((err) => output.appendLine(`embed diagnostics failed: ${err?.message || err}`));
+ }, DEBOUNCE_MS),
+ );
+ };
+
+ for (const doc of vscode.workspace.textDocuments) schedule(doc);
+
+ const codeActions = vscode.languages.registerCodeActionsProvider(SELECTOR, {
+ provideCodeActions(doc, _range, ctx) {
+ const actions = [];
+ const docFixes = fixes.get(doc.uri.toString());
+ if (!docFixes) return actions;
+ for (const d of ctx.diagnostics) {
+ if (d.source !== 'three.ws') continue;
+ const fix = docFixes.get(fixKey(d.code, d.range));
+ if (!fix) continue;
+ const action = new vscode.CodeAction(fix.title, vscode.CodeActionKind.QuickFix);
+ action.diagnostics = [d];
+ action.isPreferred = true;
+ action.edit = new vscode.WorkspaceEdit();
+ action.edit.replace(doc.uri, range(doc, fix.start, fix.end), fix.text);
+ actions.push(action);
+ }
+ return actions;
+ },
+ });
+
+ const hover = vscode.languages.registerHoverProvider(SELECTOR, {
+ provideHover(doc, position) {
+ const text = doc.getText();
+ const offset = doc.offsetAt(position);
+ for (const embed of findEmbeds(text)) {
+ if (offset < embed.start || offset > embed.end) continue;
+ if (offset <= embed.tagEnd) {
+ return new vscode.Hover(markdown(tagHover(origin())), range(doc, embed.start, embed.tagEnd));
+ }
+ const a = attributeAt(embed, offset);
+ const def = a && ATTRIBUTES.find((x) => x.name === a.name.toLowerCase());
+ if (def) {
+ const lines = [`**\`${def.name}\`**`, '', def.doc];
+ if (def.values) lines.push('', `Values: ${def.values.map((v) => `\`${v}\``).join(', ')}`);
+ if (def.flag) lines.push('', 'Boolean: present or absent.');
+ lines.push('', `[Embedding guide](${origin()}/docs/embedding)`);
+ return new vscode.Hover(markdown(lines.join('\n')), range(doc, a.start, a.nameEnd));
+ }
+ }
+ return null;
+ },
+ });
+
+ const completions = vscode.languages.registerCompletionItemProvider(
+ SELECTOR,
+ {
+ provideCompletionItems(doc, position) {
+ const text = doc.getText();
+ const ctx = completionContext(text, doc.offsetAt(position));
+ if (!ctx) return null;
+ if (ctx.kind === 'value') {
+ if (!ctx.def?.values) return null;
+ return ctx.def.values.map((v) => {
+ const item = new vscode.CompletionItem(v, vscode.CompletionItemKind.EnumMember);
+ item.detail = `${ctx.def.name} value`;
+ return item;
+ });
+ }
+ const present = new Set(ctx.embed.attrs.map((a) => a.name.toLowerCase()));
+ return ATTRIBUTES.filter((a) => !present.has(a.name)).map((a, i) => {
+ const item = new vscode.CompletionItem(a.name, vscode.CompletionItemKind.Property);
+ item.documentation = markdown(a.doc);
+ item.detail = a.source ? 'source' : a.flag ? 'flag' : 'attribute';
+ item.sortText = `${a.source ? '0' : '1'}${String(i).padStart(3, '0')}`;
+ item.insertText = a.flag
+ ? a.name
+ : new vscode.SnippetString(`${a.name}="\${1${a.values ? `|${a.values.join(',')}|` : ''}}"`);
+ return item;
+ });
+ },
+ },
+ ' ',
+ '"',
+ );
+
+ const lenses = vscode.languages.registerCodeLensProvider(SELECTOR, {
+ provideCodeLenses(doc) {
+ if (!mentions(doc)) return [];
+ const text = doc.getText();
+ const out = [];
+ for (const embed of findEmbeds(text)) {
+ const r = range(doc, embed.start, embed.tagEnd);
+ const model = embed.attrs.find((a) => ['body', 'src'].includes(a.name.toLowerCase()) && a.value && /^https?:\/\/.*\.(glb|gltf)(\?.*)?$/i.test(a.value.trim()));
+ if (model) {
+ out.push(new vscode.CodeLens(r, { title: '$(eye) Preview model', command: 'threews3d.previewUrl', arguments: [model.value.trim()] }));
+ }
+ out.push(new vscode.CodeLens(r, { title: 'Embedding guide', command: 'vscode.open', arguments: [vscode.Uri.parse(`${origin()}/docs/embedding`)] }));
+ }
+ for (const script of findLibraryScripts(text)) {
+ if (!script.exact) {
+ out.push(new vscode.CodeLens(range(doc, script.start, script.end), { title: '$(pin) Pin the library version', command: 'editor.action.quickFix' }));
+ }
+ }
+ return out;
+ },
+ });
+
+ return vscode.Disposable.from(
+ diagnostics,
+ codeActions,
+ hover,
+ completions,
+ lenses,
+ vscode.workspace.onDidOpenTextDocument(schedule),
+ vscode.workspace.onDidChangeTextDocument((e) => schedule(e.document)),
+ vscode.workspace.onDidChangeConfiguration((e) => {
+ if (e.affectsConfiguration('threews3d.embedDiagnostics')) {
+ for (const doc of vscode.workspace.textDocuments) schedule(doc);
+ }
+ }),
+ vscode.workspace.onDidCloseTextDocument((doc) => {
+ diagnostics.delete(doc.uri);
+ fixes.delete(doc.uri.toString());
+ }),
+ { dispose: () => timers.forEach((t) => clearTimeout(t)) },
+ );
+}
+
+function fixKey(code, r) {
+ return `${code}@${r.start.line}:${r.start.character}-${r.end.line}:${r.end.character}`;
+}
+
+function range(doc, start, end) {
+ return new vscode.Range(doc.positionAt(start), doc.positionAt(Math.max(start, end)));
+}
+
+function markdown(value) {
+ const md = new vscode.MarkdownString(value);
+ md.supportThemeIcons = true;
+ return md;
+}
diff --git a/packages/vscode-3d/src/embed-language.js b/packages/vscode-3d/src/embed-language.js
index ad108880a..cfa7ca729 100644
--- a/packages/vscode-3d/src/embed-language.js
+++ b/packages/vscode-3d/src/embed-language.js
@@ -152,26 +152,31 @@ export function attributeAt(embed, offset) {
return embed.attrs.find((a) => offset >= a.start && offset <= a.end) || null;
}
-/** Where in an embed the cursor sits: 'name' (typing an attribute), 'value' of an attribute, or null. */
+/**
+ * Where in an embed the cursor sits: naming an attribute, inside an
+ * attribute's value, or nowhere relevant. Works on a tag the user is still
+ * typing (no closing `>` yet), which is when completion matters most.
+ *
+ * @returns {{ kind: 'name', embed: { attrs: Attr[] } } | { kind: 'value', attr: { name: string }, def: object|null } | null}
+ */
export function completionContext(text, offset) {
- for (const embed of findEmbeds(text)) {
- if (offset <= embed.tagEnd || offset > embed.end) continue;
- for (const a of embed.attrs) {
- if (a.value !== null && offset > a.nameEnd && offset <= a.end) {
- const def = ATTRIBUTES.find((d) => d.name === a.name.toLowerCase());
- return { kind: 'value', attr: a, def: def || null, embed };
- }
- }
- const before = text.slice(embed.attrsStart, offset);
- if (/[\s]$/.test(before) || /[\s][^\s="']*$/.test(before)) return { kind: 'name', embed };
- }
- // An unterminated tag the user is still typing.
const open = text.lastIndexOf('<agent-3d', offset);
- if (open !== -1 && text.indexOf('>', open) === -1 || (open !== -1 && text.indexOf('>', open) >= offset)) {
- const before = text.slice(open + '<agent-3d'.length, offset);
- if (/[\s][^\s="']*$/.test(before) || /[\s]$/.test(before)) {
- return { kind: 'name', embed: { start: open, tagEnd: open + '<agent-3d'.length, attrsStart: open + '<agent-3d'.length, end: offset, attrs: parseAttrs(before, open + '<agent-3d'.length) } };
- }
+ if (open === -1) return null;
+ const attrsStart = open + '<agent-3d'.length;
+ if (offset < attrsStart) return null;
+ // A `>` between the tag and the cursor means the cursor is past this tag.
+ const close = text.indexOf('>', attrsStart);
+ if (close !== -1 && close < offset) return null;
+ const chunk = text.slice(attrsStart, offset);
+ // Inside an open quote: `mode="fl` or `mode='`.
+ const inValue = /([^\s"'<>\/=]+)\s*=\s*(["'])([^"']*)$/.exec(chunk);
+ if (inValue) {
+ const name = inValue[1].toLowerCase();
+ return { kind: 'value', attr: { name }, def: ATTRIBUTES.find((d) => d.name === name) || null };
+ }
+ // After whitespace (or a partial attribute name after whitespace): naming.
+ if (/\s$/.test(chunk) || /\s[^\s="'<>]*$/.test(chunk)) {
+ return { kind: 'name', embed: { start: open, tagEnd: attrsStart, attrsStart, end: offset, attrs: parseAttrs(chunk, attrsStart) } };
}
return null;
}
diff --git a/packages/vscode-3d/src/extension.js b/packages/vscode-3d/src/extension.js
index cc2b479e2..eefc290f7 100644
--- a/packages/vscode-3d/src/extension.js
+++ b/packages/vscode-3d/src/extension.js
@@ -1,25 +1,35 @@
// three.ws 3D: the VS Code extension entry point.
//
// Registers the model viewer (a custom editor for .glb/.gltf), the workspace and
-// gallery trees, and the commands that generate, rig, embed, and import models
-// through the live three.ws studio. Every call hits the real public API; nothing
-// here is stubbed.
+// gallery trees, the language features for <agent-3d> embeds, and the commands
+// that generate, refine, animate, optimize, compare, rig, embed, and import
+// models through the live three.ws studio. Every call hits the real public API;
+// nothing here is stubbed.
import * as vscode from 'vscode';
import { GalleryProvider } from './gallery-tree.js';
import { ModelLinks } from './links.js';
import { ModelsProvider } from './models-tree.js';
-import { ModelViewerProvider, openRemoteModel } from './viewer.js';
+import { ModelViewerProvider, VIEW_TYPE, openLocalModel, openRemoteModel, playClipIn, request } from './viewer.js';
import { TOOLS, callTool } from './studio.js';
import { downloadModel } from './download.js';
import { buildEmbedSnippet, resolveEmbedRelease, viewerUrl } from './embed.js';
-import { slugFromUrl } from './naming.js';
+import { slugFromUrl, uniqueName } from './naming.js';
import { activeViewer } from './active-panel.js';
+import { clipSlug, fetchClip, generateMotion, listLibrary } from './animations.js';
+import { refineModel } from './refine.js';
+import { PRESETS, describeSavings, optimizeGlb } from './optimize.js';
+import { committedBytes, compareModels } from './compare.js';
+import { checkQuality, qualityMarkdown } from './quality.js';
+import { createStatusBar } from './status-bar.js';
+import { registerEmbedLanguage } from './embed-language-vscode.js';
let output;
let links;
+let extensionContext;
export function activate(context) {
+ extensionContext = context;
output = vscode.window.createOutputChannel('three.ws 3D');
links = new ModelLinks(context.workspaceState);
@@ -39,6 +49,8 @@ export function activate(context) {
modelsView,
galleryView,
ModelViewerProvider.register(context, links, output),
+ createStatusBar(),
+ registerEmbedLanguage(context, { origin, output }),
);
const reg = (id, fn) => context.subscriptions.push(vscode.commands.registerCommand(id, fn));
@@ -46,22 +58,23 @@ export function activate(context) {
reg('threews3d.generateModel', () => generate('model'));
reg('threews3d.generateAvatar', () => generate('avatar'));
reg('threews3d.rigModel', (target) => rig(target));
+ reg('threews3d.refineModel', (target) => refine(target));
+ reg('threews3d.animate', (target) => animate(target));
+ reg('threews3d.animateFromText', (target) => animateFromText(target));
+ reg('threews3d.optimizeModel', (target) => optimize(target));
+ reg('threews3d.compareWithHead', (target) => compareWithHead(target));
+ reg('threews3d.checkQuality', (target) => quality(target));
reg('threews3d.insertEmbed', (target) => insertEmbed(target));
- reg('threews3d.previewUrl', () => previewUrl(context));
+ reg('threews3d.previewUrl', (url) => previewUrl(context, url));
reg('threews3d.previewCreation', (item) => {
if (item?.glbUrl) openRemoteModel(context, links, output, item.glbUrl, title(item));
});
reg('threews3d.importGalleryItem', (node) => importCreation(node?.creation || node));
reg('threews3d.openInBrowser', (target) => openInBrowser(target));
reg('threews3d.copyModelUrl', (target) => copyModelUrl(target));
- reg('threews3d.saveSnapshot', () => {
- const panel = activeViewer();
- if (!panel) {
- vscode.window.showWarningMessage('three.ws 3D: open a model in the viewer first.');
- return;
- }
- panel.webview.postMessage({ type: 'snapshot' });
- });
+ reg('threews3d.saveSnapshot', () => withViewer((panel) => panel.threews.post({ type: 'snapshot' })));
+ reg('threews3d.exportTurntable', () => withViewer((panel) => panel.threews.post({ type: 'turntable' })));
+ reg('threews3d.toggleReport', () => withViewer((panel) => panel.threews.post({ type: 'toggle-report' })));
reg('threews3d.refreshModels', () => models.refresh());
reg('threews3d.refreshGallery', () => gallery.refresh());
@@ -91,6 +104,15 @@ function title(creation) {
return creation.prompt ? creation.prompt.slice(0, 48) : `creation ${creation.id.slice(0, 8)}`;
}
+function withViewer(fn) {
+ const panel = activeViewer();
+ if (!panel?.threews) {
+ vscode.window.showWarningMessage('three.ws 3D: open a model in the viewer first.');
+ return;
+ }
+ return fn(panel);
+}
+
// ---------------------------------------------------------------- generation
async function generate(kind) {
@@ -104,12 +126,7 @@ async function generate(kind) {
kind === 'avatar'
? 'a cheerful astronaut in a white suit with an orange visor'
: 'a friendly round robot mascot, glossy white plastic, big blue eyes',
- validateInput: (v) => {
- const value = v.trim();
- if (value.length < 3) return 'Describe the subject in at least 3 characters';
- if (value.length > 1000) return 'Keep the description under 1000 characters';
- return null;
- },
+ validateInput: validatePrompt,
});
if (!prompt) return;
@@ -138,7 +155,7 @@ async function generate(kind) {
token,
progress,
});
- await links.set(saved.uri, model.glbUrl);
+ await links.set(saved.uri, model.glbUrl, { prompt: prompt.trim() });
await openInViewer(saved.uri);
await offerFollowUp(saved.uri);
} catch (err) {
@@ -149,18 +166,27 @@ async function generate(kind) {
);
}
+function validatePrompt(v) {
+ const value = v.trim();
+ if (value.length < 3) return 'Describe the subject in at least 3 characters';
+ if (value.length > 1000) return 'Keep the description under 1000 characters';
+ return null;
+}
+
async function offerFollowUp(uri) {
const choice = await vscode.window.showInformationMessage(
`Saved ${vscode.workspace.asRelativePath(uri, false)}`,
- 'Insert embed',
+ 'Refine it',
'Rig for animation',
+ 'Insert embed',
);
if (choice === 'Insert embed') await vscode.commands.executeCommand('threews3d.insertEmbed', uri);
if (choice === 'Rig for animation') await vscode.commands.executeCommand('threews3d.rigModel', uri);
+ if (choice === 'Refine it') await vscode.commands.executeCommand('threews3d.refineModel', uri);
}
async function openInViewer(uri) {
- await vscode.commands.executeCommand('vscode.openWith', uri, 'threews3d.modelViewer');
+ await vscode.commands.executeCommand('vscode.openWith', uri, VIEW_TYPE);
}
// --------------------------------------------------------------------- rigging
@@ -195,8 +221,14 @@ async function rig(target) {
token,
progress,
});
- await links.set(saved.uri, rigged.glbUrl);
+ const meta = uriOf(target) ? links.meta(uriOf(target)) : null;
+ await links.set(saved.uri, rigged.glbUrl, { prompt: meta?.prompt || null });
await openInViewer(saved.uri);
+ const next = await vscode.window.showInformationMessage(
+ `Saved ${vscode.workspace.asRelativePath(saved.uri, false)}. It has a skeleton now.`,
+ 'Try an animation',
+ );
+ if (next) await vscode.commands.executeCommand('threews3d.animate', saved.uri);
} catch (err) {
if (token.isCancellationRequested) return;
report(err, 'rigging');
@@ -205,6 +237,360 @@ async function rig(target) {
);
}
+// ------------------------------------------------------------------ refining
+
+async function refine(target, presetInstruction = '') {
+ const uri = uriOf(target);
+ const url = await resolveUrl(target, {
+ title: 'Refine a model',
+ prompt: 'https URL of the model to refine',
+ });
+ if (!url) return;
+ const meta = uri ? links.meta(uri) : null;
+ const instruction = await vscode.window.showInputBox({
+ title: 'Refine this model',
+ prompt: meta?.prompt
+ ? `Describe the change to "${meta.prompt.slice(0, 60)}". The new version keeps its form and materials.`
+ : 'Describe the change. The new version is generated anchored to this one.',
+ placeHolder: 'make it metallic · bigger helmet · add a cape · lower poly, chunkier shapes',
+ value: presetInstruction || undefined,
+ validateInput: (v) => {
+ const value = v.trim();
+ if (value.length < 1) return 'Describe the change';
+ if (value.length > 500) return 'Keep the change under 500 characters';
+ return null;
+ },
+ });
+ if (!instruction) return;
+
+ await vscode.window.withProgress(
+ { location: vscode.ProgressLocation.Notification, title: `Refining: ${instruction.trim().slice(0, 40)}`, cancellable: true },
+ async (progress, token) => {
+ const controller = new AbortController();
+ token.onCancellationRequested(() => controller.abort());
+ progress.report({ message: 'generating a new version anchored to this one…' });
+ try {
+ const result = await refineModel(
+ origin(),
+ { glbUrl: url, instruction: instruction.trim(), parentPrompt: meta?.prompt, lineage: meta?.lineage },
+ { signal: controller.signal, onStatus: (message) => progress.report({ message }) },
+ );
+ output.appendLine(`${TOOLS.refine}: ${result.glbUrl} (version ${result.lineage.length} of the lineage)`);
+ progress.report({ message: 'saving into the workspace…' });
+ const stem = uri ? stemOf(uri) : slugFromUrl(url);
+ const saved = await downloadModel(result.glbUrl, {
+ prompt: `${stem}-v${result.lineage.length}`,
+ token,
+ progress,
+ });
+ await links.set(saved.uri, result.glbUrl, { prompt: result.prompt, lineage: result.lineage });
+ // The parent keeps the same lineage so a second refinement of it branches
+ // from the right version instead of forking a fresh history.
+ if (uri && meta) await links.set(uri, meta.url, { lineage: result.lineage });
+ await openInViewer(saved.uri);
+ const next = await vscode.window.showInformationMessage(
+ `Saved ${vscode.workspace.asRelativePath(saved.uri, false)} (version ${result.lineage.length}).`,
+ 'Refine again',
+ 'Compare with previous',
+ );
+ if (next === 'Refine again') await vscode.commands.executeCommand('threews3d.refineModel', saved.uri);
+ if (next === 'Compare with previous' && uri) await compareFiles(uri, saved.uri, 'previous version', `version ${result.lineage.length}`);
+ } catch (err) {
+ if (token.isCancellationRequested) return;
+ report(err, 'refining the model');
+ }
+ },
+ );
+}
+
+// ----------------------------------------------------------------- animating
+
+/** The library is fetched once per session; 2,800 rows is under a megabyte. */
+let libraryPromise = null;
+
+function library() {
+ if (!libraryPromise) {
+ libraryPromise = listLibrary(origin()).catch((err) => {
+ libraryPromise = null;
+ throw err;
+ });
+ }
+ return libraryPromise;
+}
+
+async function animate(target) {
+ const panel = await viewerFor(target);
+ if (!panel) return;
+ if (panel.threews.stats && !panel.threews.stats.bones) {
+ const pick = await vscode.window.showWarningMessage(
+ 'This model has no skeleton, so a library clip has nothing to drive. Rig it first.',
+ 'Rig for animation',
+ );
+ if (pick) await vscode.commands.executeCommand('threews3d.rigModel', target);
+ return;
+ }
+
+ const quickPick = vscode.window.createQuickPick();
+ quickPick.title = 'Try a library animation';
+ quickPick.placeholder = 'Type to search 2,800 clips: idle, walk, wave, dance, sit, punch, or describe a new motion';
+ quickPick.matchOnDescription = true;
+ quickPick.busy = true;
+ const describe = { label: '$(sparkle) Describe a motion in words…', alwaysShow: true, kind: 'text' };
+ quickPick.items = [describe];
+ quickPick.show();
+
+ let clips;
+ try {
+ clips = await library();
+ } catch (err) {
+ quickPick.hide();
+ report(err, 'loading the animation library');
+ return;
+ }
+ quickPick.busy = false;
+ quickPick.items = [
+ describe,
+ { label: '', kind: vscode.QuickPickItemKind.Separator },
+ ...clips.map((clip) => ({
+ label: clip.label,
+ description: `${clip.duration.toFixed(1)}s${clip.loop ? ' · loops' : ''}`,
+ clip,
+ })),
+ ];
+
+ const chosen = await new Promise((resolve) => {
+ quickPick.onDidAccept(() => resolve(quickPick.selectedItems[0]));
+ quickPick.onDidHide(() => resolve(undefined));
+ });
+ quickPick.dispose();
+ if (!chosen) return;
+ if (chosen.kind === 'text') return animateFromText(target);
+ await playOnViewer(panel, chosen.clip.url, { label: chosen.clip.label, loop: chosen.clip.loop });
+}
+
+async function animateFromText(target) {
+ const panel = await viewerFor(target);
+ if (!panel) return;
+ const prompt = await vscode.window.showInputBox({
+ title: 'Animate from a text prompt',
+ prompt: 'Describe the motion. A motion model samples it on the three.ws GPU fleet and the clip is retargeted onto this rig.',
+ placeHolder: 'waving confidently with the right hand · a slow tai-chi sweep · jumping for joy',
+ validateInput: validatePrompt,
+ });
+ if (!prompt) return;
+ await vscode.window.withProgress(
+ { location: vscode.ProgressLocation.Notification, title: `Animating: ${prompt.trim().slice(0, 40)}`, cancellable: true },
+ async (progress, token) => {
+ const controller = new AbortController();
+ token.onCancellationRequested(() => controller.abort());
+ progress.report({ message: 'queued on the GPU fleet…' });
+ try {
+ const { clip } = await generateMotion(origin(), prompt.trim(), {
+ signal: controller.signal,
+ onStatus: (message) => progress.report({ message }),
+ });
+ await playClip(panel, clip, { label: prompt.trim().slice(0, 60), loop: false });
+ } catch (err) {
+ if (token.isCancellationRequested) return;
+ report(err, 'text to animation');
+ }
+ },
+ );
+}
+
+async function playOnViewer(panel, clipUrl, { label, loop }) {
+ await vscode.window.withProgress(
+ { location: vscode.ProgressLocation.Window, title: `Retargeting "${label}"` },
+ async () => {
+ try {
+ const clip = await fetchClip(clipUrl);
+ await playClip(panel, clip, { label, loop });
+ } catch (err) {
+ report(err, 'loading the clip');
+ }
+ },
+ );
+}
+
+async function playClip(panel, clip, { label, loop }) {
+ const result = await playClipIn(panel, { clip, label, loop });
+ if (result.ok) {
+ const pct = Math.round(result.coverage * 100);
+ vscode.window.setStatusBarMessage(
+ `$(play) "${label}" is playing on ${result.matched} of ${result.total} bones (${pct}% coverage). Bake clip writes it into the file.`,
+ 8000,
+ );
+ return;
+ }
+ const pick = await vscode.window.showWarningMessage(
+ result.message
+ ? `three.ws 3D: ${result.message}`
+ : `This rig matched ${result.matched} of ${result.total} bones the clip drives (${Math.round(result.coverage * 100)}%), below the floor the retargeter accepts. Its bone names are not a humanoid convention the platform knows; rigging it through three.ws gives it one that is.`,
+ 'Rig for animation',
+ );
+ if (pick) await vscode.commands.executeCommand('threews3d.rigModel', panel.threews.resource || panel.threews.remoteUrl);
+}
+
+/** The viewer showing `target`, opening it if needed; the active viewer when there is no target. */
+async function viewerFor(target) {
+ const uri = uriOf(target);
+ const active = activeViewer();
+ if (!uri) {
+ if (active?.threews) return active;
+ vscode.window.showWarningMessage('three.ws 3D: open a model in the viewer first.');
+ return null;
+ }
+ if (active?.threews && sameTarget(active, uri)) return active;
+ if (uri.scheme === 'http' || uri.scheme === 'https') {
+ const panel = openRemoteModel(extensionContext, links, output, uri.toString(), slugFromUrl(uri.toString()));
+ await ready(panel);
+ return panel;
+ }
+ await openInViewer(uri);
+ const panel = activeViewer();
+ if (!panel?.threews) return null;
+ await ready(panel);
+ return panel;
+}
+
+function sameTarget(panel, uri) {
+ const t = panel.threews;
+ if (t.resource) return t.resource.toString() === uri.toString();
+ return t.remoteUrl === uri.toString();
+}
+
+/** Wait until the webview has loaded a model (its `loaded` message sets stats). */
+function ready(panel, timeoutMs = 60_000) {
+ return new Promise((resolve) => {
+ const started = Date.now();
+ const tick = () => {
+ if (panel.threews?.stats || Date.now() - started > timeoutMs) return resolve();
+ setTimeout(tick, 150);
+ };
+ tick();
+ });
+}
+
+// ---------------------------------------------------------------- optimizing
+
+async function optimize(target) {
+ const uri = await resolveLocal(target, 'optimize');
+ if (!uri) return;
+ const preset = await vscode.window.showQuickPick(
+ Object.entries(PRESETS).map(([id, p]) => ({ label: p.label, detail: p.detail, id })),
+ { title: 'Optimize for the web', placeHolder: 'A copy is written next to the model; the original is untouched.' },
+ );
+ if (!preset) return;
+ await vscode.window.withProgress(
+ { location: vscode.ProgressLocation.Notification, title: 'Optimizing the model' },
+ async (progress) => {
+ try {
+ progress.report({ message: 'dedup, prune, weld, resample, meshopt…' });
+ const bytes = await vscode.workspace.fs.readFile(uri);
+ const result = await optimizeGlb(bytes, { preset: preset.id });
+ const folder = vscode.Uri.joinPath(uri, '..');
+ const target = await freeName(folder, `${stemOf(uri)}.web`, '.glb');
+ await vscode.workspace.fs.writeFile(target, result.bytes);
+ const meta = links.meta(uri);
+ if (meta) await links.set(target, meta.url, { prompt: meta.prompt });
+ const savings = describeSavings(result.before, result.after);
+ output.appendLine(`optimized ${uri.fsPath} -> ${target.fsPath}: ${savings}`);
+ const pick = await vscode.window.showInformationMessage(
+ `Saved ${vscode.workspace.asRelativePath(target, false)}: ${savings}.`,
+ 'Open',
+ 'Compare',
+ );
+ if (pick === 'Open') await openInViewer(target);
+ if (pick === 'Compare') await compareFiles(uri, target, 'original', 'optimized');
+ } catch (err) {
+ report(err, 'optimizing the model');
+ }
+ },
+ );
+}
+
+// ------------------------------------------------------------------ comparing
+
+async function compareWithHead(target) {
+ const uri = await resolveLocal(target, 'compare');
+ if (!uri) return;
+ await vscode.window.withProgress(
+ { location: vscode.ProgressLocation.Notification, title: 'Comparing with the committed version' },
+ async () => {
+ try {
+ const [before, after] = await Promise.all([committedBytes(uri.fsPath), vscode.workspace.fs.readFile(uri)]);
+ if (sameBytes(before, after)) {
+ vscode.window.showInformationMessage(`${vscode.workspace.asRelativePath(uri, false)} is identical to the committed version.`);
+ return;
+ }
+ const { changeset, markdown } = await compareModels(before, after, { nameA: 'HEAD', nameB: 'working tree' });
+ const stash = vscode.Uri.joinPath(extensionContext.globalStorageUri, 'compare');
+ await vscode.workspace.fs.createDirectory(stash);
+ const headCopy = vscode.Uri.joinPath(stash, `${stemOf(uri)}@HEAD.glb`);
+ await vscode.workspace.fs.writeFile(headCopy, before);
+ await showMarkdown(markdown, `${stemOf(uri)}: HEAD vs working tree`);
+ openLocalModel(extensionContext, links, output, headCopy, `${path(uri)} @ HEAD`, vscode.ViewColumn.Beside);
+ const severity = changeset?.severity ? ` Severity: ${changeset.severity}.` : '';
+ vscode.window.setStatusBarMessage(`$(git-compare) ${path(uri)} compared with HEAD.${severity}`, 8000);
+ } catch (err) {
+ report(err, 'comparing with the committed version');
+ }
+ },
+ );
+}
+
+async function compareFiles(a, b, nameA, nameB) {
+ try {
+ const [before, after] = await Promise.all([vscode.workspace.fs.readFile(a), vscode.workspace.fs.readFile(b)]);
+ const { markdown } = await compareModels(before, after, { nameA, nameB });
+ await showMarkdown(markdown, `${path(a)} vs ${path(b)}`);
+ } catch (err) {
+ report(err, 'comparing the models');
+ }
+}
+
+function sameBytes(a, b) {
+ if (a.byteLength !== b.byteLength) return false;
+ for (let i = 0; i < a.byteLength; i++) if (a[i] !== b[i]) return false;
+ return true;
+}
+
+// -------------------------------------------------------------------- quality
+
+async function quality(target) {
+ const panel = await viewerFor(target);
+ if (!panel) return;
+ const uri = panel.threews.resource;
+ const meta = uri ? links.meta(uri) : null;
+ await vscode.window.withProgress(
+ { location: vscode.ProgressLocation.Notification, title: 'Checking quality', cancellable: true },
+ async (progress, token) => {
+ const controller = new AbortController();
+ token.onCancellationRequested(() => controller.abort());
+ try {
+ progress.report({ message: 'rendering the current view…' });
+ const render = await request(panel, { type: 'render' });
+ if (!render?.dataUrl) throw new Error(render?.message || 'the viewer produced no render');
+ progress.report({ message: 'a vision model is scoring it…' });
+ const { verdict } = await checkQuality(origin(), { image: render.dataUrl, prompt: meta?.prompt }, { signal: controller.signal });
+ const name = uri ? path(uri) : panel.title;
+ output.appendLine(`quality ${name}: ${verdict.qa_available ? `${verdict.pass ? 'pass' : 'fail'} ${verdict.score}` : 'unavailable'}`);
+ await showMarkdown(qualityMarkdown(verdict, { modelName: name, prompt: meta?.prompt }), `Quality: ${name}`);
+ if (verdict.qa_available && !verdict.pass && verdict.suggested_retry_hint && (uri || panel.threews.remoteUrl)) {
+ const pick = await vscode.window.showInformationMessage(
+ `Scored ${Math.round(verdict.score)}/100. ${verdict.suggested_retry_hint}`,
+ 'Refine with this hint',
+ );
+ if (pick) await refine(uri || vscode.Uri.parse(panel.threews.remoteUrl), verdict.suggested_retry_hint);
+ }
+ } catch (err) {
+ if (token.isCancellationRequested) return;
+ report(err, 'the quality check');
+ }
+ },
+ );
+}
+
// ---------------------------------------------------------------------- embed
async function insertEmbed(target) {
@@ -238,13 +624,16 @@ async function insertEmbed(target) {
// -------------------------------------------------------------------- viewing
-async function previewUrl(context) {
- const url = await vscode.window.showInputBox({
- title: 'Preview a model URL',
- prompt: 'https URL of a .glb or .gltf file',
- placeHolder: 'https://three.ws/avatars/default.glb',
- validateInput: (v) => (isHttpUrl(v) ? null : 'Enter a valid http(s) URL'),
- });
+async function previewUrl(context, given) {
+ const url =
+ typeof given === 'string' && isHttpUrl(given)
+ ? given
+ : await vscode.window.showInputBox({
+ title: 'Preview a model URL',
+ prompt: 'https URL of a .glb or .gltf file',
+ placeHolder: 'https://three.ws/avatars/default.glb',
+ validateInput: (v) => (isHttpUrl(v) ? null : 'Enter a valid http(s) URL'),
+ });
if (!url) return;
openRemoteModel(context, links, output, url.trim(), slugFromUrl(url));
}
@@ -260,7 +649,7 @@ async function importCreation(creation) {
token,
progress,
});
- await links.set(saved.uri, creation.glbUrl);
+ await links.set(saved.uri, creation.glbUrl, { prompt: creation.prompt || null });
await openInViewer(saved.uri);
} catch (err) {
if (token.isCancellationRequested) return;
@@ -291,6 +680,27 @@ async function copyModelUrl(target) {
// -------------------------------------------------------------------- helpers
+/** Show a Markdown document in the preview, beside the current editor. */
+async function showMarkdown(markdown, name) {
+ const doc = await vscode.workspace.openTextDocument({ language: 'markdown', content: `<!-- ${name} -->\n\n${markdown}` });
+ await vscode.commands.executeCommand('markdown.showPreviewToSide', doc.uri);
+}
+
+function uriOf(target) {
+ const uri = target?.resourceUri || target;
+ if (uri instanceof vscode.Uri) return uri;
+ if (typeof uri === 'string' && isHttpUrl(uri)) return vscode.Uri.parse(uri);
+ return null;
+}
+
+/** A model on disk for commands that rewrite bytes; falls back to the active viewer's file. */
+async function resolveLocal(target, verb) {
+ const uri = uriOf(target) || activeViewer()?.threews?.resource || null;
+ if (uri && uri.scheme !== 'http' && uri.scheme !== 'https') return uri;
+ vscode.window.showWarningMessage(`three.ws 3D: open a model file from the workspace to ${verb} it.`);
+ return null;
+}
+
/**
* Work out which hosted model a command should act on.
*
@@ -320,6 +730,22 @@ async function resolveUrl(target, { title: boxTitle, prompt }) {
return entered?.trim() || null;
}
+async function freeName(folder, stem, ext) {
+ const taken = await vscode.workspace.fs.readDirectory(folder).then(
+ (entries) => new Set(entries.map(([name]) => name.toLowerCase())),
+ () => new Set(),
+ );
+ return vscode.Uri.joinPath(folder, uniqueName(stem, ext, (name) => taken.has(name.toLowerCase())));
+}
+
+function path(uri) {
+ return uri.path.split('/').pop() || '';
+}
+
+function stemOf(uri) {
+ return path(uri).replace(/\.(glb|gltf)$/i, '');
+}
+
function isHttpUrl(value) {
try {
const url = new URL(String(value).trim());
@@ -331,6 +757,7 @@ function isHttpUrl(value) {
function report(err, what) {
const message = err?.message || String(err);
+ if (message === 'cancelled') return;
output.appendLine(`${what} failed: ${message}`);
vscode.window.showErrorMessage(`three.ws 3D: ${what} failed. ${message}`, 'Show log').then((pick) => {
if (pick === 'Show log') output.show(true);
diff --git a/packages/vscode-3d/src/shims/sharp.js b/packages/vscode-3d/src/shims/sharp.js
new file mode 100644
index 000000000..ea7e82499
--- /dev/null
+++ b/packages/vscode-3d/src/shims/sharp.js
@@ -0,0 +1,8 @@
+// Stand-in for `sharp`, which @gltf-transform/functions reaches through
+// ndarray-pixels for texture resizing. The extension never resizes textures (the
+// optimize presets are geometry and compression passes), and sharp is a native
+// module that cannot ship inside a .vsix, so the bundle aliases it here. Anything
+// that does call it gets a plain explanation instead of a missing-binary crash.
+export default function sharp() {
+ throw new Error('texture resizing is not available inside the editor; run the three.ws asset pipeline for that');
+}
diff --git a/packages/vscode-3d/src/studio.js b/packages/vscode-3d/src/studio.js
index 9452506fa..02f4e52cf 100644
--- a/packages/vscode-3d/src/studio.js
+++ b/packages/vscode-3d/src/studio.js
@@ -10,6 +10,8 @@ export const TOOLS = Object.freeze({
model: 'forge_free',
avatar: 'text_to_avatar',
rig: 'rig_mesh',
+ refine: 'refine_model',
+ check: 'check_job',
});
/**
diff --git a/packages/vscode-3d/src/viewer.js b/packages/vscode-3d/src/viewer.js
index 1cc91c13a..8627cbf49 100644
--- a/packages/vscode-3d/src/viewer.js
+++ b/packages/vscode-3d/src/viewer.js
@@ -2,12 +2,16 @@
// panel for models that live on a URL.
//
// The host owns the file: it reads the bytes, runs the inspector, and streams
-// the report to the webview. The webview owns the picture.
+// the report to the webview. The webview owns the picture, and sends back what
+// only a running scene can produce: a retargeted clip to bake, a turntable, a
+// render for the quality check.
import * as vscode from 'vscode';
import { reportFor } from './inspect.js';
-import { trackPanel } from './active-panel.js';
+import { notify, trackPanel } from './active-panel.js';
import { uniqueName } from './naming.js';
+import { writeClipIntoGlb } from './bake-clip.js';
+import { clipSlug } from './animations.js';
export const VIEW_TYPE = 'threews3d.modelViewer';
@@ -45,7 +49,6 @@ export class ModelViewerProvider {
localResourceRoots: [vscode.Uri.joinPath(this.context.extensionUri, 'media'), dir],
};
panel.webview.html = viewerHtml(panel.webview, this.context.extensionUri);
- trackPanel(panel);
const src = panel.webview.asWebviewUri(document.uri).toString();
const wire = attachViewer(this.context, panel, {
@@ -55,6 +58,7 @@ export class ModelViewerProvider {
output: this.output,
readBytes: () => vscode.workspace.fs.readFile(document.uri),
});
+ trackPanel(panel);
// Regenerate the model on disk and the open viewer catches up on its own.
const watcher = vscode.workspace.createFileSystemWatcher(
@@ -69,11 +73,11 @@ export class ModelViewerProvider {
}
/** Open a model that lives on an http(s) URL in its own panel. */
-export function openRemoteModel(context, links, output, url, title) {
+export function openRemoteModel(context, links, output, url, title, column = vscode.ViewColumn.Active) {
const panel = vscode.window.createWebviewPanel(
VIEW_TYPE,
title || 'three.ws 3D',
- vscode.ViewColumn.Active,
+ column,
{
enableScripts: true,
retainContextWhenHidden: true,
@@ -81,7 +85,6 @@ export function openRemoteModel(context, links, output, url, title) {
},
);
panel.webview.html = viewerHtml(panel.webview, context.extensionUri);
- trackPanel(panel);
const wire = attachViewer(context, panel, {
src: url,
remoteUrl: url,
@@ -98,6 +101,31 @@ export function openRemoteModel(context, links, output, url, title) {
return new Uint8Array(await res.arrayBuffer());
},
});
+ trackPanel(panel);
+ panel.onDidDispose(() => wire.dispose());
+ return panel;
+}
+
+/**
+ * Open a model that lives outside the workspace (a committed version pulled
+ * from git) beside the current editor.
+ */
+export function openLocalModel(context, links, output, uri, title, column = vscode.ViewColumn.Beside) {
+ const dir = vscode.Uri.joinPath(uri, '..');
+ const panel = vscode.window.createWebviewPanel(VIEW_TYPE, title, column, {
+ enableScripts: true,
+ retainContextWhenHidden: true,
+ localResourceRoots: [vscode.Uri.joinPath(context.extensionUri, 'media'), dir],
+ });
+ panel.webview.html = viewerHtml(panel.webview, context.extensionUri);
+ const wire = attachViewer(context, panel, {
+ src: panel.webview.asWebviewUri(uri).toString(),
+ resource: uri,
+ links,
+ output,
+ readBytes: () => vscode.workspace.fs.readFile(uri),
+ });
+ trackPanel(panel);
panel.onDidDispose(() => wire.dispose());
return panel;
}
@@ -113,9 +141,13 @@ function attachViewer(context, panel, { src, resource, remoteUrl, links, output,
if (!disposed) panel.webview.postMessage(message);
};
+ // What the commands need to know about this viewer.
+ panel.threews = { resource, remoteUrl, readBytes, stats: null, fileSize: 0, post, pending: new Map() };
+
const sendReport = async () => {
try {
const bytes = await readBytes();
+ panel.threews.fileSize = bytes.byteLength;
if (bytes.byteLength > MAX_INSPECT_BYTES) return;
const { rows, suggestions } = await reportFor(bytes);
post({ type: 'report', rows, suggestions });
@@ -128,10 +160,23 @@ function attachViewer(context, panel, { src, resource, remoteUrl, links, output,
if (msg?.type === 'ready') {
post({ type: 'load', src });
sendReport();
+ } else if (msg?.type === 'loaded') {
+ panel.threews.stats = msg.stats || null;
+ notify();
} else if (msg?.type === 'error') {
output.appendLine(`viewer: ${msg.message}`);
} else if (msg?.type === 'snapshot') {
- await saveSnapshot(msg.dataUrl, resource, output);
+ await saveBeside(msg.dataUrl, { resource, output, ext: '.png', suffix: '', what: 'Snapshot' });
+ } else if (msg?.type === 'turntable') {
+ await saveBeside(msg.dataUrl, { resource, output, ext: '.gif', suffix: '-turntable', what: 'Turntable' });
+ } else if (msg?.type === 'bake-clip') {
+ await bakeClip(msg, { resource, readBytes, links, output });
+ } else if (msg?.type === 'clip-result') {
+ const waiter = panel.threews.pending.get(msg.requestId);
+ if (waiter) {
+ panel.threews.pending.delete(msg.requestId);
+ waiter(msg);
+ }
} else if (msg?.type === 'action') {
await runAction(msg.action, { resource, remoteUrl });
}
@@ -151,46 +196,112 @@ function attachViewer(context, panel, { src, resource, remoteUrl, links, output,
async function runAction(action, { resource, remoteUrl }) {
const target = resource || (remoteUrl ? vscode.Uri.parse(remoteUrl) : undefined);
- if (action === 'openInBrowser') {
- await vscode.commands.executeCommand('threews3d.openInBrowser', target);
- } else if (action === 'embed') {
- await vscode.commands.executeCommand('threews3d.insertEmbed', target);
- } else if (action === 'rig') {
- await vscode.commands.executeCommand('threews3d.rigModel', target);
+ const commands = {
+ openInBrowser: 'threews3d.openInBrowser',
+ embed: 'threews3d.insertEmbed',
+ rig: 'threews3d.rigModel',
+ animate: 'threews3d.animate',
+ refine: 'threews3d.refineModel',
+ quality: 'threews3d.checkQuality',
+ optimize: 'threews3d.optimizeModel',
+ compare: 'threews3d.compareWithHead',
+ };
+ if (commands[action]) await vscode.commands.executeCommand(commands[action], target);
+}
+
+/**
+ * Ask the webview for something only the running scene can produce and wait
+ * for its answer (a `clip-result` message carrying the same requestId).
+ *
+ * @param {vscode.WebviewPanel} panel
+ * @param {object} message
+ * @param {number} [timeoutMs]
+ * @returns {Promise<any>}
+ */
+export function request(panel, message, timeoutMs = 30_000) {
+ const requestId = `${Date.now()}-${Math.random().toString(36).slice(2)}`;
+ return new Promise((resolve) => {
+ panel.threews.pending.set(requestId, resolve);
+ panel.threews.post({ ...message, requestId });
+ setTimeout(() => {
+ if (panel.threews.pending.delete(requestId)) {
+ resolve({ ok: false, coverage: 0, matched: 0, total: 0, message: 'the viewer did not answer' });
+ }
+ }, timeoutMs);
+ });
+}
+
+/**
+ * Send a clip to the webview and wait for it to say whether the rig took it.
+ * @param {vscode.WebviewPanel} panel
+ * @param {{ clip: object, label: string, loop: boolean }} payload
+ * @returns {Promise<{ ok: boolean, coverage: number, matched: number, total: number, message?: string }>}
+ */
+export function playClipIn(panel, payload) {
+ return request(panel, { type: 'play-clip', ...payload });
+}
+
+/** Write the retargeted clip into a copy of the model, next to it. */
+async function bakeClip(msg, { resource, readBytes, links, output }) {
+ const label = String(msg.label || msg.clip?.name || 'clip');
+ try {
+ const bytes = await readBytes();
+ const result = await writeClipIntoGlb(bytes, msg.clip, { name: label });
+ const folder = resource ? vscode.Uri.joinPath(resource, '..') : await defaultFolder();
+ const stem = `${resource ? path(resource).replace(/\.(glb|gltf)$/i, '') : 'model'}-${clipSlug(label)}`;
+ const target = await freeName(folder, stem, '.glb');
+ await vscode.workspace.fs.writeFile(target, result.bytes);
+ if (resource) {
+ const meta = links.meta(resource);
+ if (meta) await links.set(target, meta.url, { prompt: meta.prompt });
+ }
+ output.appendLine(`baked "${label}" into ${target.fsPath} (${result.channels} channels${result.dropped.length ? `, ${result.dropped.length} tracks had no node` : ''})`);
+ const open = await vscode.window.showInformationMessage(
+ `Saved ${path(target)} with the "${label}" clip baked in.`,
+ 'Open',
+ );
+ if (open === 'Open') await vscode.commands.executeCommand('vscode.openWith', target, VIEW_TYPE);
+ } catch (err) {
+ output.appendLine(`bake failed: ${err?.message || err}`);
+ vscode.window.showErrorMessage(`three.ws 3D: baking the clip failed. ${err?.message || err}`);
}
}
-/** Write the webview's PNG next to the model (or into the workspace root). */
-async function saveSnapshot(dataUrl, resource, output) {
- const match = /^data:image\/png;base64,(.+)$/.exec(String(dataUrl || ''));
+/** Write a data: URL image next to the model (or into the workspace root). */
+async function saveBeside(dataUrl, { resource, output, ext, suffix, what }) {
+ const match = /^data:image\/(png|gif);base64,(.+)$/.exec(String(dataUrl || ''));
if (!match) {
- vscode.window.showErrorMessage('three.ws 3D: the snapshot came back empty.');
+ vscode.window.showErrorMessage(`three.ws 3D: the ${what.toLowerCase()} came back empty.`);
return;
}
- const bytes = Buffer.from(match[1], 'base64');
- const folder = resource
- ? vscode.Uri.joinPath(resource, '..')
- : vscode.workspace.workspaceFolders?.[0]?.uri;
- if (!folder) {
- vscode.window.showErrorMessage('three.ws 3D: open a folder to save a snapshot into.');
+ const bytes = Buffer.from(match[2], 'base64');
+ let folder;
+ try {
+ folder = resource ? vscode.Uri.joinPath(resource, '..') : await defaultFolder();
+ } catch (err) {
+ vscode.window.showErrorMessage(`three.ws 3D: ${err.message}`);
return;
}
- const stem = resource ? path(resource).replace(/\.(glb|gltf)$/i, '') : 'model';
+ const stem = `${resource ? path(resource).replace(/\.(glb|gltf)$/i, '') : 'model'}${suffix}`;
+ const target = await freeName(folder, stem, ext);
+ await vscode.workspace.fs.writeFile(target, bytes);
+ output.appendLine(`${what.toLowerCase()} written to ${target.fsPath}`);
+ const open = await vscode.window.showInformationMessage(`${what} saved as ${path(target)}`, 'Open');
+ if (open === 'Open') await vscode.commands.executeCommand('vscode.open', target);
+}
+
+async function defaultFolder() {
+ const folder = vscode.workspace.workspaceFolders?.[0]?.uri;
+ if (!folder) throw new Error('open a folder to save into.');
+ return folder;
+}
+
+async function freeName(folder, stem, ext) {
const taken = await vscode.workspace.fs.readDirectory(folder).then(
(entries) => new Set(entries.map(([name]) => name.toLowerCase())),
() => new Set(),
);
- const target = vscode.Uri.joinPath(
- folder,
- uniqueName(stem, '.png', (name) => taken.has(name.toLowerCase())),
- );
- await vscode.workspace.fs.writeFile(target, bytes);
- output.appendLine(`snapshot written to ${target.fsPath}`);
- const open = await vscode.window.showInformationMessage(
- `Snapshot saved as ${path(target)}`,
- 'Open',
- );
- if (open === 'Open') await vscode.commands.executeCommand('vscode.open', target);
+ return vscode.Uri.joinPath(folder, uniqueName(stem, ext, (name) => taken.has(name.toLowerCase())));
}
function path(uri) {
@@ -208,6 +319,8 @@ export function viewerHtml(webview, extensionUri) {
environment: setting('environment', 'studio'),
showGrid: setting('showGrid', true),
autoRotate: setting('autoRotate', false),
+ turntableFrames: setting('turntableFrames', 36),
+ turntableSize: setting('turntableSize', 480),
};
// wasm-unsafe-eval: the meshopt, Draco, and Basis decoders are WebAssembly.
// blob: workers: Draco and KTX2 build their worker from a blob at runtime.
@@ -245,6 +358,11 @@ export function viewerHtml(webview, extensionUri) {
external buffers needs those files next to it in the workspace.</p>
</div>
+<div id="busy" hidden role="status" aria-live="polite">
+ <div class="spinner" aria-hidden="true"></div>
+ <span id="busy-label"></span>
+</div>
+
<aside id="report" hidden aria-label="Model report">
<h2>Model report</h2>
<dl id="report-body"></dl>
@@ -256,6 +374,7 @@ export function viewerHtml(webview, extensionUri) {
<button id="play" data-action="play" aria-pressed="true" title="Play or pause the animation">Pause</button>
<input id="scrub" type="range" min="0" max="1" step="0.001" value="0" aria-label="Animation time">
<select id="clips" hidden aria-label="Animation clip"></select>
+ <button id="bake" data-action="bake" hidden title="Write this library clip into a copy of the model so it plays in any engine">Bake clip</button>
</span>
<button data-toggle="grid" aria-pressed="${config.showGrid ? 'true' : 'false'}" title="Show the ground grid">Grid</button>
<button data-toggle="wireframe" aria-pressed="false" title="Draw the mesh as a wireframe">Wireframe</button>
@@ -263,7 +382,14 @@ export function viewerHtml(webview, extensionUri) {
<button data-toggle="rotate" aria-pressed="${config.autoRotate ? 'true' : 'false'}" title="Orbit the camera automatically">Rotate</button>
<button data-action="reset" title="Frame the model again">Reset view</button>
<button data-toggle="report" aria-pressed="false" title="Triangles, materials, textures, rig, and optimization notes">Report</button>
+ <span class="sep" aria-hidden="true"></span>
+ <button data-action="animate" title="Try any of 2,800 library animations on this rig, or describe a motion in words">Animate</button>
+ <button data-action="refine" title="Describe a change and generate a new version anchored to this model">Refine</button>
+ <button data-action="quality" title="Ask a vision model to score this render for realism and completeness">Check quality</button>
+ <button data-action="optimize" title="Dedup, weld, resample and meshopt-compress a copy for the web">Optimize</button>
+ <span class="sep" aria-hidden="true"></span>
<button data-action="snapshot" title="Save the current view as a PNG next to the model">Snapshot</button>
+ <button data-action="turntable" title="Save a looping turntable GIF next to the model">Turntable</button>
<button data-action="embed" title="Insert an &lt;agent-3d&gt; embed snippet">Embed</button>
<button data-action="rig" title="Add a humanoid skeleton with the three.ws rigger">Rig</button>
<button data-action="openInBrowser" title="Open this model in the three.ws viewer">three.ws</button>
diff --git a/packages/vscode-3d/test/activate.test.mjs b/packages/vscode-3d/test/activate.test.mjs
index 0d8bf0e50..a5140d0a0 100644
--- a/packages/vscode-3d/test/activate.test.mjs
+++ b/packages/vscode-3d/test/activate.test.mjs
@@ -24,6 +24,7 @@ const extension = require('../dist/extension.cjs');
const context = {
subscriptions: [],
extensionUri: vscode.Uri.file('/ext'),
+ globalStorageUri: vscode.Uri.file('/global'),
workspaceState: (() => {
const store = new Map();
return { get: (k, d) => (store.has(k) ? store.get(k) : d), update: (k, v) => (store.set(k, v), Promise.resolve()) };
@@ -70,10 +71,87 @@ test('the viewer webview is nonce-gated and loads only local assets', async () =
assert.ok(html.includes("'wasm-unsafe-eval'"), 'the wasm decoders would be blocked');
assert.ok(!/src="http(s)?:\/\/(?!fake\.vscode-cdn\.net)/.test(html), 'the webview loads a remote asset');
assert.ok(html.includes('id="toolbar"'));
+ for (const action of ['animate', 'refine', 'quality', 'optimize', 'turntable', 'bake', 'snapshot', 'embed', 'rig']) {
+ assert.ok(html.includes(`data-action="${action}"`), `toolbar lacks ${action}`);
+ }
assert.ok(panel.webview.options.enableScripts);
assert.equal(panel.webview.options.localResourceRoots.length, 2);
});
+test('the webview bundle carries the retargeter and the GIF encoder, and no CDN URL', () => {
+ const bundle = readFileSync(new URL('../media/viewer.js', import.meta.url), 'utf8');
+ assert.ok(bundle.includes('retargetClip'), 'the retargeter is not bundled');
+ assert.ok(bundle.includes('GIF89a'), 'the GIF encoder is not bundled');
+ assert.ok(!/https:\/\/(cdn|unpkg|cdnjs|jsdelivr)\./.test(bundle), 'the webview reaches a CDN');
+});
+
+test('the host bundle does not carry the native sharp module', () => {
+ const bundle = readFileSync(new URL('../dist/extension.cjs', import.meta.url), 'utf8');
+ assert.ok(!bundle.includes('node_modules/sharp/'), 'sharp was bundled into the extension host');
+ assert.ok(bundle.includes('texture resizing is not available inside the editor'));
+});
+
+test('language features register for HTML and the frameworks people embed from', () => {
+ const languages = new Set(state.providers.hovers[0].selector.map((s) => s.language));
+ for (const id of ['html', 'javascriptreact', 'typescriptreact', 'vue', 'svelte', 'astro', 'markdown']) {
+ assert.ok(languages.has(id), `${id} has no hover provider`);
+ }
+ assert.equal(state.providers.codeActions.length, 1);
+ assert.equal(state.providers.completions.length, 1);
+ assert.equal(state.providers.codeLenses.length, 1);
+ assert.equal(state.statusBarItems.length, 1);
+ assert.equal(state.statusBarItems[0].shown, false, 'the status bar shows with no viewer open');
+});
+
+test('opening an HTML file with a broken embed produces diagnostics with quick fixes', async () => {
+ const doc = new vscode.TextDocument(
+ vscode.Uri.file('/workspace/index.html'),
+ 'html',
+ '<script type="module" src="https://three.ws/agent-3d/latest/agent-3d.js"></script>\n<agent-3d modee="floating"></agent-3d>\n',
+ );
+ state.documents.push(doc);
+ state.openDocument.fire(doc);
+ // Diagnostics are debounced and the release lookup may hit the network; wait for either.
+ for (let i = 0; i < 60 && !state.diagnostics.has(doc.uri.toString()); i++) await new Promise((r) => setTimeout(r, 100));
+ const diagnostics = state.diagnostics.get(doc.uri.toString());
+ assert.ok(diagnostics, 'no diagnostics were published');
+ const codes = diagnostics.map((d) => d.code).sort();
+ assert.ok(codes.includes('no-source'), codes.join());
+ assert.ok(codes.includes('no-size'), codes.join());
+ assert.ok(codes.includes('unknown-attribute'), codes.join());
+ assert.ok(codes.includes('unpinned-library'), codes.join());
+ for (const d of diagnostics) assert.equal(d.source, 'three.ws');
+
+ const { provider } = state.providers.codeActions[0];
+ const actions = provider.provideCodeActions(doc, null, { diagnostics });
+ const titles = actions.map((a) => a.title);
+ assert.ok(titles.includes('Rename to mode'), titles.join(' | '));
+ assert.ok(titles.includes('Add an inline size (400×500)'), titles.join(' | '));
+ const rename = actions.find((a) => a.title === 'Rename to mode');
+ assert.equal(rename.edit.edits[0].text, 'mode');
+
+ const hover = state.providers.hovers[0].provider.provideHover(doc, doc.positionAt(doc.getText().indexOf('<agent-3d') + 3));
+ assert.match(hover.contents.value, /live three\.ws agent/);
+
+ const completions = state.providers.completions[0].provider.provideCompletionItems(doc, doc.positionAt(doc.getText().indexOf('modee')));
+ assert.ok(completions.some((c) => c.label === 'src'));
+ assert.ok(completions.some((c) => c.label === 'body'));
+
+ const lenses = state.providers.codeLenses[0].provider.provideCodeLenses(doc);
+ assert.ok(lenses.some((l) => l.command.title.includes('Pin the library version')));
+ assert.ok(lenses.some((l) => l.command.title === 'Embedding guide'));
+
+ state.closeDocument.fire(doc);
+ assert.ok(!state.diagnostics.has(doc.uri.toString()));
+});
+
+test('a document with no embed gets no diagnostics and is never scanned', async () => {
+ const doc = new vscode.TextDocument(vscode.Uri.file('/workspace/plain.html'), 'html', '<div>hello</div>');
+ state.openDocument.fire(doc);
+ await new Promise((r) => setTimeout(r, 400));
+ assert.ok(!state.diagnostics.has(doc.uri.toString()));
+});
+
test('the viewer hands the webview the model as soon as it is ready', async () => {
const provider = state.customEditors.get('threews3d.modelViewer');
const uri = vscode.Uri.file('/workspace/models/robot.glb');
diff --git a/packages/vscode-3d/test/animations.test.mjs b/packages/vscode-3d/test/animations.test.mjs
new file mode 100644
index 000000000..62c0dcd81
--- /dev/null
+++ b/packages/vscode-3d/test/animations.test.mjs
@@ -0,0 +1,164 @@
+// The library, clip fetch, and text to motion clients against a local stand-in
+// for the three.ws API, so the paging, polling, and error paths are pinned.
+
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { createServer } from 'node:http';
+import { clipSlug, fetchClip, generateMotion, listLibrary, normalizeClip } from '../src/animations.js';
+import { readRefineResult } from '../src/refine.js';
+import { checkQuality, qualityMarkdown } from '../src/quality.js';
+
+function serve(handler) {
+ return new Promise((resolve) => {
+ const server = createServer(handler);
+ server.listen(0, '127.0.0.1', () => {
+ const { port } = server.address();
+ resolve({ origin: `http://127.0.0.1:${port}`, close: () => new Promise((r) => server.close(r)) });
+ });
+ });
+}
+
+const CLIP = { name: 'mx-wave-abc', duration: 1, tracks: [{ name: 'Hips.quaternion', type: 'quaternion', times: [0], values: [0, 0, 0, 1] }] };
+
+test('normalizeClip keeps only fetchable rows and clipSlug strips the hash', () => {
+ assert.equal(normalizeClip({ label: 'X' }), null);
+ assert.deepEqual(normalizeClip({ name: 'mx-happy-idle-c9cd5a01b96c', label: 'Happy Idle', loop: true, duration: 2.5, url: 'https://cdn/x.json', thumb: 'https://cdn/x.webp' }), {
+ name: 'mx-happy-idle-c9cd5a01b96c',
+ label: 'Happy Idle',
+ loop: true,
+ duration: 2.5,
+ url: 'https://cdn/x.json',
+ thumb: 'https://cdn/x.webp',
+ });
+ assert.equal(clipSlug('Happy Idle'), 'happy-idle');
+ assert.equal(clipSlug('mx-happy-idle-c9cd5a01b96c'), 'happy-idle');
+ assert.equal(clipSlug('waving confidently with the right hand'), 'waving-confidently-with-the-right-hand');
+});
+
+test('listLibrary pages through the manifest until next_offset is null', async () => {
+ const seen = [];
+ const { origin, close } = await serve((req, res) => {
+ const url = new URL(req.url, origin);
+ const offset = Number(url.searchParams.get('offset'));
+ seen.push(offset);
+ const page = offset === 0
+ ? { clips: [{ name: 'a', label: 'A', url: 'https://cdn/a.json' }, { name: 'bad' }], total: 3, next_offset: 1000 }
+ : { clips: [{ name: 'b', label: 'B', url: 'https://cdn/b.json', loop: true }], total: 3, next_offset: null };
+ res.setHeader('content-type', 'application/json');
+ res.end(JSON.stringify(page));
+ });
+ try {
+ const clips = await listLibrary(origin);
+ assert.deepEqual(clips.map((c) => c.label), ['A', 'B']);
+ assert.deepEqual(seen, [0, 1000]);
+ } finally {
+ await close();
+ }
+});
+
+test('fetchClip rejects a clip with no tracks', async () => {
+ const { origin, close } = await serve((req, res) => {
+ res.setHeader('content-type', 'application/json');
+ res.end(JSON.stringify(req.url.includes('empty') ? { tracks: [] } : CLIP));
+ });
+ try {
+ assert.equal((await fetchClip(`${origin}/clip.json`)).name, 'mx-wave-abc');
+ await assert.rejects(fetchClip(`${origin}/empty.json`), /no animation tracks/);
+ } finally {
+ await close();
+ }
+});
+
+test('generateMotion starts a job, polls it, and returns the finished clip', async () => {
+ let polls = 0;
+ const statuses = [];
+ const { origin, close } = await serve((req, res) => {
+ res.setHeader('content-type', 'application/json');
+ if (req.method === 'POST') return res.end(JSON.stringify({ job_id: 'job-1', status: 'queued' }));
+ if (req.url.startsWith('/api/forge-motion?job=job-1')) {
+ polls++;
+ return res.end(JSON.stringify(polls < 2 ? { status: 'running' } : { status: 'done', clip_url: `${origin}/clip.json` }));
+ }
+ res.end(JSON.stringify(CLIP));
+ });
+ try {
+ const result = await generateMotion(origin, 'wave', { onStatus: (s) => statuses.push(s), pollMs: 20 });
+ assert.equal(result.clipUrl, `${origin}/clip.json`);
+ assert.equal(result.clip.name, 'mx-wave-abc');
+ assert.ok(statuses.some((s) => /sampling/.test(s)));
+ } finally {
+ await close();
+ }
+});
+
+test('generateMotion explains an unconfigured lane and a failed job', async () => {
+ let mode = 'unconfigured';
+ const { origin, close } = await serve((req, res) => {
+ res.setHeader('content-type', 'application/json');
+ if (mode === 'unconfigured') {
+ res.statusCode = 503;
+ return res.end(JSON.stringify({ error: 'unconfigured', message: 'Text-to-animation is not configured.' }));
+ }
+ if (req.method === 'POST') return res.end(JSON.stringify({ job_id: 'job-2' }));
+ res.end(JSON.stringify({ status: 'failed', error: 'the motion model rejected the prompt' }));
+ });
+ try {
+ await assert.rejects(generateMotion(origin, 'wave', { pollMs: 20 }), /not configured/);
+ mode = 'failing';
+ await assert.rejects(generateMotion(origin, 'wave', { pollMs: 20 }), /rejected the prompt/);
+ } finally {
+ await close();
+ }
+});
+
+test('readRefineResult understands finished, pending, and refused answers', () => {
+ const done = readRefineResult({
+ structuredContent: {
+ glbUrl: 'https://cdn/v2.glb',
+ prompt: 'a robot, metallic',
+ instruction: 'make it metallic',
+ lineage: [
+ { index: 0, parentIndex: null, glbUrl: 'https://cdn/v1.glb', prompt: 'a robot' },
+ { index: 1, parentIndex: 0, glbUrl: 'https://cdn/v2.glb', prompt: 'a robot, metallic', instruction: 'make it metallic' },
+ ],
+ },
+ });
+ assert.equal(done.pending, false);
+ assert.equal(done.glbUrl, 'https://cdn/v2.glb');
+ assert.equal(done.lineage.length, 2);
+ assert.equal(done.lineage[1].instruction, 'make it metallic');
+ assert.deepEqual(readRefineResult({ structuredContent: { status: 'pending', jobId: 'j' } }), { pending: true, jobId: 'j' });
+ assert.throws(() => readRefineResult({ isError: true, content: [{ type: 'text', text: 'That prompt is not allowed.' }] }), /not allowed/);
+});
+
+test('checkQuality posts the render and renders the verdict as Markdown', async () => {
+ let body = null;
+ const { origin, close } = await serve((req, res) => {
+ let raw = '';
+ req.on('data', (c) => (raw += c));
+ req.on('end', () => {
+ body = JSON.parse(raw);
+ res.setHeader('content-type', 'application/json');
+ res.end(JSON.stringify({
+ verdict: { pass: false, score: 42, realism: 40, completeness: 50, subject: 'robot', subject_detected: 'a toy robot', is_photoreal: false, defects: ['fused fingers'], reason: 'plastic look', suggested_retry_hint: 'brushed steel, worn edges', qa_available: true, provider: 'vertex', model: 'gemini' },
+ retry: { prompt: 'x' },
+ }));
+ });
+ });
+ try {
+ const { verdict } = await checkQuality(origin, { image: 'data:image/png;base64,AAAA', prompt: 'a robot' });
+ assert.equal(body.prompt, 'a robot');
+ assert.equal(body.image, 'data:image/png;base64,AAAA');
+ assert.equal(verdict.score, 42);
+ const md = qualityMarkdown(verdict, { modelName: 'robot.glb', prompt: 'a robot' });
+ assert.match(md, /\*\*Below the realism bar\*\* · score 42\/100/);
+ assert.match(md, /- fused fingers/);
+ assert.match(md, /brushed steel/);
+ assert.match(md, /Refine this Model/);
+ const unavailable = qualityMarkdown({ qa_available: false, reason: 'qa_unavailable: no provider' }, { modelName: 'x' });
+ assert.match(unavailable, /could not run/);
+ await assert.rejects(checkQuality(origin, { image: 'nope' }), /PNG, JPEG, or WebP/);
+ } finally {
+ await close();
+ }
+});
diff --git a/packages/vscode-3d/test/bake-clip.test.mjs b/packages/vscode-3d/test/bake-clip.test.mjs
new file mode 100644
index 000000000..94edf7655
--- /dev/null
+++ b/packages/vscode-3d/test/bake-clip.test.mjs
@@ -0,0 +1,85 @@
+// Baking writes real glTF animation channels into a real avatar. cz.glb is the
+// committed reference rig the clip library is authored against, and it ships
+// meshopt-compressed, so this also proves the encoder round trip.
+
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { readFileSync } from 'node:fs';
+import { WebIO } from '@gltf-transform/core';
+import { ALL_EXTENSIONS } from '@gltf-transform/extensions';
+import { MeshoptDecoder } from 'meshoptimizer';
+import { parseTrackName, sanitizeNodeName, writeClipIntoGlb } from '../src/bake-clip.js';
+
+const CZ = new URL('../../../public/avatars/cz.glb', import.meta.url);
+
+function clip(name = 'test-wave') {
+ return {
+ name,
+ duration: 1,
+ tracks: [
+ { name: 'Hips.position', type: 'vector', times: [0, 0.5, 1], values: [0, 1, 0, 0, 1.1, 0, 0, 1, 0] },
+ { name: 'Hips.quaternion', type: 'quaternion', times: [0, 1], values: [0, 0, 0, 1, 0, 0.707, 0, 0.707] },
+ { name: 'LeftArm.quaternion', type: 'quaternion', times: [0, 1], values: [0, 0, 0, 1, 0.1, 0, 0, 0.99], interpolation: 2300 },
+ { name: 'NoSuchBone.quaternion', type: 'quaternion', times: [0], values: [0, 0, 0, 1] },
+ { name: 'Wolf3D_Head.morphTargetInfluences[mouthOpen]', type: 'number', times: [0], values: [1] },
+ ],
+ };
+}
+
+test('sanitizeNodeName mirrors what GLTFLoader does to node names', () => {
+ assert.equal(sanitizeNodeName('mixamorig:Hips'), 'mixamorigHips');
+ assert.equal(sanitizeNodeName('Left Arm.001'), 'Left_Arm001');
+ assert.equal(sanitizeNodeName('Hips'), 'Hips');
+});
+
+test('parseTrackName splits node and property and rejects non-node tracks', () => {
+ assert.deepEqual(parseTrackName('Hips.quaternion'), { node: 'Hips', property: 'quaternion' });
+ assert.deepEqual(parseTrackName('Left_Arm001.position'), { node: 'Left_Arm001', property: 'position' });
+ assert.equal(parseTrackName('Head.morphTargetInfluences[smile]'), null);
+ assert.equal(parseTrackName('nodots'), null);
+});
+
+test('a clip becomes glTF channels on the matching nodes and survives a re-read', async () => {
+ const bytes = new Uint8Array(readFileSync(CZ));
+ const result = await writeClipIntoGlb(bytes, clip(), { name: 'test-wave' });
+ assert.equal(result.channels, 3);
+ assert.deepEqual(result.dropped, ['NoSuchBone.quaternion', 'Wolf3D_Head.morphTargetInfluences[mouthOpen]']);
+
+ await MeshoptDecoder.ready;
+ const io = new WebIO().registerExtensions(ALL_EXTENSIONS).registerDependencies({ 'meshopt.decoder': MeshoptDecoder });
+ const doc = await io.readBinary(result.bytes);
+ const anim = doc.getRoot().listAnimations().find((a) => a.getName() === 'test-wave');
+ assert.ok(anim, 'the animation was not written');
+ const channels = anim.listChannels();
+ assert.deepEqual(
+ channels.map((c) => `${c.getTargetNode().getName()}:${c.getTargetPath()}`).sort(),
+ ['Hips:rotation', 'Hips:translation', 'LeftArm:rotation'],
+ );
+ const step = channels.find((c) => c.getTargetNode().getName() === 'LeftArm').getSampler();
+ assert.equal(step.getInterpolation(), 'STEP');
+ const hipsPos = channels.find((c) => c.getTargetPath() === 'translation').getSampler();
+ assert.equal(hipsPos.getInput().getCount(), 3);
+ assert.equal(hipsPos.getOutput().getType(), 'VEC3');
+ // The original animation is still there and the meshes still decode.
+ assert.ok(doc.getRoot().listAnimations().length >= 2);
+ assert.ok(doc.getRoot().listMeshes().length > 0);
+});
+
+test('baking the same name again replaces the earlier take', async () => {
+ const bytes = new Uint8Array(readFileSync(CZ));
+ const once = await writeClipIntoGlb(bytes, clip('take'), { name: 'take' });
+ const twice = await writeClipIntoGlb(once.bytes, clip('take'), { name: 'take' });
+ await MeshoptDecoder.ready;
+ const io = new WebIO().registerExtensions(ALL_EXTENSIONS).registerDependencies({ 'meshopt.decoder': MeshoptDecoder });
+ const doc = await io.readBinary(twice.bytes);
+ assert.equal(doc.getRoot().listAnimations().filter((a) => a.getName() === 'take').length, 1);
+});
+
+test('a clip whose tracks match nothing is refused, not written as an empty animation', async () => {
+ const bytes = new Uint8Array(readFileSync(CZ));
+ await assert.rejects(
+ writeClipIntoGlb(bytes, { name: 'x', duration: 1, tracks: [{ name: 'Nope.quaternion', type: 'quaternion', times: [0], values: [0, 0, 0, 1] }] }),
+ /none of the clip tracks matched/,
+ );
+ await assert.rejects(writeClipIntoGlb(bytes, { name: 'x', tracks: [] }), /no tracks/);
+});
diff --git a/packages/vscode-3d/test/embed-language.test.mjs b/packages/vscode-3d/test/embed-language.test.mjs
new file mode 100644
index 000000000..3a89b8a1c
--- /dev/null
+++ b/packages/vscode-3d/test/embed-language.test.mjs
@@ -0,0 +1,154 @@
+// The embed rules run on strings, so each one is pinned here with the exact
+// text a developer would have in a page.
+
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import {
+ ATTRIBUTES,
+ SOURCE_ATTRIBUTES,
+ closest,
+ completionContext,
+ diagnose,
+ findEmbeds,
+ findLibraryScripts,
+} from '../src/embed-language.js';
+
+const RELEASE = { channel: '1.5.2', integrity: 'sha384-pinned' };
+
+test('findEmbeds parses attributes with offsets', () => {
+ const text = `<p>hi</p>\n<agent-3d body="https://x/y.glb" mode='floating' eager style="width:1px;height:2px"></agent-3d>`;
+ const [embed] = findEmbeds(text);
+ assert.ok(embed);
+ assert.equal(text.slice(embed.start, embed.tagEnd), '<agent-3d');
+ const names = embed.attrs.map((a) => a.name);
+ assert.deepEqual(names, ['body', 'mode', 'eager', 'style']);
+ const body = embed.attrs[0];
+ assert.equal(body.value, 'https://x/y.glb');
+ assert.equal(text.slice(body.start, body.nameEnd), 'body');
+ assert.equal(embed.attrs[2].value, null);
+});
+
+test('a tag with no source is an error, one with a source is not', () => {
+ const bad = diagnose('<agent-3d style="width:1px;height:1px"></agent-3d>');
+ assert.equal(bad.find((f) => f.code === 'no-source')?.severity, 'error');
+ for (const name of SOURCE_ATTRIBUTES) {
+ const ok = diagnose(`<agent-3d ${name}="x" style="width:1px;height:1px"></agent-3d>`);
+ assert.equal(ok.filter((f) => f.code === 'no-source').length, 0, name);
+ }
+});
+
+test('an unsized inline element warns and the fix inserts a size', () => {
+ const text = '<agent-3d body="https://x/y.glb"></agent-3d>';
+ const [f] = diagnose(text).filter((x) => x.code === 'no-size');
+ assert.ok(f);
+ const fixed = text.slice(0, f.fix.start) + f.fix.text + text.slice(f.fix.end);
+ assert.match(fixed, /^<agent-3d style="width: 400px; height: 500px; display: block;" body=/);
+ assert.equal(diagnose(fixed).filter((x) => x.code === 'no-size').length, 0);
+});
+
+test('size is satisfied by width+height attributes, a class, responsive, or a non-inline mode', () => {
+ for (const tag of [
+ '<agent-3d src="a" width="320px" height="420px">',
+ '<agent-3d src="a" class="hero">',
+ '<agent-3d src="a" className="hero">',
+ '<agent-3d src="a" responsive>',
+ '<agent-3d src="a" mode="floating">',
+ '<agent-3d src="a" mode="section">',
+ '<agent-3d src="a" style={{ width: 1 }}>',
+ ]) {
+ assert.equal(diagnose(tag).filter((x) => x.code === 'no-size').length, 0, tag);
+ }
+ assert.equal(diagnose('<agent-3d src="a" mode="inline">').filter((x) => x.code === 'no-size').length, 1);
+});
+
+test('unknown attributes are flagged with a suggestion; globals and JSX are not', () => {
+ const findings = diagnose('<agent-3d src="a" class="x" data-x="1" aria-label="y" onclick="f()" modee="floating" style="width:1px;height:1px">');
+ const unknown = findings.filter((x) => x.code === 'unknown-attribute');
+ assert.equal(unknown.length, 1);
+ assert.match(unknown[0].message, /Did you mean `mode`/);
+ assert.equal(unknown[0].fix.text, 'mode');
+ assert.equal(closest('brian'), 'brain');
+ assert.equal(closest('completelyunrelated'), null);
+});
+
+test('enumerated values are checked and an api key in HTML warns', () => {
+ const findings = diagnose('<agent-3d src="a" mode="popup" api-key="sk-live" style="width:1px;height:1px">');
+ assert.ok(findings.some((x) => x.code === 'bad-value' && /inline.*floating/.test(x.message)));
+ assert.ok(findings.some((x) => x.code === 'key-in-html'));
+ assert.equal(diagnose('<agent-3d src="a" mode={mode} style="width:1px;height:1px">').filter((x) => x.code === 'bad-value').length, 0);
+});
+
+test('library scripts are found on any host and channel', () => {
+ const text = [
+ '<script type="module" src="https://three.ws/agent-3d/latest/agent-3d.js"></script>',
+ '<script src="https://cdn.example.com/agent-3d/1.5.2/agent-3d.umd.cjs" integrity="sha384-old"></script>',
+ '<script src="https://unpkg.com/three.ws"></script>',
+ ].join('\n');
+ const scripts = findLibraryScripts(text);
+ assert.equal(scripts.length, 2);
+ assert.deepEqual(scripts.map((s) => [s.channel, s.exact, s.host]), [
+ ['latest', false, 'three.ws'],
+ ['1.5.2', true, 'cdn.example.com'],
+ ]);
+ assert.equal(scripts[1].integrity.value, 'sha384-old');
+});
+
+test('the latest channel warns and the fix pins the release with its hash', () => {
+ const text = '<script type="module" src="https://three.ws/agent-3d/latest/agent-3d.js"></script>';
+ const [f] = diagnose(text, { release: RELEASE }).filter((x) => x.code === 'unpinned-library');
+ assert.equal(f.severity, 'warning');
+ const fixed = text.slice(0, f.fix.start) + f.fix.text + text.slice(f.fix.end);
+ assert.match(fixed, /src="https:\/\/three\.ws\/agent-3d\/1\.5\.2\/agent-3d\.js"/);
+ assert.match(fixed, /integrity="sha384-pinned"/);
+ assert.match(fixed, /type="module"/);
+ assert.match(fixed, /crossorigin="anonymous"/);
+ assert.equal(diagnose(fixed, { release: RELEASE }).length, 0);
+});
+
+test('a minor channel is only a hint, and offline there is no pin fix', () => {
+ const text = '<script src="https://three.ws/agent-3d/1.5/agent-3d.js"></script>';
+ const [f] = diagnose(text, { release: RELEASE });
+ assert.equal(f.severity, 'hint');
+ assert.ok(f.fix);
+ const [offline] = diagnose(text);
+ assert.equal(offline.fix, null);
+});
+
+test('a stale integrity hash is an error with the published hash as the fix', () => {
+ const text = '<script src="https://three.ws/agent-3d/1.5.2/agent-3d.js" integrity="sha384-old" crossorigin="anonymous"></script>';
+ const [f] = diagnose(text, { release: RELEASE });
+ assert.equal(f.code, 'stale-integrity');
+ assert.equal(f.severity, 'error');
+ assert.equal(text.slice(f.fix.start, f.fix.end), 'integrity="sha384-old"');
+ assert.equal(f.fix.text, 'integrity="sha384-pinned"');
+ assert.equal(diagnose(text.replace('sha384-old', 'sha384-pinned'), { release: RELEASE }).length, 0);
+});
+
+test('a pinned script without integrity gets a hint, an older pin a newer-release hint', () => {
+ const [missing] = diagnose('<script src="https://three.ws/agent-3d/1.5.2/agent-3d.js"></script>', { release: RELEASE });
+ assert.equal(missing.code, 'missing-integrity');
+ assert.match(missing.fix.text, /integrity="sha384-pinned"/);
+ const [older] = diagnose('<script src="https://three.ws/agent-3d/1.4.0/agent-3d.js" integrity="sha384-x"></script>', { release: RELEASE });
+ assert.equal(older.code, 'newer-release');
+ assert.equal(older.severity, 'hint');
+ assert.equal(diagnose('<script src="https://three.ws/agent-3d/1.6.0/agent-3d.js" integrity="sha384-x"></script>', { release: RELEASE }).length, 0);
+});
+
+test('completion knows when the cursor is naming an attribute or choosing a value', () => {
+ const text = '<agent-3d src="a" mode="fl';
+ const value = completionContext(text, text.length);
+ assert.equal(value?.kind, 'value');
+ assert.equal(value.def.name, 'mode');
+ const naming = '<agent-3d src="a" ';
+ assert.equal(completionContext(naming, naming.length)?.kind, 'name');
+ const closed = '<agent-3d src="a" mo></agent-3d>';
+ assert.equal(completionContext(closed, closed.indexOf('mo') + 2)?.kind, 'name');
+ assert.equal(completionContext('<div class="x">', 12), null);
+});
+
+test('every documented attribute has docs and the element reads it', () => {
+ for (const a of ATTRIBUTES) {
+ assert.ok(a.doc.length > 10, a.name);
+ assert.match(a.name, /^[a-z][a-z-]*$/);
+ }
+});
diff --git a/packages/vscode-3d/test/fake-vscode.cjs b/packages/vscode-3d/test/fake-vscode.cjs
index 036cdb8f2..3532f23f6 100644
--- a/packages/vscode-3d/test/fake-vscode.cjs
+++ b/packages/vscode-3d/test/fake-vscode.cjs
@@ -77,6 +77,93 @@ class RelativePattern {
}
}
+class Position {
+ constructor(line, character) {
+ this.line = line;
+ this.character = character;
+ }
+}
+
+class Range {
+ constructor(start, end) {
+ this.start = start;
+ this.end = end;
+ }
+}
+
+class Diagnostic {
+ constructor(range, message, severity) {
+ this.range = range;
+ this.message = message;
+ this.severity = severity;
+ }
+}
+
+class CodeAction {
+ constructor(title, kind) {
+ this.title = title;
+ this.kind = kind;
+ }
+}
+
+class WorkspaceEdit {
+ constructor() {
+ this.edits = [];
+ }
+ replace(uri, range, text) {
+ this.edits.push({ uri, range, text });
+ }
+}
+
+class Hover {
+ constructor(contents, range) {
+ this.contents = contents;
+ this.range = range;
+ }
+}
+
+class CompletionItem {
+ constructor(label, kind) {
+ this.label = label;
+ this.kind = kind;
+ }
+}
+
+class SnippetString {
+ constructor(value) {
+ this.value = value;
+ }
+}
+
+class CodeLens {
+ constructor(range, command) {
+ this.range = range;
+ this.command = command;
+ }
+}
+
+/** A text document the language features can run over. */
+class TextDocument {
+ constructor(uri, languageId, text) {
+ this.uri = uri;
+ this.languageId = languageId;
+ this.text = text;
+ }
+ getText() {
+ return this.text;
+ }
+ offsetAt(position) {
+ const lines = this.text.split('\n');
+ let offset = 0;
+ for (let i = 0; i < position.line; i++) offset += lines[i].length + 1;
+ return offset + position.character;
+ }
+ positionAt(offset) {
+ const before = this.text.slice(0, offset).split('\n');
+ return new Position(before.length - 1, before[before.length - 1].length);
+ }
+}
+
function watcher() {
return {
onDidCreate: () => ({ dispose() {} }),
@@ -95,6 +182,14 @@ const state = {
config: {},
files: [],
fileBytes: new Map(),
+ statusBarItems: [],
+ diagnostics: new Map(),
+ providers: { codeActions: [], hovers: [], completions: [], codeLenses: [] },
+ documents: [],
+ openDocument: new EventEmitter(),
+ changeDocument: new EventEmitter(),
+ closeDocument: new EventEmitter(),
+ changeConfiguration: new EventEmitter(),
};
const vscode = {
@@ -104,12 +199,52 @@ const vscode = {
ThemeIcon,
MarkdownString,
RelativePattern,
+ Position,
+ Range,
+ Diagnostic,
+ CodeAction,
+ WorkspaceEdit,
+ Hover,
+ CompletionItem,
+ SnippetString,
+ CodeLens,
+ TextDocument,
+ Disposable: {
+ from: (...items) => ({ dispose: () => items.forEach((d) => d?.dispose?.()) }),
+ },
TreeItemCollapsibleState: { None: 0, Collapsed: 1, Expanded: 2 },
- ViewColumn: { Active: -1 },
- ProgressLocation: { Notification: 15 },
+ ViewColumn: { Active: -1, Beside: -2 },
+ ProgressLocation: { Notification: 15, Window: 10 },
ConfigurationTarget: { Global: 1 },
+ StatusBarAlignment: { Left: 1, Right: 2 },
+ DiagnosticSeverity: { Error: 0, Warning: 1, Information: 2, Hint: 3 },
+ CodeActionKind: { QuickFix: 'quickfix' },
+ CompletionItemKind: { Property: 9, EnumMember: 19 },
+ QuickPickItemKind: { Separator: -1, Default: 0 },
+ languages: {
+ createDiagnosticCollection(name) {
+ const collection = {
+ name,
+ set: (uri, items) => state.diagnostics.set(uri.toString(), items),
+ delete: (uri) => state.diagnostics.delete(uri.toString()),
+ dispose() {},
+ };
+ return collection;
+ },
+ registerCodeActionsProvider: (selector, provider) => (state.providers.codeActions.push({ selector, provider }), { dispose() {} }),
+ registerHoverProvider: (selector, provider) => (state.providers.hovers.push({ selector, provider }), { dispose() {} }),
+ registerCompletionItemProvider: (selector, provider) => (state.providers.completions.push({ selector, provider }), { dispose() {} }),
+ registerCodeLensProvider: (selector, provider) => (state.providers.codeLenses.push({ selector, provider }), { dispose() {} }),
+ },
window: {
activeTextEditor: undefined,
+ createStatusBarItem() {
+ const item = { shown: false, show: () => (item.shown = true), hide: () => (item.shown = false), dispose() {} };
+ state.statusBarItems.push(item);
+ return item;
+ },
+ setStatusBarMessage: () => ({ dispose() {} }),
+ createQuickPick: () => ({ items: [], show() {}, hide() {}, dispose() {}, onDidAccept: () => ({ dispose() {} }), onDidHide: () => ({ dispose() {} }) }),
createOutputChannel(name) {
const channel = { name, lines: [], appendLine: (l) => channel.lines.push(l), show() {}, dispose() {} };
state.outputs.push(channel);
@@ -155,6 +290,14 @@ const vscode = {
}),
findFiles: () => Promise.resolve(state.files),
createFileSystemWatcher: watcher,
+ get textDocuments() {
+ return state.documents;
+ },
+ onDidOpenTextDocument: (fn) => state.openDocument.event(fn),
+ onDidChangeTextDocument: (fn) => state.changeDocument.event(fn),
+ onDidCloseTextDocument: (fn) => state.closeDocument.event(fn),
+ onDidChangeConfiguration: (fn) => state.changeConfiguration.event(fn),
+ openTextDocument: ({ language, content }) => Promise.resolve(new TextDocument(Uri.parse(`untitled://doc-${Date.now()}`), language, content)),
asRelativePath: (uri) => String(uri.path || uri).replace('/workspace/', ''),
fs: {
stat: () => Promise.resolve({ size: 1024 }),
diff --git a/packages/vscode-3d/test/optimize-compare.test.mjs b/packages/vscode-3d/test/optimize-compare.test.mjs
new file mode 100644
index 000000000..b988433e3
--- /dev/null
+++ b/packages/vscode-3d/test/optimize-compare.test.mjs
@@ -0,0 +1,55 @@
+// Optimization and comparison run on committed avatars, so what is measured is
+// the real pipeline on real bytes.
+
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { readFileSync } from 'node:fs';
+import { fileURLToPath } from 'node:url';
+import { PRESETS, describeSavings, optimizeGlb } from '../src/optimize.js';
+import { committedBytes, compareModels } from '../src/compare.js';
+
+const CZ = new URL('../../../public/avatars/cz.glb', import.meta.url);
+const MICHELLE = new URL('../../../public/avatars/michelle.glb', import.meta.url);
+
+test('the balanced preset keeps geometry precision and writes meshopt', async () => {
+ const bytes = new Uint8Array(readFileSync(CZ));
+ const result = await optimizeGlb(bytes, { preset: 'balanced' });
+ assert.ok(result.bytes.byteLength > 1000);
+ assert.equal(result.before.fileSize, bytes.byteLength);
+ assert.equal(result.after.fileSize, result.bytes.byteLength);
+ assert.ok(result.after.extensions.includes('EXT_meshopt_compression'));
+ assert.ok(result.after.triangles > 0);
+ // cz.glb is already pipeline-optimized, so the pass must not bloat it much.
+ assert.ok(result.after.fileSize < result.before.fileSize * 1.15, `${result.after.fileSize} vs ${result.before.fileSize}`);
+});
+
+test('the compact preset quantizes and every preset is described', async () => {
+ const bytes = new Uint8Array(readFileSync(CZ));
+ const result = await optimizeGlb(bytes, { preset: 'compact' });
+ assert.ok(result.after.extensions.includes('KHR_mesh_quantization'));
+ for (const p of Object.values(PRESETS)) assert.ok(p.label && p.detail);
+});
+
+test('describeSavings reads as a sentence', () => {
+ assert.equal(
+ describeSavings({ fileSize: 12 * 1024 * 1024, vertices: 210_000 }, { fileSize: 3 * 1024 * 1024, vertices: 98_000 }),
+ '12.0 MB → 3.0 MB (−75%), 210k → 98.0k vertices',
+ );
+ assert.equal(describeSavings({ fileSize: 500 * 1024, vertices: 10 }, { fileSize: 500 * 1024, vertices: 10 }), '500 KB → 500 KB (−0%)');
+});
+
+test('compareModels reports a structural diff as Markdown', async () => {
+ const a = new Uint8Array(readFileSync(CZ));
+ const b = new Uint8Array(readFileSync(MICHELLE));
+ const { changeset, markdown } = await compareModels(a, b, { nameA: 'cz', nameB: 'michelle' });
+ assert.ok(changeset.severity);
+ assert.match(markdown, /\|/);
+ assert.ok(markdown.length > 200);
+});
+
+test('committedBytes reads the file straight out of git', async () => {
+ const path = fileURLToPath(CZ);
+ const bytes = await committedBytes(path);
+ assert.equal(bytes.byteLength, readFileSync(path).byteLength);
+ await assert.rejects(committedBytes(fileURLToPath(new URL('../../../public/avatars/does-not-exist.glb', import.meta.url))), /not committed|does not exist|exists on disk/);
+});
diff --git a/packages/vscode-3d/test/retarget-bake.test.mjs b/packages/vscode-3d/test/retarget-bake.test.mjs
new file mode 100644
index 000000000..6872862c3
--- /dev/null
+++ b/packages/vscode-3d/test/retarget-bake.test.mjs
@@ -0,0 +1,74 @@
+// The viewer retargets a library clip onto the open rig and hands the result to
+// the host to bake. This runs that same pair of calls in Node against the
+// committed reference avatar, so the round trip is proven without a browser:
+// canonical clip -> retargetClipToObject -> AnimationClip.toJSON -> writeClipIntoGlb.
+
+import test from 'node:test';
+import assert from 'node:assert/strict';
+import { readFileSync } from 'node:fs';
+import { AnimationClip } from 'three';
+import { WebIO } from '@gltf-transform/core';
+import { ALL_EXTENSIONS } from '@gltf-transform/extensions';
+import { MeshoptDecoder } from 'meshoptimizer';
+import { retargetClipToObject } from '../../../src/animation-retarget.js';
+import { parseGltfJson, buildBoneGraph } from '../../../tests/_helpers/glb-bone-graph.js';
+import { writeClipIntoGlb } from '../src/bake-clip.js';
+
+const CZ = new URL('../../../public/avatars/cz.glb', import.meta.url);
+
+/** A short clip on the canonical skeleton, the way every library clip is authored. */
+function canonicalClip() {
+ const q = (t) => [0, Math.sin(t / 2), 0, Math.cos(t / 2)];
+ return {
+ name: 'test-turn',
+ duration: 1,
+ tracks: [
+ { name: 'Hips.position', type: 'vector', times: [0, 1], values: [0, 1.0, 0, 0, 1.02, 0] },
+ { name: 'Hips.quaternion', type: 'quaternion', times: [0, 0.5, 1], values: [...q(0), ...q(0.4), ...q(0.8)] },
+ { name: 'Spine.quaternion', type: 'quaternion', times: [0, 1], values: [...q(0), ...q(0.1)] },
+ { name: 'LeftArm.quaternion', type: 'quaternion', times: [0, 1], values: [...q(0), ...q(0.3)] },
+ { name: 'RightArm.quaternion', type: 'quaternion', times: [0, 1], values: [...q(0), ...q(-0.3)] },
+ { name: 'Head.quaternion', type: 'quaternion', times: [0, 1], values: [...q(0), ...q(0.2)] },
+ ],
+ };
+}
+
+test('a canonical clip retargets onto cz.glb and the result bakes into the file', async () => {
+ const bytes = new Uint8Array(readFileSync(CZ));
+ const { root } = buildBoneGraph(parseGltfJson(bytes));
+ const clip = AnimationClip.parse(canonicalClip());
+ const result = retargetClipToObject(clip, root);
+ assert.ok(result.clip, `retarget refused the clip (coverage ${result.coverage})`);
+ assert.equal(result.coverage, 1, 'every bone the clip drives exists on the reference rig');
+
+ const json = AnimationClip.toJSON(result.clip);
+ assert.ok(json.tracks.length >= 6);
+ const baked = await writeClipIntoGlb(bytes, json, { name: 'test-turn' });
+ assert.equal(baked.channels, json.tracks.length);
+ assert.deepEqual(baked.dropped, []);
+
+ await MeshoptDecoder.ready;
+ const io = new WebIO().registerExtensions(ALL_EXTENSIONS).registerDependencies({ 'meshopt.decoder': MeshoptDecoder });
+ const doc = await io.readBinary(baked.bytes);
+ const anim = doc.getRoot().listAnimations().find((a) => a.getName() === 'test-turn');
+ const targets = new Set(anim.listChannels().map((c) => c.getTargetNode().getName()));
+ for (const bone of ['Hips', 'Spine', 'LeftArm', 'RightArm', 'Head']) assert.ok(targets.has(bone), `${bone} channel missing`);
+});
+
+test('a clip that drives bones the rig does not have is refused with its coverage', () => {
+ const bytes = new Uint8Array(readFileSync(CZ));
+ const { root } = buildBoneGraph(parseGltfJson(bytes));
+ const clip = AnimationClip.parse({
+ name: 'alien',
+ duration: 1,
+ tracks: [
+ { name: 'Tentacle1.quaternion', type: 'quaternion', times: [0], values: [0, 0, 0, 1] },
+ { name: 'Tentacle2.quaternion', type: 'quaternion', times: [0], values: [0, 0, 0, 1] },
+ { name: 'Hips.quaternion', type: 'quaternion', times: [0], values: [0, 0, 0, 1] },
+ ],
+ });
+ const result = retargetClipToObject(clip, root);
+ assert.equal(result.clip, null);
+ assert.ok(result.coverage < 0.5);
+ assert.equal(result.total, 3);
+});
diff --git a/packages/vscode-3d/webview/main.js b/packages/vscode-3d/webview/main.js
index 5575cbb35..1c459fa23 100644
--- a/packages/vscode-3d/webview/main.js
+++ b/packages/vscode-3d/webview/main.js
@@ -13,6 +13,8 @@ import { DRACOLoader } from 'three/addons/loaders/DRACOLoader.js';
import { KTX2Loader } from 'three/addons/loaders/KTX2Loader.js';
import { MeshoptDecoder } from 'three/addons/libs/meshopt_decoder.module.js';
import { RoomEnvironment } from 'three/addons/environments/RoomEnvironment.js';
+import { GIFEncoder, applyPalette, quantize } from 'gifenc';
+import { retargetClipToObject } from '../../../src/animation-retarget.js';
const vscode = acquireVsCodeApi();
const stage = document.getElementById('stage');
@@ -29,6 +31,9 @@ const els = {
report: document.getElementById('report'),
reportBody: document.getElementById('report-body'),
stats: document.getElementById('stats'),
+ bake: document.getElementById('bake'),
+ busy: document.getElementById('busy'),
+ busyLabel: document.getElementById('busy-label'),
};
const config = window.__THREEWS_CONFIG__ || {};
@@ -41,6 +46,9 @@ const state = {
wireframe: false,
playing: true,
radius: 1,
+ // Library clips retargeted onto this rig live beside the file's own clips;
+ // only these can be baked back into the model.
+ libraryClips: new Map(),
};
const renderer = new THREE.WebGLRenderer({ antialias: true, preserveDrawingBuffer: true });
@@ -141,6 +149,8 @@ function clearModel() {
state.mixer = null;
state.action = null;
state.clips = [];
+ state.libraryClips = new Map();
+ els.bake.hidden = true;
}
function materialsOf(mesh) {
@@ -235,18 +245,27 @@ function applyGrid(show) {
function setupAnimations(clips) {
state.clips = clips;
- els.playback.hidden = clips.length === 0;
- els.clips.innerHTML = '';
- if (!clips.length) return;
+ if (!clips.length) {
+ els.playback.hidden = true;
+ els.clips.innerHTML = '';
+ return;
+ }
state.mixer = new THREE.AnimationMixer(state.model);
- clips.forEach((clip, i) => {
+ rebuildClipPicker();
+ playClip(0);
+}
+
+function rebuildClipPicker() {
+ els.clips.innerHTML = '';
+ state.clips.forEach((clip, i) => {
const option = document.createElement('option');
option.value = String(i);
- option.textContent = `${clip.name || `clip ${i + 1}`} (${clip.duration.toFixed(2)}s)`;
+ const tag = state.libraryClips.has(clip) ? ' · library' : '';
+ option.textContent = `${clip.name || `clip ${i + 1}`} (${clip.duration.toFixed(2)}s)${tag}`;
els.clips.appendChild(option);
});
- els.clips.hidden = clips.length < 2;
- playClip(0);
+ els.playback.hidden = state.clips.length === 0;
+ els.clips.hidden = state.clips.length < 2;
}
function playClip(index) {
@@ -254,9 +273,64 @@ function playClip(index) {
if (!clip || !state.mixer) return;
state.action?.fadeOut(0.2);
state.action = state.mixer.clipAction(clip);
+ const library = state.libraryClips.get(clip);
+ if (library && !library.loop) {
+ state.action.setLoop(THREE.LoopOnce, 1);
+ state.action.clampWhenFinished = true;
+ } else {
+ state.action.setLoop(THREE.LoopRepeat, Infinity);
+ }
state.action.reset().fadeIn(0.2).play();
state.action.paused = !state.playing;
els.scrub.max = String(clip.duration);
+ els.clips.value = String(index);
+ els.bake.hidden = !library;
+}
+
+/**
+ * Retarget a library clip (authored on the canonical humanoid skeleton) onto
+ * the loaded model with the platform's retargeter, add it to the picker, and
+ * play it. Answers the host with the coverage so it can explain a refusal.
+ */
+function playLibraryClip({ requestId, clip, label, loop }) {
+ const reply = (result) => vscode.postMessage({ type: 'clip-result', requestId, ...result });
+ if (!state.model) return reply({ ok: false, coverage: 0, matched: 0, total: 0, message: 'no model is loaded' });
+ let parsed;
+ try {
+ parsed = THREE.AnimationClip.parse(clip);
+ } catch (err) {
+ return reply({ ok: false, coverage: 0, matched: 0, total: 0, message: `the clip could not be parsed: ${err?.message || err}` });
+ }
+ let result;
+ try {
+ result = retargetClipToObject(parsed, state.model);
+ } catch (err) {
+ return reply({ ok: false, coverage: 0, matched: 0, total: 0, message: err?.message || String(err) });
+ }
+ if (!result.clip) {
+ return reply({ ok: false, coverage: result.coverage, matched: result.matched, total: result.total });
+ }
+ result.clip.name = label || parsed.name;
+ if (!state.mixer) state.mixer = new THREE.AnimationMixer(state.model);
+ // Replace an earlier take of the same clip instead of piling up copies.
+ const previous = state.clips.findIndex((c) => state.libraryClips.has(c) && c.name === result.clip.name);
+ if (previous !== -1) {
+ state.libraryClips.delete(state.clips[previous]);
+ state.clips.splice(previous, 1);
+ }
+ state.clips.push(result.clip);
+ state.libraryClips.set(result.clip, { loop: Boolean(loop), label: result.clip.name });
+ rebuildClipPicker();
+ if (!state.playing) togglePlay();
+ playClip(state.clips.length - 1);
+ reply({ ok: true, coverage: result.coverage, matched: result.matched, total: result.total });
+}
+
+function bakeCurrentClip() {
+ const clip = state.action?.getClip();
+ const library = clip && state.libraryClips.get(clip);
+ if (!library) return;
+ vscode.postMessage({ type: 'bake-clip', clip: THREE.AnimationClip.toJSON(clip), label: library.label });
}
function sceneStats(gltf) {
@@ -346,9 +420,11 @@ els.toolbar.addEventListener('click', (event) => {
if (toggle === 'report') els.report.hidden = !on;
return;
}
- if (action === 'reset') resetView();
- if (action === 'snapshot') sendSnapshot();
- if (action === 'play') togglePlay();
+ if (action === 'reset') return resetView();
+ if (action === 'snapshot') return sendSnapshot();
+ if (action === 'turntable') return sendTurntable();
+ if (action === 'play') return togglePlay();
+ if (action === 'bake') return bakeCurrentClip();
if (action) vscode.postMessage({ type: 'action', action });
});
@@ -372,11 +448,107 @@ function sendSnapshot() {
vscode.postMessage({ type: 'snapshot', dataUrl: renderer.domElement.toDataURL('image/png') });
}
+/** A render sized for a vision model: the current view, at most 768px wide. */
+function sendQualityRender(requestId) {
+ renderer.render(scene, camera);
+ const source = renderer.domElement;
+ const scale = Math.min(1, 768 / Math.max(source.width, source.height));
+ const canvas = document.createElement('canvas');
+ canvas.width = Math.max(1, Math.round(source.width * scale));
+ canvas.height = Math.max(1, Math.round(source.height * scale));
+ canvas.getContext('2d').drawImage(source, 0, 0, canvas.width, canvas.height);
+ vscode.postMessage({ type: 'clip-result', requestId, ok: true, dataUrl: canvas.toDataURL('image/png') });
+}
+
+function setBusy(label) {
+ els.busy.hidden = !label;
+ els.busyLabel.textContent = label || '';
+}
+
+/**
+ * Render a full orbit around the model into a looping GIF.
+ *
+ * Frames render off-screen into a square target at the configured size so the
+ * result is independent of the panel's aspect ratio, and the animation (if one
+ * is playing) is stepped in lockstep so a walk cycle turns with the model.
+ */
+async function sendTurntable() {
+ if (!state.model) return;
+ const frames = Math.max(8, Math.min(120, Number(config.turntableFrames) || 36));
+ const size = Math.max(128, Math.min(1024, Number(config.turntableSize) || 480));
+ const delay = 60;
+ setBusy(`Rendering turntable… 0/${frames}`);
+
+ const target = new THREE.WebGLRenderTarget(size, size, { colorSpace: THREE.SRGBColorSpace });
+ const pixels = new Uint8Array(size * size * 4);
+ const flipped = new Uint8ClampedArray(size * size * 4);
+ const gif = GIFEncoder();
+ const orbit = new THREE.PerspectiveCamera(camera.fov, 1, camera.near, camera.far);
+ const focus = controls.target.clone();
+ const offset = camera.position.clone().sub(focus);
+ const radius = Math.hypot(offset.x, offset.z);
+ const startAngle = Math.atan2(offset.x, offset.z);
+ const wasPaused = state.action?.paused;
+ const clipDuration = state.action?.getClip().duration || 0;
+ const savedTime = state.action?.time || 0;
+
+ try {
+ for (let i = 0; i < frames; i++) {
+ const angle = startAngle + (i / frames) * Math.PI * 2;
+ orbit.position.set(focus.x + Math.sin(angle) * radius, camera.position.y, focus.z + Math.cos(angle) * radius);
+ orbit.lookAt(focus);
+ if (state.action && state.mixer) {
+ state.action.paused = false;
+ state.action.time = (savedTime + (i / frames) * clipDuration) % (clipDuration || 1);
+ state.mixer.update(0);
+ }
+ renderer.setRenderTarget(target);
+ renderer.render(scene, orbit);
+ renderer.readRenderTargetPixels(target, 0, 0, size, size, pixels);
+ renderer.setRenderTarget(null);
+ // GL reads bottom-up; GIF rows go top-down.
+ for (let y = 0; y < size; y++) {
+ flipped.set(pixels.subarray((size - 1 - y) * size * 4, (size - y) * size * 4), y * size * 4);
+ }
+ const palette = quantize(flipped, 256, { format: 'rgb565' });
+ const index = applyPalette(flipped, palette, 'rgb565');
+ gif.writeFrame(index, size, size, { palette, delay, repeat: 0 });
+ setBusy(`Rendering turntable… ${i + 1}/${frames}`);
+ // Yield so the busy label paints and the panel stays responsive.
+ await new Promise((r) => requestAnimationFrame(r));
+ }
+ gif.finish();
+ const bytes = gif.bytes();
+ let binary = '';
+ for (let i = 0; i < bytes.length; i += 0x8000) {
+ binary += String.fromCharCode.apply(null, bytes.subarray(i, i + 0x8000));
+ }
+ vscode.postMessage({ type: 'turntable', dataUrl: `data:image/gif;base64,${btoa(binary)}` });
+ } catch (err) {
+ vscode.postMessage({ type: 'error', message: `turntable failed: ${err?.message || err}` });
+ } finally {
+ target.dispose();
+ if (state.action) {
+ state.action.time = savedTime;
+ state.action.paused = Boolean(wasPaused);
+ state.mixer.update(0);
+ }
+ setBusy('');
+ }
+}
+
window.addEventListener('message', (event) => {
const msg = event.data;
if (msg?.type === 'load') load(msg.src);
if (msg?.type === 'report') renderReport(msg.rows || [], msg.suggestions || []);
if (msg?.type === 'snapshot') sendSnapshot();
+ if (msg?.type === 'turntable') sendTurntable();
+ if (msg?.type === 'play-clip') playLibraryClip(msg);
+ if (msg?.type === 'render') sendQualityRender(msg.requestId);
+ if (msg?.type === 'toggle-report') {
+ const button = els.toolbar.querySelector('[data-toggle="report"]');
+ button?.click();
+ }
});
const observer = new ResizeObserver(() => resize());