Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

feat(video-to-motion): launch a landing page that exports captured motion as a GLB, BVH, or FBX

nirholasaebfe1bc0327
diff --git a/CHANGELOG.md b/CHANGELOG.md
index accf3c67b..0fee87547 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -11,11 +11,13 @@ Public history for [three.ws](https://three.ws), newest first. New pages come fr
- **Docs: LiveKit and Pipecat avatar integrations** (`/docs/voice-framework-integrations`): How to drive a three.ws <agent-3d> avatar's lipsync and gestures from a LiveKit Agents or Pipecat voice pipeline, with the livekit-plugins-threews and pipecat-threews packages, runnable local examples, and the data-channel message contract both share.
- **Forge Room (Discord Activity)** (`/discord-activity`): A shared Forge room that runs embedded inside a Discord voice channel: everyone in the call prompts, watches the same 3D model generate, and votes together.
- **Open source credits** (`/open-source`): Every open-source project, GPU model, and asset source three.ws is built on, grouped by what it powers: rendering, formats and compression, generation models, motion, voice, assets, and infrastructure. Generated from package.json on every build.
+- **Video to Motion: Turn Any Video into an Animation File** (`/video-to-motion`): Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.
- **A Discord Activity brings a shared Forge room into any voice channel**: Open the new Activity from a Discord voice channel and everyone in the call shares one Forge room: one prompt, one in-progress generation, one finished model with a shared vote count, and an agent that reacts in text to what just happened. Auth maps a verified Discord identity (re-checked against Discord's own /users/@me, never trusted from the client) to a room-scoped session; generation and voting reuse the real /api/forge pipeline and forge_votes table rather than a separate throwaway pipeline. No coin, wallet, or purchase path. Docs: /docs/discord-activity. Submission packet for Developer Portal setup and Discovery copy: marketing/growth/submissions/discord-activity.md. (`/docs/discord-activity`) `[feature, docs]`
- **A three.ws avatar now lipsyncs to any LiveKit Agents or Pipecat voice pipeline**: Two new packages, livekit-plugins-threews and pipecat-threews, republish a running voice agent's TTS and user-turn frames as the same transcript/gesture/speech_started/speech_ended message shape over the room's data channel, which <agent-3d voice="livekit"> already renders client-side: no server-side video rendering, no change to the browser code at all. The LiveKit package is an AvatarSession plugin dropped into an AgentSession; the Pipecat package is a FrameProcessor placed after the TTS service. Each ships a runnable local example against a real open-source livekit-server with a real local TTS, verified end to end with real browser automation driving the actual avatar lipsync and gesture cues off a live bot in both frameworks. Docs: /docs/voice-framework-integrations. (`/docs/voice-framework-integrations`) `[feature, sdk, docs]`
- **Get it printed: live Craftcloud prices for any Forge model, buyer pays direct**: A new "Get it printed" action on model pages and in the Forge export menu uploads the model straight to Craftcloud (All3DP) and shows live, real prices across whichever of its roughly 186 print shops can run the job, cheapest first with shipping attached. This is not Materialize: three.ws never takes payment here. Craftcloud's public v5 API has no endpoint that hands a server-created cart back to a browser for checkout, so the panel links out to Craftcloud's own upload page instead of faking a continue-to-checkout button, where pasting the same STL reproduces the prices in an authenticated, checkout-capable session. Backed by a new fulfillment adapter, api/_lib/print/adapters/craftcloud.js, registered ahead of the manual adapter since quoting never spends. Docs: /docs/forge#get-it-printed-live-prices-from-craftcloud and /docs/materialize. (`/docs/forge#get-it-printed-live-prices-from-craftcloud`) `[feature, docs]`
- **Open source credits: every dependency, model, and asset source three.ws runs on, in one place**: A new /open-source page lists every third-party dependency, GPU worker model, and free asset source the platform runs on, grouped by area (rendering, formats and compression, generation models, motion, voice, assets, infrastructure) with license, upstream link, and what each one powers here. The dataset is generated from package.json and each worker's own README, not hand-maintained, and a guard test fails the build if a new direct dependency lands without a credited entry. Credits now also show up where the work actually happens: /objects and the object viewer name Poly Haven, the CC0 asset library the catalog sources from; the Forge export dialog names meshoptimizer and glTF-Transform; and the animation gallery shows each clip's source and license, including a fallback for studio-tier clips that lack an explicit manifest entry. Docs: /docs/open-source-credits. (`/open-source`) `[feature, docs]`
- **The public animation library is now openly licensed, every clip CC0 or self-generated**: GET /api/animations/library used to serve a bulk motion library baked from Mixamo, redistributable by anyone through scripts/mirror-animation-library.mjs even though Mixamo's own terms permit using its animations inside a project but not redistributing the raw files. The library is rebuilt from two CC0 1.0 Universal sources instead (Quaternius's Universal Animation Library 2 and Mesh2Motion's human animation packs), retargeted through the existing pipeline with every clip carrying a source and license field end to end from scripts/cc0-library.config.json through the published manifest. The /animations gallery shows each clip's pack and license next to its category chip, and the endpoint itself now refuses to publish any entry missing a license or naming Mixamo as its source, so a raw Mixamo clip can never reach the public endpoint or its mirror again. scripts/mixamo-all.mjs keeps its download and local-build phases for offline retargeting research, but its upload phase is permanently disabled. (`/docs/animations`) `[improvement, docs]`
+- **Video to Motion: download any video's motion as a GLB, BVH, or FBX**: A new page for creators who want the animation file itself, not a composited video: upload a video, preview the captured motion on a stock avatar or any rigged GLB you drop in, trim the in and out points, and download a GLB (clip baked onto the previewed avatar), a BVH (the platform's own canonical-skeleton proportions, the standard shape to re-retarget in a DCC tool), or an FBX (server-side Blender conversion that preserves the rig). Reuses the production model-video2motion worker that already powers /motion-swap for capture; signed-in users can save the clip straight to an agent's animation library. New writer: src/animation-bvh-export.js, round-tripped through three.js's own BVHLoader in tests/animation-bvh-export.test.js. Cross-linked from /motion-swap, /animations, and /mocap-studio. Docs: /docs/video-to-motion. (`/video-to-motion`) `[feature, docs]`
## 2026-10-12
diff --git a/STRUCTURE.md b/STRUCTURE.md
index c358fbeff..8ad358830 100644
--- a/STRUCTURE.md
+++ b/STRUCTURE.md
@@ -213,6 +213,7 @@ bottom.
| Data Desk | [pages/data-desk.html](pages/data-desk.html) + [src/data-desk.js](src/data-desk.js) `→ /data-desk`, API [api/data-desk.js](api/data-desk.js) `→ /api/data-desk`, writer [api/_lib/x402/fresh-workers/jobs.js](api/_lib/x402/fresh-workers/jobs.js) | Live at `/data-desk` | Live market data BOUGHT by the fresh-wallet workers with real USDC over x402: the newest purchase of each of 18 datasets (global snapshot, trending, gas, DeFi TVL, stablecoins, news pulse, fees, DEX volumes, mood, hacks, yields, DeFi radar, chains, crypto intel, market pulse, heatmap, $THREE intel, coin markets), each rendered with a slug-specific layout and its receipt (fresh wallet, price, Solscan link), a ledger of recent purchases, and the lane's 24 h numbers (wallets closed, USDC recirculated, SOL burned in fees, rent recycled). Honest feed: no synthetic rows; before the first purchase the page shows its designed empty state. Auto-refreshes every minute, `/` focuses the filter. |
| Agent-Forged Gallery | [pages/forged.html](pages/forged.html) + [src/forged-gallery.js](src/forged-gallery.js) → `/forged`, API [api/forged.js](api/forged.js), writer [api/_lib/x402/pipelines/forge-content.js](api/_lib/x402/pipelines/forge-content.js) | Live at `/forged` | 3D props BOUGHT by the autonomous x402 agents with real USDC: the hourly forge-content pipeline pays `/api/x402/forge` (draft tier), the generated GLB lands in `forge_autonomous_props` with payment provenance (payer wallet, price, Solana settlement signature), and `/forged` renders each prop with its on-chain receipt linked to Solscan. Honest feed: no synthetic entries; before the first paid generation the page shows its designed empty state. Category chips built from the eight families in [api/_lib/x402/pipelines/forge-catalog.js](api/_lib/x402/pipelines/forge-catalog.js) (club-decor, ar-object, diorama-set, avatar-item, vehicle, container, furniture, terrain), prompt search, newest/priciest/most-novel sorts. |
| Motion Swap (avatar body-swap in video) | [pages/motion-swap.html](pages/motion-swap.html) `→ /motion-swap` + [src/motion-swap.js](src/motion-swap.js), API [api/motion-swap.js](api/motion-swap.js), worker [workers/model-video2motion/](workers/model-video2motion) | Live | Upload a video of one person, get it back with a 3D avatar performing their motion. The worker (CPU Cloud Run, MediaPipe pose + hands + multiclass person segmentation, all Apache-2.0) tracks 33 body world landmarks plus 21 per hand each frame, solves them onto the canonical Wolf3D skeleton, body and all 30 finger bones, with an in-house bend-plane IK solver ([pose_solver.py](workers/model-video2motion/pose_solver.py), pure NumPy, unit-tested), and emits a standard `AnimationClip.toJSON()` + person-mask video + per-frame screen anchors. The browser composites in one WebGL scene: footage on a shader quad with the subject pixelated under the mask, avatar retargeted via [src/animation-retarget.js](src/animation-retarget.js) and pinned to the subject's anchor each frame, MediaRecorder export with original audio. Provider mode `video2motion` (`GCP_VIDEO2MOTION_URL`) in [api/_providers/gcp.js](api/_providers/gcp.js). Docs: [docs/motion-swap.md](docs/motion-swap.md). Tests: [test_pose_solver.py](workers/model-video2motion/test_pose_solver.py) (solver) and [test_service.py](workers/model-video2motion/test_service.py) (HTTP contract, ffmpeg normalize/mask encode, SSRF guard, live MediaPipe pass). |
+| Video to Motion (video → downloadable animation file) | [pages/video-to-motion.html](pages/video-to-motion.html) `→ /video-to-motion` + [src/video-to-motion.js](src/video-to-motion.js), reuses API [api/motion-swap.js](api/motion-swap.js) and worker [workers/model-video2motion/](workers/model-video2motion) for capture, export via [api/avatar/presign-glb.js](api/avatar/presign-glb.js) + [api/forge-remesh.js](api/forge-remesh.js) (FBX) | Live | For creators who want the animation file itself rather than a composited video: upload a video, preview the captured motion on a stock avatar or any rigged GLB you drop in or paste a URL for, trim the clip's in/out points, and download it as a GLB (clip retargeted onto the previewed avatar via [src/animation-retarget.js](src/animation-retarget.js) and baked with `GLTFExporter`), a BVH (the platform's own canonical-skeleton proportions, written by [src/animation-bvh-export.js](src/animation-bvh-export.js), the standard shape for re-retargeting in a DCC tool), and an FBX (server-side Blender conversion that preserves the rig, via `/api/forge-remesh`'s `operation:"convert"`; the exported GLB is hosted anonymously through `/api/avatar/presign-glb` first so the worker's SSRF-guarded `mesh_url` fetch can reach it). Signed-in users can save the clip straight to an agent's animation library (`PUT /api/agents/:id/animations`). Cross-linked from `/motion-swap`, `/animations`, and `/mocap-studio`. Docs: [docs/video-to-motion.md](docs/video-to-motion.md). Tests: [tests/animation-bvh-export.test.js](tests/animation-bvh-export.test.js) (BVH writer, round-tripped through three.js's own `BVHLoader`) and [tests/e2e/video-to-motion.spec.js](tests/e2e/video-to-motion.spec.js) (empty/processing/ready/failed/limit states at 320/768/1440px, a non-Mixamo Daz/Genesis rig retargeting in the browser, and the real exported GLB/BVH bytes). |
| Smart home (`/smart-home`) | Page [pages/smart-home.html](pages/smart-home.html) `→ /smart-home` and `/smart-home/:id` + [src/home/](src/home) (connect, manage, scene-model, scene-render, scene-fallback, satellite, satellite-link, pcm-player, plan, api). Store [api/_lib/home/store.js](api/_lib/home/store.js), runtime + backpressure [runtime.js](api/_lib/home/runtime.js) + [admission.js](api/_lib/home/admission.js), gate + tools [tools.js](api/_lib/home/tools.js) + [confirm.js](api/_lib/home/confirm.js), roster [members.js](api/_lib/home/members.js), privacy [privacy.js](api/_lib/home/privacy.js), plans + quotas [entitlements.js](api/_lib/home/entitlements.js) + [usage.js](api/_lib/home/usage.js), SSRF pin [api/_lib/home-url-guard.js](api/_lib/home-url-guard.js). HTTP [api/home/](api/home) (index, [id], [id]/call, [id]/activate, [id]/stream, [id]/grants, [id]/members, [id]/confirm, invites/[token], pair, pair/redeem, privacy, plan). Client [packages/home-bridge/](packages/home-bridge/README.md), MCP server [packages/home-mcp/](packages/home-mcp/README.md), hosted MCP tools [api/_mcp/tools/home.js](api/_mcp/tools/home.js). Dial-out for LAN-only houses [services/home-relay/](services/home-relay/README.md) + [home-assistant-integration/](home-assistant-integration) (HACS). Voice [services/home-satellite/](services/home-satellite/README.md) (Wyoming). Harness [scripts/home-test-instance.mjs](scripts/home-test-instance.mjs), unroutable-LAN relay rig [scripts/home-relay-live.mjs](scripts/home-relay-live.mjs) + [scripts/home-relay-e2e.mjs](scripts/home-relay-e2e.mjs) + [scripts/home-relay-gate-proof.mjs](scripts/home-relay-gate-proof.mjs), load fleet + envelope + chaos [scripts/home-fleet.mjs](scripts/home-fleet.mjs) / [scripts/home-load.mjs](scripts/home-load.mjs) / [scripts/home-chaos.mjs](scripts/home-chaos.mjs) (`npm run home:fleet`, `home:load`, `home:chaos`). Migrations `api/_lib/migrations/*_home_*.sql` | Live | The agent past the screen and into the building. **Zero device code, ever:** Home Assistant owns Zigbee, Z-Wave, Matter, Thread, BLE and 1,500 integrations, and three.ws is the layer in between: a connection record with the token encrypted at rest, a pooled per-home WebSocket, a room graph the 3D scene renders live, intent resolved to the household's own scenes, and one gate in front of everything. **The gate is the product decision:** reads are free, moves toward safety (lock, close, arm) run immediately and never prompt, and moves that OPEN the house (unlock, open a door/gate/garage, disarm) never run from a model call at all. They mint a pending confirmation a signed-in person redeems at `POST /api/home/:id/confirm`, which refuses every bearer principal including one holding `home:act`. It exists because Home Assistant's own `intent__HassTurnOff` is documented as performing an **unlock** on a lock, verified live. Entity and area names are untrusted input travelling in `structuredContent`, never interpolated into prose. Three reachability answers, all shipped: a remote https URL, a browser or MCP server inside the network, and the HACS integration that dials out so no Home Assistant credential ever leaves the house. Docs: [docs/smart-home.md](docs/smart-home.md), [docs/tutorials/connect-your-home.md](docs/tutorials/connect-your-home.md), [docs/home-households.md](docs/home-households.md), [docs/home-privacy.md](docs/home-privacy.md), [docs/home-plans.md](docs/home-plans.md), [docs/ops/home-operations.md](docs/ops/home-operations.md) (SLOs, the alerts, the measured scale envelope, the backpressure ladder, the seven chaos scenarios and the Cloud Run sizing). Evidence: [tasks/home/](tasks/home). Tests: `tests/home-*.test.js`, [tests/e2e/home-connect.spec.js](tests/e2e/home-connect.spec.js), [packages/home-bridge/tests/](packages/home-bridge/tests), [packages/home-mcp/tests/](packages/home-mcp/tests) |
| Sign Language (feature home) | [pages/sign-language.html](pages/sign-language.html) `→ /sign-language` + [src/sign-language-page.js](src/sign-language-page.js) | Live | The front door for signing avatars: a live avatar signing on arrival (PoseStage + [src/sign-speech.js](src/sign-speech.js) SignSpeaker, portrait-framed), an input that signs or spells anything you type (reporting which words were which), a playable vocabulary of every word with a real sign, and an embedded webcam sign-recognition demo. Controls for signing speed, left- vs right-handed signing, and which avatar signs: [src/sign-avatars.js](src/sign-avatars.js) owns the shared rig list, the one preference block every signing surface reads, and the Avatar pill group whose third pill opens the avatar gallery, so any avatar you own (or any public one) signs with the same vocabulary; a rig with no usable skeleton is rejected with a named message and the working rig is remounted. Verified in a real browser by `npm run verify:sign-avatar` ([scripts/verify-sign-avatar.mjs](scripts/verify-sign-avatar.mjs)). Engines: [src/sign-rig.js](src/sign-rig.js) (skeleton measured from the bind pose → forward kinematics, two-bone arm IK, and hand geometry, so a sign is authored as a place in signing space or as CONTACT, "fingertips on the palm", never as joint angles), [src/sign-handshapes.js](src/sign-handshapes.js) (A, Z, 0, 9, and the named shapes), [src/sign-clip.js](src/sign-clip.js) (resting signer, phase poser, non-manual face markers, eased timeline → clip JSON), [src/sign-dictionary.js](src/sign-dictionary.js) (the lexical vocabulary, one- and two-handed), [src/fingerspelling.js](src/fingerspelling.js) (the manual alphabet), [src/sign-speech.js](src/sign-speech.js) (text→one signing clip: hands rise once and lower once, signs where they exist and spelling for the rest), [src/sign-input.js](src/sign-input.js) (browser Holistic landmarks → text). Face markers ride a morph-target lane through [src/animation-retarget.js](src/animation-retarget.js), re-pointed at whichever meshes the target avatar owns. Tools: `npm run sign:sheet` (contact sheet of every sign, [scripts/sign-contact-sheet.mjs](scripts/sign-contact-sheet.mjs)), `npm run build:sign-goldens` (positional snapshot). Docs: [docs/sign-language.md](docs/sign-language.md). Tests: `tests/{sign-rig,sign-dictionary,sign-linguistics,sign-goldens,fingerspelling,sign-speech,sign-input}.test.js`. |
| ASL alphabet (learn + practice) | [pages/asl-alphabet.html](pages/asl-alphabet.html) `→ /asl-alphabet` + [src/asl-alphabet-page.js](src/asl-alphabet-page.js), letter notes [src/asl-alphabet-data.js](src/asl-alphabet-data.js) | Live | The manual alphabet as a reference you can turn around: every letter A-Z and digit 0-9 formed on a live portrait-framed rig ([src/fingerspelling.js](src/fingerspelling.js) via PoseStage), each with a written handshape description and the letters it is confused with (F/9, M/N, K/V, G/Q). A single letter holds its final frame (`playOnce({ settleTo: null })`) so it can be orbited; a word settles like a signer finishing. Typing any letter key signs it. Spelling a word lights each key in step with the hand, driven by the `marks` schedule the clip builder emits rather than a parallel timer. A practice drill spells a random letter or word and checks what you type, keeping a streak in localStorage. Shares speed / dominant-hand / avatar preferences with [/sign-language](pages/sign-language.html) under one key via [src/sign-avatars.js](src/sign-avatars.js), including a custom avatar picked from the gallery on either page. Deep links `?letter=W`, `?spell=HELLO`. Docs: [docs/sign-language.md](docs/sign-language.md). |
diff --git a/data/changelog.json b/data/changelog.json
index c0ab5067c..a25f32c22 100644
--- a/data/changelog.json
+++ b/data/changelog.json
@@ -1,6 +1,16 @@
{
"$comment": "Curated changelog entries for three.ws, the editorial layer of the public changelog. New PAGE launches are tracked automatically via the `added` field in data/pages.json; this file is for everything else holders care about: improvements, fixes, SDK releases, trading/payments capabilities, security work. scripts/build-page-index.mjs merges both sources into CHANGELOG.md, public/changelog.json, and public/changelog.xml. Newest first. Allowed tags: feature, improvement, fix, sdk, infra, docs, security, desktop (desktop entries become the desktop app's release notes). `link` is optional and must be a live three.ws path.",
"entries": [
+ {
+ "date": "2026-10-13",
+ "title": "Video to Motion: download any video's motion as a GLB, BVH, or FBX",
+ "link": "/video-to-motion",
+ "summary": "A new page for creators who want the animation file itself, not a composited video: upload a video, preview the captured motion on a stock avatar or any rigged GLB you drop in, trim the in and out points, and download a GLB (clip baked onto the previewed avatar), a BVH (the platform's own canonical-skeleton proportions, the standard shape to re-retarget in a DCC tool), or an FBX (server-side Blender conversion that preserves the rig). Reuses the production model-video2motion worker that already powers /motion-swap for capture; signed-in users can save the clip straight to an agent's animation library. New writer: src/animation-bvh-export.js, round-tripped through three.js's own BVHLoader in tests/animation-bvh-export.test.js. Cross-linked from /motion-swap, /animations, and /mocap-studio. Docs: /docs/video-to-motion.",
+ "tags": [
+ "feature",
+ "docs"
+ ]
+ },
{
"date": "2026-10-13",
"title": "Get it printed: live Craftcloud prices for any Forge model, buyer pays direct",
diff --git a/data/pages.json b/data/pages.json
index 0a3a8b69a..866abe36b 100644
--- a/data/pages.json
+++ b/data/pages.json
@@ -910,6 +910,14 @@
"changefreq": "monthly",
"added": "2026-07-19"
},
+ {
+ "path": "/video-to-motion",
+ "title": "Video to Motion: Turn Any Video into an Animation File",
+ "description": "Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.",
+ "priority": 0.7,
+ "changefreq": "monthly",
+ "added": "2026-10-13"
+ },
{
"path": "/forge-nim",
"title": "Forge · NIM: Self-Hosted Image to 3D",
@@ -5831,6 +5839,13 @@
"priority": 0.6,
"changefreq": "monthly"
},
+ {
+ "path": "/docs/video-to-motion",
+ "title": "Docs · Video to Motion",
+ "description": "Upload a video of a person and download the motion itself as a GLB, BVH, or FBX animation file, instead of a composited video. Preview on any rigged avatar, trim it, and save it to your animation library.",
+ "priority": 0.6,
+ "changefreq": "monthly"
+ },
{
"path": "/docs/multi-agent",
"title": "Docs · Multi-agent scenes",
diff --git a/docs/video-to-motion.md b/docs/video-to-motion.md
new file mode 100644
index 000000000..93d523a38
--- /dev/null
+++ b/docs/video-to-motion.md
@@ -0,0 +1,76 @@
+# Video to Motion: turn any video into an animation file
+
+**Page:** [/video-to-motion](https://three.ws/video-to-motion) · **API:** reuses `POST /api/motion-swap` for capture, `POST /api/avatar/presign-glb` + `POST /api/forge-remesh` for FBX export · **Worker:** `workers/model-video2motion`
+
+Upload a video of someone moving and get the motion itself back, not a
+composited video: a GLB, a BVH, and an FBX if your browser can build one.
+Built for animators, VTubers, and anyone whose online markerless mocap
+service closed and who still has footage to turn into a usable clip. If you
+want a video with your avatar performing the motion instead, see
+[/motion-swap](https://three.ws/motion-swap).
+
+## How to use it
+
+1. Open [/video-to-motion](https://three.ws/video-to-motion) and drop in a
+ video (MP4, MOV, or WebM, up to 256 MB; the first 90 seconds are
+ processed). One person, full body in frame, tracks best.
+2. Wait for capture (about two minutes for a 30-second clip; the same
+ worker and timing as Motion Swap). The server tracks pose in every frame.
+3. Preview the motion on a stock avatar, or load your own rigged humanoid
+ GLB by URL or file upload. The clip retargets onto whatever rig is
+ loaded, right in your browser.
+4. Trim the in/out points with the two sliders under the preview.
+5. Download: **GLB** carries the clip baked onto the avatar you previewed
+ on. **BVH** carries it on the platform's own canonical-skeleton
+ proportions, the standard shape to re-retarget in a DCC tool (Blender,
+ MotionBuilder, Unreal's IK Retargeter). **FBX** is a server-side
+ conversion that preserves the rig; it can take up to a minute.
+6. Signed in? **Save clip** attaches it to one of your agents' animation
+ libraries, the same store [/animations](https://three.ws/animations)
+ reads from.
+
+## How it works
+
+1. **Capture (server, shared with Motion Swap).** MediaPipe
+ PoseLandmarker, HandLandmarker, and ImageSegmenter track the subject and
+ solve the landmarks onto the canonical Wolf3D skeleton
+ (`workers/model-video2motion/pose_solver.py`), emitting a standard
+ three.js `AnimationClip.toJSON()` document: the same clip format the
+ [animation library](https://three.ws/animations) and Motion Swap use.
+2. **Preview and trim (browser).** [src/animation-retarget.js](../src/animation-retarget.js)
+ binds the clip onto whichever avatar is loaded (`retargetClipToObject`),
+ renaming non-canonical bone names first via
+ [src/glb-canonicalize.js](../src/glb-canonicalize.js) so Mixamo, Avaturn,
+ VRM, Daz, and MakeHuman rigs all work. Trimming (`trimClip`) re-samples
+ track boundaries through each track's own interpolant, so a cut between
+ keyframes still starts and ends on the right pose.
+3. **Export (browser + one server hop for FBX).** GLB is written with
+ three.js's own `GLTFExporter` against the previewed avatar's scene
+ graph. BVH is written by [src/animation-bvh-export.js](../src/animation-bvh-export.js)
+ directly from the canonical-skeleton clip, independent of whichever
+ avatar is on screen, using the same rest offsets
+ [src/animation-retarget.js](../src/animation-retarget.js) binds clips
+ against. FBX has no browser-side writer available under a permissive
+ license, so the exported GLB is uploaded anonymously through
+ `POST /api/avatar/presign-glb` (a public URL is required because the
+ converter fetches the source itself, SSRF-guarded) and converted
+ server-side by the Blender-based exporter behind
+ `POST /api/forge-remesh` (`operation: "convert"`,
+ `output_format: "fbx"`), the same converter the Forge export menu uses.
+
+## Why BVH is on the canonical skeleton, not the previewed avatar
+
+A BVH carries no mesh, only a skeleton and its motion. Mixamo's own BVH
+downloads use Mixamo's fixed skeleton proportions regardless of which
+character you previewed the clip on, because a BVH is meant to be
+re-retargeted in your own DCC tool rather than played back as-is. This page
+follows the same convention: the BVH always uses the platform's canonical
+rest pose, so it stays stable across avatar choices and matches what every
+other BVH-producing pipeline on three.ws already does.
+
+## Related
+
+- [/motion-swap](https://three.ws/motion-swap): the same capture, composited back into the source video instead of exported as a file
+- [/animations](https://three.ws/animations): the clip library a saved capture lands in
+- [/mocap-studio](https://three.ws/mocap-studio): live face and body mocap, not from a video file
+- [docs/motion-swap.md](motion-swap.md) · [docs/animations.md](animations.md)
diff --git a/pages/animations.html b/pages/animations.html
index 571c65341..d3722e768 100644
--- a/pages/animations.html
+++ b/pages/animations.html
@@ -758,6 +758,7 @@
</div>
<h1 data-i18n="animations.animation_gallery">Animation Gallery</h1>
<p data-i18n="animations.every_clip_in_the_three_ws_motion">Every clip in the three.ws motion library: mocap performances, studio presets, and community-authored animations. Hover any card for a live preview, open the details for playback controls, or send it straight to the Studio.</p>
+ <p data-i18n-html="animations.have_your_own_footage_turn_it_into">Have your own footage? Turn it into a clip at <a href="/video-to-motion">Video to Motion</a>, then save it straight into this library.</p>
<div class="ag-hero-actions">
<a href="/pose" class="ag-btn ag-btn--primary" data-i18n="animations.open_animation_studio">Open Animation Studio</a>
<a href="/gallery" class="ag-btn ag-btn--secondary" data-i18n="animations.avatar_gallery">Avatar Gallery</a>
diff --git a/pages/mocap-studio.html b/pages/mocap-studio.html
index 84724bb7a..5487ac0e4 100644
--- a/pages/mocap-studio.html
+++ b/pages/mocap-studio.html
@@ -476,6 +476,7 @@
<div class="page-eyebrow">Labs · Face Capture</div>
<h2 class="page-title" data-i18n="mocap_studio.mocap_studio">Mocap Studio</h2>
<p class="page-sub" data-i18n="mocap_studio.record_your_face_save_the_clip_replay">Record your face, save the clip, replay it on any avatar — yours or someone else's.</p>
+ <p class="page-sub" data-i18n-html="mocap_studio.already_have_a_video_turn_it_into">Already have a video of someone moving? Turn it into a downloadable animation file at <a href="/video-to-motion">Video to Motion</a>.</p>
</div>
<div id="auth-warning" class="auth-banner">
diff --git a/pages/motion-swap.html b/pages/motion-swap.html
index c4e874466..9fd09e43b 100644
--- a/pages/motion-swap.html
+++ b/pages/motion-swap.html
@@ -154,7 +154,8 @@
Upload a video of yourself — a house tour, a product demo, a talking walk-through — and get
it back with your <a href="/create">3D avatar</a> performing your exact motion instead of you.
Pose tracking runs server-side; you stay pixelated under the avatar. Compositing and export
- happen right here in your browser.
+ happen right here in your browser. Want the animation file itself instead of a composited
+ video? Try <a href="/video-to-motion">Video to Motion</a>.
</p>
</div>
diff --git a/pages/video-to-motion.html b/pages/video-to-motion.html
new file mode 100644
index 000000000..88db13b9a
--- /dev/null
+++ b/pages/video-to-motion.html
@@ -0,0 +1,311 @@
+<!doctype html>
+<html lang="en">
+ <head>
+ <script>/* three.ws theme boot, no-flash */(function(){try{var m=localStorage.getItem('twx_theme');var l=m==='auto'?(window.matchMedia&&window.matchMedia('(prefers-color-scheme: light)').matches):m==='light';document.documentElement.setAttribute('data-theme',l?'light':'dark');}catch(e){document.documentElement.setAttribute('data-theme','dark');}})();</script>
+ <meta charset="utf-8" />
+ <meta http-equiv="X-UA-Compatible" content="IE=edge" />
+ <title data-i18n="video_to_motion.meta_title">Video to Motion: Turn Any Video into an Animation File · three.ws</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0" />
+ <meta name="description" content="Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library." data-i18n-attr="content:video_to_motion.meta_description" />
+ <meta name="theme-color" content="#000000" />
+ <link rel="canonical" href="https://three.ws/video-to-motion" />
+
+ <meta property="og:type" content="website" />
+ <meta property="og:site_name" content="three.ws" />
+ <meta property="og:title" content="Video to Motion: Turn Any Video into an Animation File · three.ws" data-i18n-attr="content:video_to_motion.meta_og_title" />
+ <meta property="og:description" content="Turn any video into a downloadable animation file: GLB, BVH, or FBX. Preview on any rigged avatar, trim it, save it to your library." data-i18n-attr="content:video_to_motion.meta_og_description" />
+ <meta property="og:url" content="https://three.ws/video-to-motion" />
+ <meta property="og:image" content="https://three.ws/api/page-og?s=build&amp;t=Video+to+Motion%3A+Turn+Any+Video+into+an+Animation+File&amp;d=Upload+a+video+of+a+person+and+download+the+motion+as+a+GLB%2C+BVH%2C+or+FBX+animation+file&amp;p=%2Fvideo-to-motion" />
+ <meta name="twitter:card" content="summary_large_image" />
+ <meta name="twitter:site" content="@trythreews" />
+ <meta name="twitter:title" content="Video to Motion: Turn Any Video into an Animation File · three.ws" data-i18n-attr="content:video_to_motion.meta_twitter_title" />
+ <meta name="twitter:description" content="Turn any video into a downloadable animation file: GLB, BVH, or FBX. Preview on any rigged avatar, trim it, save it to your library." data-i18n-attr="content:video_to_motion.meta_twitter_description" />
+ <meta name="twitter:image" content="https://three.ws/api/page-og?s=build&amp;t=Video+to+Motion%3A+Turn+Any+Video+into+an+Animation+File&amp;d=Upload+a+video+of+a+person+and+download+the+motion+as+a+GLB%2C+BVH%2C+or+FBX+animation+file&amp;p=%2Fvideo-to-motion" />
+
+ <link rel="icon" href="/favicon.ico" sizes="any" />
+ <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
+ <link rel="apple-touch-icon" href="/apple-touch-icon.png" />
+ <link rel="manifest" href="/manifest.webmanifest" />
+ <link rel="stylesheet" href="/style.css" />
+ <link rel="stylesheet" href="/nav.css" />
+ <link rel="stylesheet" href="/footer.css" />
+ <link rel="stylesheet" href="/fonts/fonts.css" />
+
+ <style>
+ .vtm-page {
+ min-height: 100vh;
+ background: var(--bg, #060606);
+ color: var(--text, #f6f6f6);
+ font-family: var(--font-body, 'Inter', system-ui, sans-serif);
+ -webkit-font-smoothing: antialiased;
+ }
+ .vtm-wrap { max-width: 1180px; margin: 0 auto; padding: 40px 24px 72px; }
+ .vtm-chip {
+ display: inline-flex; align-items: center; gap: 7px;
+ padding: 4px 11px; background: var(--surface-1, #141414);
+ border: 1px solid var(--hairline, #1f1f1f); border-radius: 20px;
+ font-size: 11px; color: var(--text-2, #b4b4b4); letter-spacing: 0.06em;
+ text-transform: uppercase; margin-bottom: 16px;
+ }
+ .vtm-chip-dot { width: 6px; height: 6px; border-radius: 50%; background: #6ee7a2; }
+ .vtm-head h1 { font-size: clamp(28px, 4.5vw, 44px); margin: 0 0 12px; letter-spacing: -0.02em; }
+ .vtm-head p { max-width: 680px; color: var(--text-2, #b4b4b4); line-height: 1.6; margin: 0 0 28px; }
+ .vtm-head a { color: var(--text, #f6f6f6); }
+
+ .vtm-stage {
+ position: relative; border: 1px solid var(--hairline, #1f1f1f);
+ border-radius: 14px; background: var(--surface-0, #0b0b0b); overflow: hidden;
+ }
+ #vtm-canvas-wrap { width: 100%; min-height: 320px; display: flex; justify-content: center; }
+ #vtm-canvas { max-width: 100%; height: auto; display: block; }
+
+ .vtm-overlay {
+ position: absolute; inset: 0; display: flex; flex-direction: column;
+ align-items: center; justify-content: center; text-align: center;
+ padding: 32px; gap: 10px; background: var(--surface-0, #0b0b0b);
+ }
+ .vtm-overlay[hidden] { display: none; }
+ .vtm-overlay h3 { margin: 0; font-size: 18px; }
+ .vtm-overlay p { margin: 0; color: var(--text-2, #b4b4b4); max-width: 460px; line-height: 1.55; font-size: 14px; }
+
+ .vtm-dropzone {
+ border: 1.5px dashed var(--hairline-2, #2c2c2c); border-radius: 12px;
+ padding: 42px 32px; cursor: pointer; transition: border-color var(--duration-fast), background var(--duration-fast);
+ display: flex; flex-direction: column; align-items: center; gap: 10px; width: min(520px, 90%);
+ }
+ .vtm-dropzone:hover, .vtm-dropzone:focus-visible, .vtm-dropzone.is-drag {
+ border-color: var(--text-2, #b4b4b4); background: var(--surface-1, #121212); outline: none;
+ }
+ .vtm-glyph {
+ width: 46px; height: 46px; border-radius: 12px; display: grid; place-items: center;
+ background: var(--surface-1, #161616); border: 1px solid var(--hairline, #1f1f1f);
+ }
+ .vtm-spinner {
+ width: 26px; height: 26px; border-radius: 50%;
+ border: 2.5px solid var(--hairline-2, #2c2c2c); border-top-color: var(--text, #f6f6f6);
+ animation: vtm-spin 0.9s linear infinite;
+ }
+ @keyframes vtm-spin { to { transform: rotate(360deg); } }
+ .vtm-progress {
+ width: min(320px, 90%); height: 4px; border-radius: 2px; background: var(--surface-1, #161616);
+ overflow: hidden; margin-top: 4px;
+ }
+ .vtm-progress > span { display: block; height: 100%; background: #6ee7a2; width: 0%; transition: width 0.3s ease; }
+
+ .vtm-layout { display: grid; grid-template-columns: minmax(0, 1fr) 300px; gap: 20px; margin-top: 26px; }
+ @media (max-width: 900px) { .vtm-layout { grid-template-columns: 1fr; } }
+
+ .vtm-card {
+ border: 1px solid var(--hairline, #1f1f1f); border-radius: 12px;
+ background: var(--surface-0, #0d0d0d); padding: 18px;
+ }
+ .vtm-card + .vtm-card { margin-top: 16px; }
+ .vtm-card h2 { margin: 0 0 10px; font-size: 13px; text-transform: uppercase; letter-spacing: 0.07em; color: var(--text-2, #b4b4b4); }
+ .vtm-note { color: var(--text-2, #b4b4b4); font-size: 13.5px; line-height: 1.6; margin: 0; }
+ .vtm-note + .vtm-note { margin-top: 8px; }
+ .vtm-note code { font-size: 12px; }
+
+ .vtm-controls { display: flex; flex-wrap: wrap; gap: 8px; align-items: center; padding: 12px 14px; border-top: 1px solid var(--hairline, #1f1f1f); }
+ .vtm-trim { display: flex; flex-wrap: wrap; align-items: center; gap: 10px; padding: 10px 14px; border-top: 1px solid var(--hairline, #1f1f1f); font-size: 12.5px; color: var(--text-2, #b4b4b4); }
+ .vtm-trim input[type='range'] { flex: 1; min-width: 120px; accent-color: #6ee7a2; }
+ .vtm-trim output { font-variant-numeric: tabular-nums; min-width: 42px; text-align: right; }
+
+ .vtm-btn {
+ appearance: none; border: 1px solid var(--hairline-2, #2c2c2c); border-radius: 8px;
+ background: var(--surface-1, #151515); color: var(--text, #f6f6f6);
+ padding: 8px 14px; font-size: 13px; cursor: pointer;
+ transition: background var(--duration-fast), border-color var(--duration-fast), transform var(--duration-instant);
+ }
+ .vtm-btn:hover { background: var(--surface-2, #1d1d1d); border-color: var(--text-3, #555); }
+ .vtm-btn:active { transform: translateY(1px); }
+ .vtm-btn:focus-visible { outline: 2px solid var(--text-2, #b4b4b4); outline-offset: 1px; }
+ .vtm-btn[disabled] { opacity: 0.55; cursor: default; }
+ .vtm-btn--primary { background: var(--text, #f6f6f6); color: var(--bg, #060606); border-color: var(--text, #f6f6f6); font-weight: 600; }
+ .vtm-btn--primary:hover { background: #dcdcdc; }
+
+ #vtm-avatars { display: flex; flex-wrap: wrap; gap: 7px; margin-bottom: 12px; }
+ .vtm-avatar-chip {
+ appearance: none; border: 1px solid var(--hairline-2, #2c2c2c); border-radius: 18px;
+ background: var(--surface-1, #141414); color: var(--text-2, #b4b4b4);
+ padding: 6px 13px; font-size: 12.5px; cursor: pointer; transition: all var(--duration-fast);
+ }
+ .vtm-avatar-chip:hover { color: var(--text, #f6f6f6); border-color: var(--text-3, #555); }
+ .vtm-avatar-chip[aria-pressed='true'] { background: var(--text, #f6f6f6); color: var(--bg, #060606); border-color: var(--text, #f6f6f6); font-weight: 600; }
+ .vtm-url-row { display: flex; gap: 7px; }
+ .vtm-url-row input {
+ flex: 1; min-width: 0; background: var(--surface-1, #121212); color: var(--text, #f6f6f6);
+ border: 1px solid var(--hairline-2, #2c2c2c); border-radius: 8px; padding: 8px 10px; font-size: 12.5px;
+ }
+ .vtm-url-row input:focus { outline: none; border-color: var(--text-3, #555); }
+
+ .vtm-export-grid { display: grid; grid-template-columns: repeat(3, 1fr); gap: 8px; }
+ @media (max-width: 420px) { .vtm-export-grid { grid-template-columns: 1fr; } }
+ .vtm-export-grid .vtm-btn { width: 100%; }
+
+ .vtm-save-row { display: flex; flex-direction: column; gap: 8px; margin-top: 10px; }
+ .vtm-save-row select { background: var(--surface-1, #121212); color: var(--text, #f6f6f6); border: 1px solid var(--hairline-2, #2c2c2c); border-radius: 8px; padding: 8px 10px; font-size: 12.5px; }
+
+ #vtm-status { margin-top: 12px; font-size: 13.5px; color: var(--text-2, #b4b4b4); }
+ #vtm-status[data-kind='error'] { color: #ff7a7a; }
+ #vtm-status[data-kind='success'] { color: #6ee7a2; }
+ #vtm-status[hidden] { display: none; }
+
+ .vtm-related { margin-top: 30px; display: flex; flex-wrap: wrap; gap: 10px; font-size: 13px; color: var(--text-2, #b4b4b4); }
+ .vtm-related a { color: var(--text, #f6f6f6); border: 1px solid var(--hairline, #1f1f1f); border-radius: 8px; padding: 6px 12px; text-decoration: none; }
+ .vtm-related a:hover { border-color: var(--text-3, #555); }
+ </style>
+
+ <meta name="robots" content="index, follow, max-image-preview:large, max-snippet:-1">
+ <meta property="og:locale" content="en_US">
+ <meta property="og:image:alt" content="Video to Motion: Turn Any Video into an Animation File, three.ws">
+ <script type="application/ld+json">{"@context":"https://schema.org","@graph":[{"@type":"WebPage","name":"Video to Motion: Turn Any Video into an Animation File","description":"Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.","url":"https://three.ws/video-to-motion","isPartOf":{"@type":"WebSite","name":"three.ws","url":"https://three.ws"},"primaryImageOfPage":"https://three.ws/api/page-og?s=build&t=Video+to+Motion%3A+Turn+Any+Video+into+an+Animation+File&d=Upload+a+video+of+a+person+and+download+the+motion+as+a+GLB%2C+BVH%2C+or+FBX+animation+file&p=%2Fvideo-to-motion"}]}</script>
+ </head>
+ <body>
+ <div class="vtm-page">
+ <div id="nav-container"></div>
+ <script type="module" src="/nav.js"></script>
+
+ <div class="vtm-wrap">
+ <div class="vtm-head">
+ <span class="vtm-chip" data-i18n-html="video_to_motion.video_motion_capture_export"><span class="vtm-chip-dot"></span> Video motion capture, animation export</span>
+ <h1 data-i18n="video_to_motion.turn_any_video_into_an_animation_file">Turn any video into an animation file.</h1>
+ <p data-i18n-html="video_to_motion.upload_a_video_of_someone_moving_get">
+ Upload a video of someone moving and get the motion itself back: a GLB, a BVH, and an
+ FBX if the shape of your clip supports it. Preview it on a default avatar or drop in
+ your own rig, trim the clip, and download the files. Signed-in users can also save a
+ clip straight to their <a href="/animations">animation library</a> and apply it to an agent.
+ </p>
+ </div>
+
+ <div class="vtm-layout">
+ <div>
+ <div class="vtm-stage">
+ <div id="vtm-canvas-wrap"><canvas id="vtm-canvas" width="960" height="540" aria-label="Avatar motion preview"></canvas></div>
+
+ <div class="vtm-overlay" id="vtm-stage-empty">
+ <div class="vtm-dropzone" id="vtm-drop" role="button" tabindex="0" aria-label="Upload a video">
+ <div class="vtm-glyph">
+ <svg width="24" height="24" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.6"><path d="m22 8-6 4 6 4V8Z"/><rect x="2" y="6" width="14" height="12" rx="2"/></svg>
+ </div>
+ <h3 data-i18n="video_to_motion.drop_a_video_of_one_person">Drop a video of one person</h3>
+ <p data-i18n="video_to_motion.mp4_mov_or_webm_up_to_256">MP4, MOV, or WebM up to 256 MB. The first 90 seconds are processed. Full body in frame tracks best.</p>
+ </div>
+ <input id="vtm-file" type="file" accept="video/mp4,video/quicktime,video/webm" hidden />
+ </div>
+
+ <div class="vtm-overlay" id="vtm-stage-processing" hidden>
+ <div class="vtm-spinner" role="status" aria-label="Processing"></div>
+ <h3 id="vtm-processing-title" data-i18n="video_to_motion.capturing_your_motion">Capturing the motion</h3>
+ <p id="vtm-processing-note" class="vtm-note" data-i18n="video_to_motion.tracking_pose_frame_by_frame">Tracking pose frame-by-frame. A 30-second clip takes about two minutes.</p>
+ <div class="vtm-progress"><span id="vtm-progress-bar"></span></div>
+ </div>
+
+ <div class="vtm-overlay" id="vtm-stage-failed" hidden>
+ <h3 data-i18n="video_to_motion.that_capture_failed">That capture failed</h3>
+ <p id="vtm-fail-reason" class="vtm-note">Make sure one person is clearly visible and well lit.</p>
+ <button class="vtm-btn vtm-btn--primary" id="vtm-retry" type="button" data-i18n="video_to_motion.try_again">Try again</button>
+ </div>
+
+ <div class="vtm-overlay" id="vtm-stage-limit" hidden>
+ <h3 data-i18n="video_to_motion.limit_reached">Capture limit reached</h3>
+ <p class="vtm-note" data-i18n="video_to_motion.too_many_captures_from_this_connection">Too many captures from this connection recently. Wait a bit and try again.</p>
+ </div>
+
+ <div id="vtm-stage-ready" hidden></div>
+
+ <div class="vtm-controls">
+ <button class="vtm-btn" id="vtm-play" type="button" data-i18n="video_to_motion.play_pause">Play / Pause</button>
+ <button class="vtm-btn" id="vtm-restart" type="button" data-i18n="video_to_motion.restart">Restart</button>
+ <button class="vtm-btn" id="vtm-new" type="button" data-i18n="video_to_motion.new_video">New video</button>
+ </div>
+ <div class="vtm-trim" id="vtm-trim-row" hidden>
+ <span data-i18n="video_to_motion.trim">Trim</span>
+ <input id="vtm-trim-start" type="range" min="0" max="1" step="0.01" value="0" aria-label="Trim start" />
+ <output id="vtm-trim-start-out">0.0s</output>
+ <input id="vtm-trim-end" type="range" min="0" max="1" step="0.01" value="1" aria-label="Trim end" />
+ <output id="vtm-trim-end-out">0.0s</output>
+ </div>
+ </div>
+ <p id="vtm-status" hidden></p>
+ <video id="vtm-video" playsinline hidden></video>
+ </div>
+
+ <aside>
+ <div class="vtm-card">
+ <h2 data-i18n="video_to_motion.avatar">Avatar</h2>
+ <div id="vtm-avatars"></div>
+ <div class="vtm-url-row">
+ <input id="vtm-avatar-url" type="url" placeholder="…or paste a rigged GLB URL" aria-label="Avatar GLB URL" />
+ <button class="vtm-btn" id="vtm-avatar-url-go" type="button" data-i18n="video_to_motion.load">Load</button>
+ </div>
+ <div class="vtm-url-row" style="margin-top: 7px">
+ <input id="vtm-avatar-file" type="file" accept=".glb,model/gltf-binary" aria-label="Upload a rigged GLB" />
+ </div>
+ <p class="vtm-note" style="margin-top: 10px" data-i18n-html="video_to_motion.any_rigged_humanoid_glb_works">
+ Any rigged humanoid GLB works, including rigs from other platforms: Mixamo,
+ Avaturn, VRM/VRoid, Daz, MakeHuman, or one you <a href="/create">generated here</a>.
+ </p>
+ </div>
+
+ <div class="vtm-card">
+ <h2 data-i18n="video_to_motion.download">Download</h2>
+ <div class="vtm-export-grid">
+ <button class="vtm-btn" id="vtm-export-glb" type="button" disabled>GLB</button>
+ <button class="vtm-btn" id="vtm-export-bvh" type="button" disabled>BVH</button>
+ <button class="vtm-btn" id="vtm-export-fbx" type="button" disabled>FBX</button>
+ </div>
+ <p class="vtm-note" style="margin-top: 10px" data-i18n="video_to_motion.glb_carries_the_clip_on_your_chosen">
+ GLB carries the clip on your chosen avatar. BVH carries the clip on the platform's
+ own skeleton proportions, the standard format for re-retargeting in a DCC tool.
+ FBX is a server-side conversion (Blender) that preserves the rig; it can take
+ up to a minute.
+ </p>
+ </div>
+
+ <div class="vtm-card">
+ <h2 data-i18n="video_to_motion.save_to_library">Save to library</h2>
+ <div id="vtm-save-signed-out">
+ <p class="vtm-note" data-i18n-html="video_to_motion.sign_in_to_save_this_clip">
+ <a href="/login">Sign in</a> to save this clip to your animation library and apply
+ it to one of your agents.
+ </p>
+ </div>
+ <div id="vtm-save-signed-in" hidden>
+ <div class="vtm-save-row">
+ <select id="vtm-save-agent" aria-label="Agent to attach this clip to"></select>
+ <button class="vtm-btn vtm-btn--primary" id="vtm-save-go" type="button" data-i18n="video_to_motion.save_clip">Save clip</button>
+ </div>
+ </div>
+ </div>
+
+ <div class="vtm-card">
+ <h2 data-i18n="video_to_motion.how_it_works">How it works</h2>
+ <p class="vtm-note" data-i18n="video_to_motion.a_server_worker_tracks_pose_every">
+ A server worker tracks pose in every frame and solves it onto the platform's
+ canonical skeleton, the same retarget engine that drives every avatar animation
+ here, including hands.
+ </p>
+ <p class="vtm-note" data-i18n="video_to_motion.the_clip_retargets_onto_whichever_rig">
+ The clip retargets onto whichever rig you preview it on, right in your browser.
+ Trim it, then export; nothing is uploaded except the video itself and, for FBX
+ only, the chosen avatar's mesh for conversion.
+ </p>
+ </div>
+ </aside>
+ </div>
+
+ <div class="vtm-related">
+ <a href="/motion-swap" data-i18n="video_to_motion.related_motion_swap">Want a composited video instead? Try Motion Swap →</a>
+ <a href="/animations" data-i18n="video_to_motion.related_animations">Browse the animation library →</a>
+ <a href="/mocap-studio" data-i18n="video_to_motion.related_mocap_studio">Face mocap? Try Mocap Studio →</a>
+ </div>
+ </div>
+
+ <div id="footer-container"></div>
+ <script src="/footer.js" defer></script>
+ </div>
+
+ <script type="module" src="/src/video-to-motion.js"></script>
+ <script type="module" src="/i18n.js"></script>
+ </body>
+</html>
diff --git a/public/changelog-recent.json b/public/changelog-recent.json
index 1dfcfacdf..c16792f65 100644
--- a/public/changelog-recent.json
+++ b/public/changelog-recent.json
@@ -1,11 +1,11 @@
{
- "generated_at": "2026-10-13T07:01:56.347Z",
+ "generated_at": "2026-10-13T16:58:17.417Z",
"generated_by": "scripts/build-page-index.mjs from data/pages.json + data/changelog.json",
"site": {
"name": "three.ws",
"url": "https://three.ws"
},
- "total": 4057,
+ "total": 4059,
"complete": false,
"full_feed": "/changelog.json",
"entries": [
@@ -59,6 +59,16 @@
"launch"
]
},
+ {
+ "date": "2026-10-13",
+ "type": "launch",
+ "title": "Video to Motion: Turn Any Video into an Animation File",
+ "summary": "Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.",
+ "link": "/video-to-motion",
+ "tags": [
+ "launch"
+ ]
+ },
{
"date": "2026-10-13",
"type": "update",
@@ -115,6 +125,17 @@
"docs"
]
},
+ {
+ "date": "2026-10-13",
+ "type": "update",
+ "title": "Video to Motion: download any video's motion as a GLB, BVH, or FBX",
+ "summary": "A new page for creators who want the animation file itself, not a composited video: upload a video, preview the captured motion on a stock avatar or any rigged GLB you drop in, trim the in and out points, and download a GLB (clip baked onto the previewed avatar), a BVH (the platform's own canonical-skeleton proportions, the standard shape to re-retarget in a DCC tool), or an FBX (server-side Blender conversion that preserves the rig). Reuses the production model-video2motion worker that already powers /motion-swap for capture; signed-in users can save the clip straight to an agent's animation library. New writer: src/animation-bvh-export.js, round-tripped through three.js's own BVHLoader in tests/animation-bvh-export.test.js. Cross-linked from /motion-swap, /animations, and /mocap-studio. Docs: /docs/video-to-motion.",
+ "link": "/video-to-motion",
+ "tags": [
+ "feature",
+ "docs"
+ ]
+ },
{
"date": "2026-10-12",
"type": "launch",
@@ -1536,27 +1557,6 @@
"tags": [
"feature"
]
- },
- {
- "date": "2026-10-10",
- "type": "update",
- "title": "Talk to your squad in one sentence",
- "summary": "Squad chat turns a plain sentence like \"research $THREE and buy 0.05 SOL if it looks safe\" into a plan of steps for your specialist team or any of your agents. Research and entry checks run live with their evidence, and anything that signs, moves funds, launches a coin or goes over your cap stops as an approval card with the full details. Approve and the squad picks the run back up and finishes with receipts. Paper mode is the default and never signs. Say \"remember my default trade size is 0.1 SOL\" and the squad keeps it. Coin names and descriptions are treated as data, so a coin written to trick AI agents cannot change the plan. You can type or dictate, and the squad's 3D avatars react as they work.",
- "link": "/team-chat",
- "tags": [
- "feature",
- "security"
- ]
- },
- {
- "date": "2026-10-10",
- "type": "update",
- "title": "The 3D agent demos show their agent again",
- "summary": "Every page in the agent interaction lab had stopped drawing its 3D character, so the demos showed only text and buttons. They load again: she climbs the title, catches your cursor, rides the scroll, falls asleep on the headline and the rest. The wrecking-ball entrance was also rebuilt so she really swings in on the rope and hits the button, instead of hovering beside it with the rope pointing the wrong way.",
- "link": "/demos/agents",
- "tags": [
- "fix"
- ]
}
]
}
diff --git a/public/changelog.json b/public/changelog.json
index 6beb93de8..a36e73f59 100644
--- a/public/changelog.json
+++ b/public/changelog.json
@@ -1,5 +1,5 @@
{
- "generated_at": "2026-10-13T07:01:56.324Z",
+ "generated_at": "2026-10-13T16:58:17.389Z",
"generated_by": "scripts/build-page-index.mjs from data/pages.json + data/changelog.json",
"site": {
"name": "three.ws",
@@ -56,6 +56,16 @@
"launch"
]
},
+ {
+ "date": "2026-10-13",
+ "type": "launch",
+ "title": "Video to Motion: Turn Any Video into an Animation File",
+ "summary": "Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.",
+ "link": "/video-to-motion",
+ "tags": [
+ "launch"
+ ]
+ },
{
"date": "2026-10-13",
"type": "update",
@@ -112,6 +122,17 @@
"docs"
]
},
+ {
+ "date": "2026-10-13",
+ "type": "update",
+ "title": "Video to Motion: download any video's motion as a GLB, BVH, or FBX",
+ "summary": "A new page for creators who want the animation file itself, not a composited video: upload a video, preview the captured motion on a stock avatar or any rigged GLB you drop in, trim the in and out points, and download a GLB (clip baked onto the previewed avatar), a BVH (the platform's own canonical-skeleton proportions, the standard shape to re-retarget in a DCC tool), or an FBX (server-side Blender conversion that preserves the rig). Reuses the production model-video2motion worker that already powers /motion-swap for capture; signed-in users can save the clip straight to an agent's animation library. New writer: src/animation-bvh-export.js, round-tripped through three.js's own BVHLoader in tests/animation-bvh-export.test.js. Cross-linked from /motion-swap, /animations, and /mocap-studio. Docs: /docs/video-to-motion.",
+ "link": "/video-to-motion",
+ "tags": [
+ "feature",
+ "docs"
+ ]
+ },
{
"date": "2026-10-12",
"type": "launch",
diff --git a/public/changelog.xml b/public/changelog.xml
index 7ed50fb29..f0d6db2f3 100644
--- a/public/changelog.xml
+++ b/public/changelog.xml
@@ -46,6 +46,14 @@
<category>launch</category>
<description>Every open-source project, GPU model, and asset source three.ws is built on, grouped by what it powers: rendering, formats and compression, generation models, motion, voice, assets, and infrastructure. Generated from package.json on every build.</description>
</item>
+ <item>
+ <title>Video to Motion: Turn Any Video into an Animation File</title>
+ <link>https://three.ws/video-to-motion</link>
+ <guid isPermaLink="false">2026-10-13:Video to Motion: Turn Any Video into an Animation File</guid>
+ <pubDate>Tue, 13 Oct 2026 12:00:00 GMT</pubDate>
+ <category>launch</category>
+ <description>Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.</description>
+ </item>
<item>
<title>A Discord Activity brings a shared Forge room into any voice channel</title>
<link>https://three.ws/docs/discord-activity</link>
@@ -86,6 +94,14 @@
<category>improvement</category>
<description>GET /api/animations/library used to serve a bulk motion library baked from Mixamo, redistributable by anyone through scripts/mirror-animation-library.mjs even though Mixamo's own terms permit using its animations inside a project but not redistributing the raw files. The library is rebuilt from two CC0 1.0 Universal sources instead (Quaternius's Universal Animation Library 2 and Mesh2Motion's human animation packs), retargeted through the existing pipeline with every clip carrying a source and license field end to end from scripts/cc0-library.config.json through the published manifest. The /animations gallery shows each clip's pack and license next to its category chip, and the endpoint itself now refuses to publish any entry missing a license or naming Mixamo as its source, so a raw Mixamo clip can never reach the public endpoint or its mirror again. scripts/mixamo-all.mjs keeps its download and local-build phases for offline retargeting research, but its upload phase is permanently disabled.</description>
</item>
+ <item>
+ <title>Video to Motion: download any video's motion as a GLB, BVH, or FBX</title>
+ <link>https://three.ws/video-to-motion</link>
+ <guid isPermaLink="false">2026-10-13:Video to Motion: download any video's motion as a GLB, BVH, or FBX</guid>
+ <pubDate>Tue, 13 Oct 2026 12:00:00 GMT</pubDate>
+ <category>feature</category>
+ <description>A new page for creators who want the animation file itself, not a composited video: upload a video, preview the captured motion on a stock avatar or any rigged GLB you drop in, trim the in and out points, and download a GLB (clip baked onto the previewed avatar), a BVH (the platform's own canonical-skeleton proportions, the standard shape to re-retarget in a DCC tool), or an FBX (server-side Blender conversion that preserves the rig). Reuses the production model-video2motion worker that already powers /motion-swap for capture; signed-in users can save the clip straight to an agent's animation library. New writer: src/animation-bvh-export.js, round-tripped through three.js's own BVHLoader in tests/animation-bvh-export.test.js. Cross-linked from /motion-swap, /animations, and /mocap-studio. Docs: /docs/video-to-motion.</description>
+ </item>
<item>
<title>Docs · Engine bridges</title>
<link>https://three.ws/docs/engine-bridges</link>
diff --git a/public/features.json b/public/features.json
index fadc6460f..7e14057dc 100644
--- a/public/features.json
+++ b/public/features.json
@@ -1,5 +1,5 @@
{
- "generated_at": "2026-10-13T07:01:56.308Z",
+ "generated_at": "2026-10-13T16:58:17.372Z",
"generated_by": "scripts/build-page-index.mjs from data/pages.json",
"site": {
"name": "three.ws",
@@ -1317,6 +1317,18 @@
"priority": 0.7,
"job": "animate"
},
+ {
+ "path": "/video-to-motion",
+ "title": "Video to Motion: Turn Any Video into an Animation File",
+ "description": "Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.",
+ "added": "2026-10-13",
+ "auth": null,
+ "indexable": true,
+ "tags": [],
+ "showcase": false,
+ "priority": 0.7,
+ "job": "make"
+ },
{
"path": "/forge-nim",
"title": "Forge · NIM: Self-Hosted Image to 3D",
@@ -8960,6 +8972,18 @@
"priority": 0.6,
"job": null
},
+ {
+ "path": "/docs/video-to-motion",
+ "title": "Docs · Video to Motion",
+ "description": "Upload a video of a person and download the motion itself as a GLB, BVH, or FBX animation file, instead of a composited video. Preview on any rigged avatar, trim it, and save it to your animation library.",
+ "added": null,
+ "auth": null,
+ "indexable": true,
+ "tags": [],
+ "showcase": false,
+ "priority": 0.6,
+ "job": null
+ },
{
"path": "/docs/multi-agent",
"title": "Docs · Multi-agent scenes",
diff --git a/public/llms-full.txt b/public/llms-full.txt
index 76773b80a..eb180df4a 100644
--- a/public/llms-full.txt
+++ b/public/llms-full.txt
@@ -645,6 +645,12 @@ URL: https://three.ws/motion-swap
Upload a video of yourself and get it back with your 3D avatar performing your motion instead of you. Server-side pose tracking and person masking, browser-side compositing and export: stay off camera without staying off video.
+### Video to Motion: Turn Any Video into an Animation File
+
+URL: https://three.ws/video-to-motion
+
+Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.
+
### Forge · NIM: Self-Hosted Image to 3D
URL: https://three.ws/forge-nim
@@ -4508,6 +4514,12 @@ URL: https://three.ws/docs/motion-swap
Upload a video of yourself and get it back with your 3D avatar performing your exact motion instead of you. Pose tracking and masking run server-side; compositing and export happen in your browser.
+### Docs · Video to Motion
+
+URL: https://three.ws/docs/video-to-motion
+
+Upload a video of a person and download the motion itself as a GLB, BVH, or FBX animation file, instead of a composited video. Preview on any rigged avatar, trim it, and save it to your animation library.
+
### Docs · Multi-agent scenes
URL: https://three.ws/docs/multi-agent
diff --git a/public/llms.txt b/public/llms.txt
index 630f661ab..eb928567a 100644
--- a/public/llms.txt
+++ b/public/llms.txt
@@ -101,6 +101,6 @@ Machine-readable entry points. Fetch these instead of scraping HTML.
## Optional
-- [llms-full.txt](https://three.ws/llms-full.txt): Every public page (1006), grouped by section, with full descriptions.
+- [llms-full.txt](https://three.ws/llms-full.txt): Every public page (1008), grouped by section, with full descriptions.
- [Sitemap](https://three.ws/sitemap): The same index for people, with search.
- [sitemap.xml](https://three.ws/sitemap.xml): XML sitemap for crawlers, including every agent and avatar page.
diff --git a/public/locales/localized-pages.json b/public/locales/localized-pages.json
index 0252b59d7..272803c86 100644
--- a/public/locales/localized-pages.json
+++ b/public/locales/localized-pages.json
@@ -1,6 +1,6 @@
{
"generated": true,
- "count": 434,
+ "count": 435,
"paths": [
"/",
"/3d",
@@ -422,6 +422,7 @@
"/vault",
"/vaults",
"/viability",
+ "/video-to-motion",
"/viewer",
"/voice",
"/walk-leaderboard",
diff --git a/public/sitemap/index.html b/public/sitemap/index.html
index 6e3f8b6c4..16b4ffb85 100644
--- a/public/sitemap/index.html
+++ b/public/sitemap/index.html
@@ -96,7 +96,7 @@
<main class="sm-wrap">
<div class="sm-hero">
<h1>Sitemap</h1>
- <p><span>1066</span> <span data-i18n="sitemap_index.pages_grouped">pages on three.ws, grouped by purpose. Looking for the machine-readable versions?</span></p>
+ <p><span>1068</span> <span data-i18n="sitemap_index.pages_grouped">pages on three.ws, grouped by purpose. Looking for the machine-readable versions?</span></p>
<div class="sm-formats">
<a href="/everything"><code>/everything</code> · browse by goal, not by section</a>
<a href="/sitemap.xml"><code>sitemap.xml</code> · for search engines</a>
@@ -109,7 +109,7 @@
</div>
<div class="sm-filter">
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" aria-hidden="true"><circle cx="11" cy="11" r="7"/><path d="m21 21-4.3-4.3"/></svg>
- <input id="sm-filter-input" type="search" placeholder="Filter 1066 pages by name, path, or description…" aria-label="Filter pages" autocomplete="off" spellcheck="false" />
+ <input id="sm-filter-input" type="search" placeholder="Filter 1068 pages by name, path, or description…" aria-label="Filter pages" autocomplete="off" spellcheck="false" />
<kbd id="sm-filter-kbd" aria-hidden="true">/</kbd>
</div>
<p class="sm-filter-count" role="status" aria-live="polite"><span id="sm-filter-count"></span><span id="sm-filter-hint" aria-hidden="true"></span></p>
@@ -117,6 +117,14 @@
<h2 id="sm-new-title">Newest<a class="sm-new-more" href="/changelog">full changelog &rarr;</a></h2>
<p class="sm-section-desc">The 9 most recently launched pages, straight to the live surface.</p>
<ul class="sm-list">
+ <li>
+ <a href="/video-to-motion">
+ <span class="sm-new-date" translate="no">Oct 13, 2026</span>
+ <span class="sm-title"><span data-i18n="sitemap_index.t_video_to_motion">Video to Motion: Turn Any Video into an Animation File</span></span>
+ <span class="sm-path" translate="no">/video-to-motion</span>
+ <span class="sm-desc" data-i18n="sitemap_index.d_video_to_motion">Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.</span>
+ </a>
+ </li>
<li>
<a href="/docs/event-markets-staking">
<span class="sm-new-date" translate="no">Oct 13, 2026</span>
@@ -181,14 +189,6 @@
<span class="sm-desc" data-i18n="sitemap_index.d_everything">Every three.ws feature organised by what you want to do: make a 3D avatar, build an agent, embed it, sell skills, launch a token, trade and more. Searchable, with pins and a new-this-month feed.</span>
</a>
</li>
- <li>
- <a href="/apps">
- <span class="sm-new-date" translate="no">Oct 11, 2026</span>
- <span class="sm-title"><span data-i18n="sitemap_index.t_apps">Get the three.ws apps</span></span>
- <span class="sm-path" translate="no">/apps</span>
- <span class="sm-desc" data-i18n="sitemap_index.d_apps">Every way to run three.ws: the web desktop, Android, Solana Seeker, iPhone and iPad, and the Windows and Linux desktop app.</span>
- </a>
- </li>
</ul>
</section>
<nav class="sm-toc" aria-label="Sections">
@@ -719,7 +719,7 @@
<section class="sm-section" id="build">
<header>
- <h2><span data-i18n="sitemap_index.section_build">Build</span><span class="sm-count" data-count>92</span></h2>
+ <h2><span data-i18n="sitemap_index.section_build">Build</span><span class="sm-count" data-count>93</span></h2>
<p class="sm-section-desc">Create and customize an agent.</p>
</header>
<ul class="sm-list">
@@ -919,6 +919,13 @@
<span class="sm-desc" data-i18n="sitemap_index.d_motion_swap">Upload a video of yourself and get it back with your 3D avatar performing your motion instead of you. Server-side pose tracking and person masking, browser-side compositing and export: stay off camera without staying off video.</span>
</a>
</li>
+ <li>
+ <a href="/video-to-motion">
+ <span class="sm-title"><span data-i18n="sitemap_index.t_video_to_motion">Video to Motion: Turn Any Video into an Animation File</span></span>
+ <span class="sm-path" translate="no">/video-to-motion</span>
+ <span class="sm-desc" data-i18n="sitemap_index.d_video_to_motion">Upload a video of a person and download the motion as a GLB, BVH, or FBX animation file. Preview it on a default avatar or your own rig, trim it, and save it to your animation library.</span>
+ </a>
+ </li>
<li>
<a href="/forge-nim">
<span class="sm-title"><span data-i18n="sitemap_index.t_forge_nim">Forge · NIM: Self-Hosted Image to 3D</span></span>
@@ -2962,7 +2969,7 @@
<section class="sm-section" id="learn">
<header>
- <h2><span data-i18n="sitemap_index.section_learn">Learn</span><span class="sm-count" data-count>502</span></h2>
+ <h2><span data-i18n="sitemap_index.section_learn">Learn</span><span class="sm-count" data-count>503</span></h2>
<p class="sm-section-desc">Docs, tutorials, and reference.</p>
</header>
<ul class="sm-list">
@@ -5402,6 +5409,13 @@
<span class="sm-desc" data-i18n="sitemap_index.d_docs_motion_swap">Upload a video of yourself and get it back with your 3D avatar performing your exact motion instead of you. Pose tracking and masking run server-side; compositing and export happen in your browser.</span>
</a>
</li>
+ <li>
+ <a href="/docs/video-to-motion">
+ <span class="sm-title"><span data-i18n="sitemap_index.t_docs_video_to_motion">Docs · Video to Motion</span></span>
+ <span class="sm-path" translate="no">/docs/video-to-motion</span>
+ <span class="sm-desc" data-i18n="sitemap_index.d_docs_video_to_motion">Upload a video of a person and download the motion itself as a GLB, BVH, or FBX animation file, instead of a composited video. Preview on any rigged avatar, trim it, and save it to your animation library.</span>
+ </a>
+ </li>
<li>
<a href="/docs/multi-agent">
<span class="sm-title"><span data-i18n="sitemap_index.t_docs_multi_agent">Docs · Multi-agent scenes</span></span>
diff --git a/src/video-to-motion.js b/src/video-to-motion.js
new file mode 100644
index 000000000..c8b7eb55c
--- /dev/null
+++ b/src/video-to-motion.js
@@ -0,0 +1,697 @@
+// /video-to-motion: turn any video into a downloadable animation file.
+//
+// Reuses the production model-video2motion worker (the same one /motion-swap
+// drives) for pose capture, but this page is aimed at the clip itself rather
+// than a composited video: preview the motion on a default avatar or one the
+// user uploads, trim it, then download GLB (retargeted onto the previewed
+// avatar), BVH (on the platform's canonical skeleton), and FBX (server-side
+// conversion that preserves the rig, via /api/forge-remesh).
+
+const STOCK_AVATARS = [
+ { id: 'default', name: 'Standard', url: '/avatars/default.glb' },
+ { id: 'michelle', name: 'Michelle', url: '/avatars/michelle.glb' },
+ { id: 'mannequin', name: 'Mannequin', url: '/avatars/mannequin.glb' },
+ { id: 'realistic-female', name: 'Realistic', url: '/avatars/realistic-female.glb' },
+];
+
+const POLL_MS = 3000;
+const MAX_UPLOAD_BYTES = 256 * 1024 * 1024;
+const CONVERT_POLL_MS = 2_500;
+const CONVERT_TIMEOUT_MS = 180_000;
+
+const $ = (id) => document.getElementById(id);
+
+const state = {
+ three: null, // { THREE, GLTFLoader, renderer, scene, camera, mixer, action, clock }
+ job: null,
+ artifacts: null, // { clip_url }
+ clipJSON: null,
+ sourceClip: null, // AnimationClip, canonical skeleton, untrimmed
+ trimStart: 0,
+ trimEnd: 0,
+ avatar: null, // { root, group, retargetMod }
+ avatarUrl: STOCK_AVATARS[0].url,
+ boundClip: null, // AnimationClip retargeted + trimmed, bound to the current avatar
+ session: false,
+ agents: [],
+};
+
+function setStatus(text, kind = 'info') {
+ const el = $('vtm-status');
+ if (!el) return;
+ el.textContent = text;
+ el.dataset.kind = kind;
+ el.hidden = !text;
+}
+
+function showStage(stage) {
+ for (const s of ['empty', 'processing', 'failed', 'limit']) {
+ const el = $(`vtm-stage-${s}`);
+ if (el) el.hidden = s !== stage;
+ }
+ const ready = $('vtm-stage-ready');
+ if (ready) ready.hidden = stage !== 'ready';
+ const trimRow = $('vtm-trim-row');
+ if (trimRow) trimRow.hidden = stage !== 'ready';
+ for (const id of ['vtm-export-glb', 'vtm-export-bvh', 'vtm-export-fbx']) {
+ const btn = $(id);
+ if (btn) btn.disabled = stage !== 'ready';
+ }
+}
+
+// ---------------------------------------------------------------------------
+// Upload + job lifecycle (same /api/motion-swap worker /motion-swap uses)
+// ---------------------------------------------------------------------------
+
+async function uploadVideo(file) {
+ if (file.size > MAX_UPLOAD_BYTES) {
+ throw new Error('Video is over 256 MB. Trim it down (90 seconds is the processing cap anyway).');
+ }
+ const type = file.type || 'video/mp4';
+ setStatus('Requesting upload slot…');
+ const presign = await fetch('/api/motion-swap', {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ action: 'upload', content_type: type, size_bytes: file.size }),
+ });
+ const slot = await presign.json();
+ if (!presign.ok) throw new Error(slot?.message || 'Upload is unavailable right now.');
+
+ setStatus('Uploading video…');
+ const put = await fetch(slot.upload_url, {
+ method: slot.method || 'PUT',
+ headers: slot.headers || { 'content-type': type },
+ body: file,
+ });
+ if (!put.ok) throw new Error(`Upload failed (${put.status}). Try again.`);
+ return slot.public_url;
+}
+
+async function submitJob(videoUrl) {
+ setStatus('Starting capture…');
+ const res = await fetch('/api/motion-swap', {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ video_url: videoUrl }),
+ });
+ const data = await res.json();
+ if (res.status === 429) {
+ const err = new Error(data?.message || 'Too many captures from this connection. Try again later.');
+ err.limited = true;
+ throw err;
+ }
+ if (!res.ok) throw new Error(data?.message || 'Capture could not start.');
+ return data;
+}
+
+async function pollJob(jobId, etaSeconds) {
+ const started = Date.now();
+ for (;;) {
+ await new Promise((r) => setTimeout(r, POLL_MS));
+ const res = await fetch(`/api/motion-swap?job=${encodeURIComponent(jobId)}`);
+ const data = await res.json().catch(() => ({}));
+ if (res.status >= 400) {
+ throw new Error(data?.message || `Capture status unavailable (${res.status}).`);
+ }
+ if (data.status === 'done' && data.clip_url) return data;
+ if (data.status === 'failed') {
+ throw new Error(data.error || 'Capture failed. Make sure one person is clearly visible.');
+ }
+ const elapsed = Math.round((Date.now() - started) / 1000);
+ const pct = etaSeconds ? Math.min(95, Math.round((elapsed / etaSeconds) * 100)) : null;
+ const bar = $('vtm-progress-bar');
+ if (bar && pct !== null) bar.style.width = `${pct}%`;
+ const note = $('vtm-processing-note');
+ if (note) {
+ note.textContent =
+ pct !== null
+ ? `Tracking motion… ${pct}% (about ${Math.max(1, etaSeconds - elapsed)}s left)`
+ : `Tracking motion… ${elapsed}s`;
+ }
+ }
+}
+
+async function handleFile(file) {
+ try {
+ showStage('processing');
+ const bar = $('vtm-progress-bar');
+ if (bar) bar.style.width = '0%';
+ const videoUrl = await uploadVideo(file);
+ const job = await submitJob(videoUrl);
+ state.job = job.job_id;
+ const done = await pollJob(job.job_id, job.eta_seconds);
+ state.artifacts = done;
+ setStatus('Loading motion…');
+ await loadClip(done);
+ showStage('ready');
+ setStatus('');
+ } catch (err) {
+ if (err?.limited) {
+ showStage('limit');
+ } else {
+ showStage('failed');
+ const reason = $('vtm-fail-reason');
+ if (reason) reason.textContent = err?.message || 'Make sure one person is clearly visible and well lit.';
+ }
+ setStatus('');
+ }
+}
+
+// ---------------------------------------------------------------------------
+// three.js preview
+// ---------------------------------------------------------------------------
+
+async function bootThree() {
+ if (state.three) return state.three;
+ const [THREE, { GLTFLoader }] = await Promise.all([
+ import('three'),
+ import('three/addons/loaders/GLTFLoader.js'),
+ ]);
+ const canvas = $('vtm-canvas');
+ const renderer = new THREE.WebGLRenderer({ canvas, antialias: true });
+ renderer.setPixelRatio(Math.min(2, window.devicePixelRatio || 1));
+ renderer.outputColorSpace = THREE.SRGBColorSpace;
+
+ const scene = new THREE.Scene();
+ scene.background = new THREE.Color(0x0b0b0b);
+ const camera = new THREE.PerspectiveCamera(35, 16 / 9, 0.1, 50);
+ camera.position.set(0, 1.2, 3.4);
+ camera.lookAt(0, 1, 0);
+
+ scene.add(new THREE.HemisphereLight(0xffffff, 0x303033, 2.4));
+ const key = new THREE.DirectionalLight(0xffffff, 1.8);
+ key.position.set(1.5, 3, 4);
+ scene.add(key);
+ const grid = new THREE.GridHelper(6, 24, 0x2a2a2a, 0x1a1a1a);
+ scene.add(grid);
+
+ state.three = {
+ THREE,
+ GLTFLoader,
+ renderer,
+ scene,
+ camera,
+ mixer: null,
+ action: null,
+ clock: new THREE.Clock(),
+ playing: true,
+ };
+ return state.three;
+}
+
+function layout(t) {
+ const holder = $('vtm-canvas-wrap');
+ const w = holder.clientWidth || 960;
+ const h = Math.max(240, Math.round(w * 9 / 16));
+ t.renderer.setSize(w, h, false);
+ t.camera.aspect = w / h;
+ t.camera.updateProjectionMatrix();
+}
+
+async function loadAvatar(t, url) {
+ setStatus('Loading avatar…');
+ const [buf, canonMod, retargetMod, { getMeshoptDecoder }] = await Promise.all([
+ fetch(url).then((r) => {
+ if (!r.ok) throw new Error(`Avatar fetch failed (${r.status}).`);
+ return r.arrayBuffer();
+ }),
+ import('./glb-canonicalize.js'),
+ import('./animation-retarget.js'),
+ import('./viewer/internal.js'),
+ ]);
+ let glb = buf;
+ try {
+ const canon = canonMod.canonicalizeGLBBones(buf);
+ if (canon?.buffer) glb = canon.buffer;
+ else if (canon instanceof ArrayBuffer) glb = canon;
+ } catch {
+ // Not canonicalizable (already canonical or unusual container): retarget
+ // still resolves bone names through the same mapping at clip-bind time.
+ }
+ const loader = new t.GLTFLoader();
+ loader.setMeshoptDecoder(await getMeshoptDecoder());
+ const gltf = await loader.parseAsync(glb, '');
+ const root = gltf.scene;
+
+ if (state.avatar?.group) t.scene.remove(state.avatar.group);
+ if (t.mixer) {
+ t.mixer.stopAllAction();
+ t.mixer = null;
+ t.action = null;
+ }
+
+ const group = new t.THREE.Group();
+ group.add(root);
+ t.scene.add(group);
+ root.updateMatrixWorld(true);
+
+ const bbox = new t.THREE.Box3().setFromObject(root);
+ // Floor the avatar at the scene origin regardless of its own rest pose.
+ group.position.y = -bbox.min.y;
+
+ state.avatar = { root, group, retargetMod, sourceUrl: url };
+ if (state.clipJSON) bindClip(t);
+ setStatus('');
+}
+
+function bindClip(t) {
+ const { retargetMod } = state.avatar;
+ const clip = retargetMod.parseClipJSON(state.clipJSON, 'video-to-motion');
+ const result = retargetMod.retargetClipToObject(clip, state.avatar.root, { minCoverage: 0.3 });
+ const bound = result?.clip;
+ if (!bound) {
+ throw new Error(
+ 'That model has no compatible humanoid skeleton. Load a rigged avatar GLB (generate one at /create).',
+ );
+ }
+ state.sourceClip = clip;
+ if (!state.trimEnd) state.trimEnd = clip.duration;
+ const trimmed = retargetMod.trimClip(bound, state.trimStart, state.trimEnd);
+ state.boundClip = trimmed;
+
+ t.mixer = new t.THREE.AnimationMixer(state.avatar.root);
+ t.action = t.mixer.clipAction(trimmed);
+ t.action.play();
+ t.mixer.setTime(0);
+}
+
+async function loadClip(artifacts) {
+ const t = await bootThree();
+ const clipJSON = await fetch(artifacts.clip_url).then((r) => r.json());
+ state.clipJSON = clipJSON;
+ state.trimStart = 0;
+ state.trimEnd = clipJSON.duration ?? 0;
+
+ layout(t);
+ await loadAvatar(t, state.avatarUrl);
+ wireTrimRange(clipJSON.duration ?? 0);
+
+ cancelAnimationFrame(state._raf);
+ const tick = () => {
+ state._raf = requestAnimationFrame(tick);
+ if (t.mixer && t.playing) t.mixer.update(t.clock.getDelta());
+ else t.clock.getDelta();
+ t.renderer.render(t.scene, t.camera);
+ };
+ tick();
+}
+
+function wireTrimRange(duration) {
+ const startRange = $('vtm-trim-start');
+ const endRange = $('vtm-trim-end');
+ const startOut = $('vtm-trim-start-out');
+ const endOut = $('vtm-trim-end-out');
+ if (!startRange || !endRange) return;
+ startRange.min = '0';
+ startRange.max = String(duration);
+ startRange.step = '0.1';
+ startRange.value = '0';
+ endRange.min = '0';
+ endRange.max = String(duration);
+ endRange.step = '0.1';
+ endRange.value = String(duration);
+ if (startOut) startOut.textContent = '0.0s';
+ if (endOut) endOut.textContent = `${duration.toFixed(1)}s`;
+}
+
+function applyTrim() {
+ const t = state.three;
+ if (!t || !state.avatar) return;
+ try {
+ bindClip(t);
+ } catch (err) {
+ setStatus(err?.message || 'Could not retarget that trim range.', 'error');
+ }
+}
+
+// ---------------------------------------------------------------------------
+// Exports: a retargeted GLB, a canonical-skeleton BVH, and a rig-preserving
+// FBX converted server-side.
+// ---------------------------------------------------------------------------
+
+function downloadBlob(blob, filename) {
+ const a = document.createElement('a');
+ a.href = URL.createObjectURL(blob);
+ a.download = filename;
+ a.click();
+ setTimeout(() => URL.revokeObjectURL(a.href), 30_000);
+}
+
+async function exportGlbBlob() {
+ const t = state.three;
+ if (!t || !state.avatar || !state.boundClip) throw new Error('Load a capture first.');
+ const { GLTFExporter } = await import('three/addons/exporters/GLTFExporter.js');
+ const buffer = await new Promise((resolve, reject) => {
+ new GLTFExporter().parse(
+ state.avatar.root,
+ (glb) => resolve(glb),
+ (err) => reject(new Error(`GLTFExporter error: ${err?.message || err}`)),
+ { binary: true, animations: [state.boundClip] },
+ );
+ });
+ return new Blob([buffer], { type: 'model/gltf-binary' });
+}
+
+async function exportGlb() {
+ const btn = $('vtm-export-glb');
+ btn.disabled = true;
+ try {
+ const blob = await exportGlbBlob();
+ downloadBlob(blob, 'motion.glb');
+ } catch (err) {
+ setStatus(err?.message || 'GLB export failed.', 'error');
+ } finally {
+ btn.disabled = false;
+ }
+}
+
+async function exportBvh() {
+ const btn = $('vtm-export-bvh');
+ btn.disabled = true;
+ try {
+ if (!state.clipJSON) throw new Error('Load a capture first.');
+ // Trim at the clip.json level too, on the untrimmed clip's own time base,
+ // so the BVH (canonical skeleton, not the previewed avatar) matches the
+ // same trim window the GLB preview uses.
+ const [{ AnimationClip }, { trimClip }, { clipToBVH }] = await Promise.all([
+ import('three'),
+ import('./animation-retarget.js'),
+ import('./animation-bvh-export.js'),
+ ]);
+ const full = AnimationClip.parse(state.clipJSON);
+ const trimmed = trimClip(full, state.trimStart, state.trimEnd);
+ const bvhText = clipToBVH(trimmed, { fps: 30 });
+ downloadBlob(new Blob([bvhText], { type: 'text/plain' }), 'motion.bvh');
+ } catch (err) {
+ setStatus(err?.message || 'BVH export failed.', 'error');
+ } finally {
+ btn.disabled = false;
+ }
+}
+
+async function exportFbx() {
+ const btn = $('vtm-export-fbx');
+ const original = btn.textContent;
+ btn.disabled = true;
+ btn.textContent = 'Converting…';
+ try {
+ const glbBlob = await exportGlbBlob();
+ setStatus('Hosting GLB for conversion…');
+ const presign = await fetch('/api/avatar/presign-glb', {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ content_type: 'model/gltf-binary', bytes: glbBlob.size }),
+ });
+ const slot = await presign.json();
+ if (!presign.ok) throw new Error(slot?.message || 'Could not prepare FBX conversion.');
+ const put = await fetch(slot.upload_url, {
+ method: 'PUT',
+ headers: { 'content-type': 'model/gltf-binary' },
+ body: glbBlob,
+ });
+ if (!put.ok) throw new Error(`Upload for conversion failed (${put.status}).`);
+
+ setStatus('Converting to FBX…');
+ const res = await fetch('/api/forge-remesh', {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ mesh_url: slot.public_url, operation: 'convert', output_format: 'fbx' }),
+ });
+ const started = await res.json().catch(() => ({}));
+ if (!res.ok || !started.job_id) {
+ throw new Error(started.message || started.error || `FBX conversion returned ${res.status}`);
+ }
+ const deadline = Date.now() + CONVERT_TIMEOUT_MS;
+ let resultUrl = null;
+ while (Date.now() < deadline) {
+ await new Promise((r) => setTimeout(r, CONVERT_POLL_MS));
+ const pollRes = await fetch(`/api/forge-remesh?job=${encodeURIComponent(started.job_id)}`, {
+ headers: { accept: 'application/json' },
+ });
+ const data = await pollRes.json().catch(() => ({}));
+ if (!pollRes.ok) throw new Error(data.message || `FBX conversion returned ${pollRes.status}`);
+ if (data.status === 'done' && data.result_url) {
+ resultUrl = data.result_url;
+ break;
+ }
+ if (data.status === 'failed') throw new Error(data.error || 'FBX conversion failed.');
+ setStatus(data.status === 'running' ? 'Converting to FBX…' : 'Queued for conversion…');
+ }
+ if (!resultUrl) throw new Error('FBX conversion timed out.');
+ const fbxRes = await fetch(resultUrl);
+ if (!fbxRes.ok) throw new Error(`Could not fetch the converted FBX (${fbxRes.status}).`);
+ const fbxBlob = await fbxRes.blob();
+ downloadBlob(fbxBlob, 'motion.fbx');
+ setStatus('');
+ } catch (err) {
+ setStatus(err?.message || 'FBX export failed.', 'error');
+ } finally {
+ btn.disabled = false;
+ btn.textContent = original;
+ }
+}
+
+// ---------------------------------------------------------------------------
+// Save to library (signed-in only)
+// ---------------------------------------------------------------------------
+
+async function checkSession() {
+ try {
+ const res = await fetch('/api/auth/me', { credentials: 'include' });
+ const data = await res.json().catch(() => ({}));
+ state.session = Boolean(data?.user);
+ } catch {
+ state.session = false;
+ }
+ const { noteSession } = await import('./api.js');
+ noteSession(state.session);
+ $('vtm-save-signed-out').hidden = state.session;
+ $('vtm-save-signed-in').hidden = !state.session;
+ if (state.session) await loadAgents();
+}
+
+async function loadAgents() {
+ try {
+ const res = await fetch('/api/agents', { credentials: 'include' });
+ const data = await res.json().catch(() => ({}));
+ state.agents = Array.isArray(data?.agents) ? data.agents : Array.isArray(data) ? data : [];
+ } catch {
+ state.agents = [];
+ }
+ const select = $('vtm-save-agent');
+ if (!select) return;
+ select.innerHTML = '';
+ if (!state.agents.length) {
+ const opt = document.createElement('option');
+ opt.value = '';
+ opt.textContent = 'No agents yet';
+ select.appendChild(opt);
+ return;
+ }
+ for (const agent of state.agents) {
+ const opt = document.createElement('option');
+ opt.value = agent.id;
+ opt.textContent = agent.name || 'Agent';
+ select.appendChild(opt);
+ }
+}
+
+async function saveToLibrary() {
+ const btn = $('vtm-save-go');
+ const agentId = $('vtm-save-agent')?.value;
+ if (!agentId) {
+ setStatus('No agent selected to save this clip to.', 'error');
+ return;
+ }
+ btn.disabled = true;
+ btn.textContent = 'Saving…';
+ try {
+ const glbBlob = await exportGlbBlob();
+ const presignRes = await fetch('/api/animations/presign', {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ credentials: 'include',
+ body: JSON.stringify({ size_bytes: glbBlob.size, content_type: 'model/gltf-binary', slug: 'video-to-motion' }),
+ });
+ const presign = await presignRes.json();
+ if (!presignRes.ok) throw new Error(presign?.message || 'Could not prepare the save.');
+
+ const put = await fetch(presign.upload_url, {
+ method: presign.method || 'PUT',
+ headers: presign.headers || { 'content-type': 'model/gltf-binary' },
+ body: glbBlob,
+ });
+ if (!put.ok) throw new Error(`Upload failed (${put.status}).`);
+
+ const { apiFetch } = await import('./api.js');
+ const agentRes = await fetch(`/api/agents/${encodeURIComponent(agentId)}`, { credentials: 'include' }).then((r) =>
+ r.json(),
+ );
+ const current = Array.isArray(agentRes?.meta?.animations) ? agentRes.meta.animations : [];
+ const entry = {
+ name: `Video capture ${new Date().toLocaleDateString()}`,
+ url: presign.storage_key,
+ loop: true,
+ source: 'custom',
+ addedAt: new Date().toISOString(),
+ };
+ const putRes = await apiFetch(`/api/agents/${encodeURIComponent(agentId)}/animations`, {
+ method: 'PUT',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ animations: [...current, entry] }),
+ });
+ const putData = await putRes.json().catch(() => ({}));
+ if (!putRes.ok) throw new Error(putData?.message || 'Could not save the clip.');
+ setStatus('Saved to your animation library.', 'success');
+ } catch (err) {
+ setStatus(err?.message || 'Save failed.', 'error');
+ } finally {
+ btn.disabled = false;
+ btn.textContent = 'Save clip';
+ }
+}
+
+// ---------------------------------------------------------------------------
+// UI wiring
+// ---------------------------------------------------------------------------
+
+function wireUI() {
+ const drop = $('vtm-drop');
+ const input = $('vtm-file');
+
+ drop.addEventListener('click', () => input.click());
+ drop.addEventListener('keydown', (e) => {
+ if (e.key === 'Enter' || e.key === ' ') {
+ e.preventDefault();
+ input.click();
+ }
+ });
+ input.addEventListener('change', () => {
+ if (input.files?.[0]) handleFile(input.files[0]);
+ input.value = '';
+ });
+ for (const evt of ['dragover', 'dragenter']) {
+ drop.addEventListener(evt, (e) => {
+ e.preventDefault();
+ drop.classList.add('is-drag');
+ });
+ }
+ for (const evt of ['dragleave', 'drop']) {
+ drop.addEventListener(evt, (e) => {
+ e.preventDefault();
+ drop.classList.remove('is-drag');
+ });
+ }
+ drop.addEventListener('drop', (e) => {
+ const file = e.dataTransfer?.files?.[0];
+ if (file && file.type.startsWith('video/')) handleFile(file);
+ else if (file) setStatus('That is not a video file.', 'error');
+ });
+
+ $('vtm-retry').addEventListener('click', () => {
+ showStage('empty');
+ setStatus('');
+ });
+ $('vtm-new').addEventListener('click', () => {
+ const t = state.three;
+ if (t) t.playing = false;
+ showStage('empty');
+ setStatus('');
+ history.replaceState(null, '', '/video-to-motion');
+ });
+ $('vtm-play').addEventListener('click', () => {
+ if (state.three) state.three.playing = !state.three.playing;
+ });
+ $('vtm-restart').addEventListener('click', () => {
+ if (state.three?.mixer) state.three.mixer.setTime(0);
+ });
+
+ // Avatar picker: stock avatars + URL + file upload.
+ const picker = $('vtm-avatars');
+ for (const av of STOCK_AVATARS) {
+ const b = document.createElement('button');
+ b.type = 'button';
+ b.className = 'vtm-avatar-chip';
+ b.textContent = av.name;
+ if (av.url === state.avatarUrl) b.setAttribute('aria-pressed', 'true');
+ b.addEventListener('click', async () => {
+ picker.querySelectorAll('[aria-pressed]').forEach((x) => x.removeAttribute('aria-pressed'));
+ b.setAttribute('aria-pressed', 'true');
+ state.avatarUrl = av.url;
+ if (state.three && state.clipJSON) {
+ try {
+ await loadAvatar(state.three, av.url);
+ } catch (err) {
+ setStatus(err?.message || 'Avatar failed to load.', 'error');
+ }
+ }
+ });
+ picker.appendChild(b);
+ }
+ $('vtm-avatar-url-go').addEventListener('click', async () => {
+ const url = $('vtm-avatar-url').value.trim();
+ if (!url) return;
+ picker.querySelectorAll('[aria-pressed]').forEach((x) => x.removeAttribute('aria-pressed'));
+ state.avatarUrl = url;
+ if (state.three && state.clipJSON) {
+ try {
+ await loadAvatar(state.three, url);
+ } catch (err) {
+ setStatus(err?.message || 'Avatar failed to load. Is it a rigged humanoid GLB?', 'error');
+ }
+ }
+ });
+ $('vtm-avatar-file').addEventListener('change', async (e) => {
+ const file = e.target.files?.[0];
+ if (!file) return;
+ const url = URL.createObjectURL(file);
+ picker.querySelectorAll('[aria-pressed]').forEach((x) => x.removeAttribute('aria-pressed'));
+ state.avatarUrl = url;
+ if (state.three && state.clipJSON) {
+ try {
+ await loadAvatar(state.three, url);
+ } catch (err) {
+ setStatus(err?.message || 'Avatar failed to load. Is it a rigged humanoid GLB?', 'error');
+ }
+ }
+ });
+
+ // Trim.
+ const startRange = $('vtm-trim-start');
+ const endRange = $('vtm-trim-end');
+ const applyIfSettled = (() => {
+ let timer = null;
+ return () => {
+ clearTimeout(timer);
+ timer = setTimeout(applyTrim, 200);
+ };
+ })();
+ startRange.addEventListener('input', () => {
+ const v = Number(startRange.value);
+ state.trimStart = Math.min(v, state.trimEnd - 0.1);
+ startRange.value = String(state.trimStart);
+ $('vtm-trim-start-out').textContent = `${state.trimStart.toFixed(1)}s`;
+ applyIfSettled();
+ });
+ endRange.addEventListener('input', () => {
+ const v = Number(endRange.value);
+ state.trimEnd = Math.max(v, state.trimStart + 0.1);
+ endRange.value = String(state.trimEnd);
+ $('vtm-trim-end-out').textContent = `${state.trimEnd.toFixed(1)}s`;
+ applyIfSettled();
+ });
+
+ // Exports + save.
+ $('vtm-export-glb').addEventListener('click', exportGlb);
+ $('vtm-export-bvh').addEventListener('click', exportBvh);
+ $('vtm-export-fbx').addEventListener('click', exportFbx);
+ $('vtm-save-go').addEventListener('click', saveToLibrary);
+
+ window.addEventListener('resize', () => {
+ if (state.three) layout(state.three);
+ });
+}
+
+wireUI();
+showStage('empty');
+checkSession();
diff --git a/tests/_fixtures/daz-rig.glb b/tests/_fixtures/daz-rig.glb
new file mode 100644
index 000000000..b5f81b91c
Binary files /dev/null and b/tests/_fixtures/daz-rig.glb differ
diff --git a/tests/e2e/video-to-motion.spec.js b/tests/e2e/video-to-motion.spec.js
new file mode 100644
index 000000000..dfc923eeb
--- /dev/null
+++ b/tests/e2e/video-to-motion.spec.js
@@ -0,0 +1,244 @@
+/**
+ * /video-to-motion regression cover. Drives the real page against a stubbed
+ * /api/motion-swap, /api/auth/me and /api/agents so the capture → preview →
+ * export lifecycle is deterministic:
+ * • the empty state (dropzone) renders with no console errors at three
+ * breakpoints (320, 768, 1440)
+ * • a successful capture drives processing → ready, enables the export
+ * buttons, and shows the trim row
+ * • a failed capture shows the failed overlay with a retry path back to
+ * empty; a 429 shows the limit overlay
+ * • a non-Mixamo (Daz/Genesis bone names) rig uploaded through the file
+ * input retargets in the browser with no error surfaced in #vtm-status
+ *
+ * The clip.json fixture is a two-keyframe clip on the canonical skeleton
+ * (Hips only), the smallest shape retargetClipToObject accepts.
+ */
+
+import { readFileSync } from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { test, expect } from '@playwright/test';
+
+const HERE = path.dirname(fileURLToPath(import.meta.url));
+const DAZ_RIG = readFileSync(path.join(HERE, '..', '_fixtures', 'daz-rig.glb'));
+
+const CLIP_JSON = {
+ name: 'video-to-motion-test',
+ duration: 1,
+ tracks: [
+ {
+ name: 'Hips.quaternion',
+ type: 'quaternion',
+ times: [0, 1],
+ values: [0, 0, 0, 1, 0, 0, 0, 1],
+ },
+ {
+ name: 'Hips.position',
+ type: 'vector',
+ times: [0, 1],
+ values: [0, 1, 0, 0, 1, 0],
+ },
+ ],
+};
+
+async function stubSignedOut(page) {
+ await page.route('**/api/auth/me', (r) => r.fulfill({ json: {} }));
+}
+
+async function stubSuccessfulCapture(page) {
+ await page.route('**/api/motion-swap', (r) => {
+ const body = r.request().postDataJSON?.() || {};
+ if (body.action === 'upload') {
+ return r.fulfill({
+ json: {
+ upload_url: 'https://cdn.example.invalid/upload-slot',
+ public_url: 'https://cdn.example.invalid/video.mp4',
+ method: 'PUT',
+ headers: { 'content-type': 'video/mp4' },
+ },
+ });
+ }
+ return r.fulfill({ json: { job_id: 'job-1', eta_seconds: 1 } });
+ });
+ await page.route('https://cdn.example.invalid/upload-slot', (r) => r.fulfill({ status: 200, body: '' }));
+ await page.route('**/api/motion-swap?job=job-1', (r) =>
+ r.fulfill({ json: { status: 'done', clip_url: 'https://cdn.example.invalid/clip.json' } }),
+ );
+ await page.route('https://cdn.example.invalid/clip.json', (r) => r.fulfill({ json: CLIP_JSON }));
+}
+
+test.describe('/video-to-motion', () => {
+ test.beforeEach(async ({ page }) => {
+ await stubSignedOut(page);
+ });
+
+ for (const width of [320, 768, 1440]) {
+ test(`empty state renders with no console errors at ${width}px`, async ({ page }) => {
+ test.setTimeout(90_000);
+ const errors = [];
+ page.on('console', (msg) => {
+ if (msg.type() === 'error') errors.push(msg.text());
+ });
+ page.on('pageerror', (err) => errors.push(String(err)));
+
+ await page.setViewportSize({ width, height: 900 });
+ await page.goto('/video-to-motion');
+
+ await expect(page.locator('#vtm-drop')).toBeVisible();
+ await expect(page.locator('#vtm-stage-processing')).toBeHidden();
+ await expect(page.locator('#vtm-export-glb')).toBeDisabled();
+ await expect(page.locator('#vtm-save-signed-out')).toBeVisible();
+
+ expect(errors).toEqual([]);
+ });
+ }
+
+ test('a successful capture reaches the ready stage and enables export', async ({ page }) => {
+ test.setTimeout(90_000);
+ await stubSuccessfulCapture(page);
+ await page.goto('/video-to-motion');
+
+ await page.setInputFiles('#vtm-file', {
+ name: 'clip.mp4',
+ mimeType: 'video/mp4',
+ buffer: Buffer.from('fake-video-bytes'),
+ });
+
+ await expect(page.locator('#vtm-stage-processing')).toBeVisible();
+ await expect(page.locator('#vtm-trim-row')).toBeVisible({ timeout: 30_000 });
+ await expect(page.locator('#vtm-export-glb')).toBeEnabled();
+ await expect(page.locator('#vtm-export-bvh')).toBeEnabled();
+ });
+
+ test('a failed capture shows the failed overlay, and retry returns to empty', async ({ page }) => {
+ test.setTimeout(90_000);
+ await page.route('**/api/motion-swap', (r) => {
+ const body = r.request().postDataJSON?.() || {};
+ if (body.action === 'upload') {
+ return r.fulfill({
+ json: {
+ upload_url: 'https://cdn.example.invalid/upload-slot',
+ public_url: 'https://cdn.example.invalid/video.mp4',
+ method: 'PUT',
+ headers: { 'content-type': 'video/mp4' },
+ },
+ });
+ }
+ return r.fulfill({ json: { job_id: 'job-2', eta_seconds: 1 } });
+ });
+ await page.route('https://cdn.example.invalid/upload-slot', (r) => r.fulfill({ status: 200, body: '' }));
+ await page.route('**/api/motion-swap?job=job-2', (r) =>
+ r.fulfill({ json: { status: 'failed', error: 'No person detected in the frame.' } }),
+ );
+ await page.goto('/video-to-motion');
+
+ await page.setInputFiles('#vtm-file', {
+ name: 'clip.mp4',
+ mimeType: 'video/mp4',
+ buffer: Buffer.from('fake-video-bytes'),
+ });
+
+ await expect(page.locator('#vtm-stage-failed')).toBeVisible({ timeout: 30_000 });
+ await expect(page.locator('#vtm-fail-reason')).toHaveText('No person detected in the frame.');
+
+ await page.locator('#vtm-retry').click();
+ await expect(page.locator('#vtm-stage-empty')).toBeVisible();
+ });
+
+ test('a 429 shows the capture-limit overlay', async ({ page }) => {
+ test.setTimeout(90_000);
+ await page.route('**/api/motion-swap', (r) => {
+ const body = r.request().postDataJSON?.() || {};
+ if (body.action === 'upload') {
+ return r.fulfill({
+ json: {
+ upload_url: 'https://cdn.example.invalid/upload-slot',
+ public_url: 'https://cdn.example.invalid/video.mp4',
+ method: 'PUT',
+ headers: { 'content-type': 'video/mp4' },
+ },
+ });
+ }
+ return r.fulfill({ status: 429, json: { message: 'Too many captures from this connection.' } });
+ });
+ await page.route('https://cdn.example.invalid/upload-slot', (r) => r.fulfill({ status: 200, body: '' }));
+ await page.goto('/video-to-motion');
+
+ await page.setInputFiles('#vtm-file', {
+ name: 'clip.mp4',
+ mimeType: 'video/mp4',
+ buffer: Buffer.from('fake-video-bytes'),
+ });
+
+ await expect(page.locator('#vtm-stage-limit')).toBeVisible({ timeout: 30_000 });
+ });
+
+ test('a non-Mixamo (Daz/Genesis) rig uploaded via file input retargets with no error', async ({ page }) => {
+ test.setTimeout(90_000);
+ await stubSuccessfulCapture(page);
+ await page.goto('/video-to-motion');
+
+ await page.setInputFiles('#vtm-file', {
+ name: 'clip.mp4',
+ mimeType: 'video/mp4',
+ buffer: Buffer.from('fake-video-bytes'),
+ });
+ await expect(page.locator('#vtm-export-glb')).toBeEnabled({ timeout: 30_000 });
+
+ await page.setInputFiles('#vtm-avatar-file', {
+ name: 'daz-rig.glb',
+ mimeType: 'model/gltf-binary',
+ buffer: DAZ_RIG,
+ });
+
+ // A failed retarget surfaces its message in #vtm-status with
+ // data-kind="error"; a successful one clears #vtm-status entirely.
+ await expect(page.locator('#vtm-status[data-kind="error"]')).toBeHidden({ timeout: 15_000 });
+ await expect(page.locator('#vtm-export-glb')).toBeEnabled();
+ });
+});
+
+test.describe('/video-to-motion exports', () => {
+ test.beforeEach(async ({ page }) => {
+ await stubSignedOut(page);
+ });
+
+ test('GLB export is a well-formed glTF binary carrying the clip; BVH export is a well-formed hierarchy', async ({ page }) => {
+ test.setTimeout(90_000);
+ await stubSuccessfulCapture(page);
+ await page.goto('/video-to-motion');
+ await page.setInputFiles('#vtm-file', {
+ name: 'clip.mp4',
+ mimeType: 'video/mp4',
+ buffer: Buffer.from('fake-video-bytes'),
+ });
+ await expect(page.locator('#vtm-export-glb')).toBeEnabled({ timeout: 30_000 });
+
+ const [glbDownload] = await Promise.all([
+ page.waitForEvent('download'),
+ page.locator('#vtm-export-glb').click(),
+ ]);
+ const glbBytes = readFileSync(await glbDownload.path());
+ const dv = new DataView(glbBytes.buffer, glbBytes.byteOffset, glbBytes.byteLength);
+ expect(dv.getUint32(0, true)).toBe(0x46546c67); // 'glTF' magic
+ expect(dv.getUint32(4, true)).toBe(2); // glTF version 2
+ const jsonLen = dv.getUint32(12, true);
+ const json = JSON.parse(Buffer.from(glbBytes.subarray(20, 20 + jsonLen)).toString('utf8'));
+ expect(Array.isArray(json.animations)).toBe(true);
+ expect(json.animations.length).toBeGreaterThan(0);
+ expect(json.animations[0].channels.length).toBeGreaterThan(0);
+
+ const [bvhDownload] = await Promise.all([
+ page.waitForEvent('download'),
+ page.locator('#vtm-export-bvh').click(),
+ ]);
+ const bvhText = readFileSync(await bvhDownload.path(), 'utf8');
+ expect(bvhText.startsWith('HIERARCHY')).toBe(true);
+ expect(bvhText).toContain('ROOT Hips');
+ expect(bvhText).toContain('MOTION');
+ const frameMatch = bvhText.match(/Frames:\s*(\d+)/);
+ expect(frameMatch).toBeTruthy();
+ expect(Number(frameMatch[1])).toBeGreaterThan(0);
+ });
+});
diff --git a/vercel.json b/vercel.json
index ec4244848..3f39387e9 100644
--- a/vercel.json
+++ b/vercel.json
@@ -3157,6 +3157,14 @@
"src": "/motion-swap/",
"dest": "/motion-swap.html"
},
+ {
+ "src": "/video-to-motion",
+ "dest": "/video-to-motion.html"
+ },
+ {
+ "src": "/video-to-motion/",
+ "dest": "/video-to-motion.html"
+ },
{
"src": "/integrations",
"dest": "/integrations.html"
diff --git a/vite.config.js b/vite.config.js
index af1146e99..c4c9e5284 100644
--- a/vite.config.js
+++ b/vite.config.js
@@ -1112,6 +1112,7 @@ const appConfig = {
splat: resolve(__dirname, 'pages/splat.html'),
capture: resolve(__dirname, 'pages/capture.html'),
'motion-swap': resolve(__dirname, 'pages/motion-swap.html'),
+ 'video-to-motion': resolve(__dirname, 'pages/video-to-motion.html'),
club: resolve(__dirname, 'pages/club.html'),
theater: resolve(__dirname, 'pages/theater.html'),
stage: resolve(__dirname, 'pages/stage.html'),
@@ -2084,6 +2085,8 @@ const appConfig = {
'/capture/': resolve(root, 'pages/capture.html'),
'/motion-swap': resolve(root, 'pages/motion-swap.html'),
'/motion-swap/': resolve(root, 'pages/motion-swap.html'),
+ '/video-to-motion': resolve(root, 'pages/video-to-motion.html'),
+ '/video-to-motion/': resolve(root, 'pages/video-to-motion.html'),
'/brain': resolve(root, 'pages/brain.html'),
'/brain/': resolve(root, 'pages/brain.html'),
'/lipsync': resolve(root, 'public/demos/lipsync-tts.html'),