Skip to content

A lightweight 2.5D character player for the web, with lip sync and webcam face tracking.

Docs and examples: @types/three for skipLibCheck off, a camera start that hangs says so; the Firefox and WebKit job counts in CI

LivLiv81af0e7f1793
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index bb0424f..4d7d451 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -23,11 +23,9 @@ jobs:
- run: npm run test:types
- run: npx playwright-core install --with-deps chromium-headless-shell
- run: npm run test:browser
- # Firefox and WebKit: they start WebGL2 only where they find a GPU or a software GL, and the Linux runner gives them
- # neither. Tried here first without failing the run, until it has been seen to pass on this runner.
+ # Firefox and WebKit: they start WebGL2 only where they find a GPU or a software GL, and the Linux runner gives them neither
engines:
runs-on: macos-latest
- continue-on-error: true
timeout-minutes: 15
steps:
- uses: actions/checkout@v5
diff --git a/API.md b/API.md
index 8deee1d..b7c01ae 100644
--- a/API.md
+++ b/API.md
@@ -392,7 +392,7 @@ await vrm.load('/avatars/me.vrm') // a URL, a File, or files picked
- **VRM** (`@livliv-dev/player/vrm`): VRM 0.x and 1.0, drawn with [three.js](https://threejs.org) and
[three-vrm](https://github.com/pixiv/three-vrm), which you install (`npm install three @pixiv/three-vrm`; three.js 0.170
or newer, played here on 0.170, 0.186 and 0.187 with three-vrm 3.5.5; optional peer dependencies, so the rest of the
- package does not need them). Files from VRoid Studio and UniVRM play. So does a real file from the VRM add-on for Blender; a fresh export from Blender hasn't been tested, and [issue #1](https://github.com/livliv-dev/player/issues/1) is still open. The head and neck turn, the spine sways, the eyes
+ package does not need them; TypeScript with `skipLibCheck: false` also needs `@types/three`). Files from VRoid Studio and UniVRM play. So does a real file from the VRM add-on for Blender; a fresh export from Blender hasn't been tested, and [issue #1](https://github.com/livliv-dev/player/issues/1) is still open. The head and neck turn, the spine sways, the eyes
blink and look, the mouth takes the visemes (`aa`, `ee`, `ou`, or `ih`, `oh` where those are missing, from `mouthOpen` and `vowel`),
and the expressions are the model's `happy`, `angry`, `sad`, `relaxed` and `surprised` where it has them, then any
of its own under the names it gives them; hair and clothes swing by the model's own spring bones, and its constraints apply. Framed from the head to the hips with the arms down; no hands, no full body, no camera
diff --git a/example/camera.js b/example/camera.js
index 74e37d5..675f5f1 100644
--- a/example/camera.js
+++ b/example/camera.js
@@ -58,6 +58,8 @@ export function cameraButton(button, player, status) {
before ||= status.textContent
button.disabled = true
let asking = true // the face model goes on downloading after a refusal: its progress is no news then
+ // nothing here ends a start that hangs (a model that never comes, a camera that never answers): at least say so
+ const slow = setTimeout(() => { if (asking) status.textContent = 'Taking long: a slow connection, or the camera is not answering. Reload the page to try again.' }, 30000)
try {
const { startFaceTracking } = await import('../dist/tracking.js')
running = await startFaceTracking(player, {
@@ -66,8 +68,10 @@ export function cameraButton(button, player, status) {
onProgress: (s) => { if (asking) status.textContent = s.step === 'download' ? `Face model ${Math.round(s.progress * 100)}%` : `${s.step}…` },
})
asking = false
+ clearTimeout(slow)
status.textContent = trace ? 'Demo motion, no camera in use' : 'Camera on'
} catch (e) {
+ clearTimeout(slow)
asking = false
status.textContent = before
if (!NOTES[e.name]) console.error('Face tracking could not start:', e)
diff --git a/test/smoke.mjs b/test/smoke.mjs
index 2235fb5..c648df2 100644
--- a/test/smoke.mjs
+++ b/test/smoke.mjs
@@ -412,6 +412,24 @@ try {
console.log(`${said ? 'ok ' : 'FAIL'} example/demo.html: the context lost, the page says so`)
await p.close()
}
+ // a camera start that hangs (a face model that never comes): the page says so after a while, and the button stays as it was
+ {
+ const p = await browser.newPage()
+ await p.clock.install()
+ await p.route(/face_landmarker\.task|tasks-vision.*wasm|vision_wasm/, () => {}) // never answered
+ await p.goto(`http://127.0.0.1:${server.address().port}/example/demo.html`)
+ await p.clock.runFor(100)
+ const up = await p.waitForFunction(() => window.player?.model, null, { timeout: 30000, polling: 200 }).then(() => true, () => false)
+ const cam = up && await p.getByRole('button', { name: /camera/i }).first().click().then(() => true, () => false)
+ if (cam) await p.getByRole('button', { name: 'Turn on camera' }).click()
+ await new Promise((r) => setTimeout(r, 500))
+ await p.clock.runFor(31000)
+ const said = cam && await p.evaluate(() => document.getElementById('status').textContent)
+ const ok = /Taking long/.test(said || '')
+ if (!ok) failed++
+ console.log(`${ok ? 'ok ' : 'FAIL'} example/demo.html: a camera start that hangs says so: "${String(said).slice(0, 110)}"`)
+ await p.close()
+ }
// React under StrictMode, which mounts everything twice (its files come from a CDN: without a connection, not tested)
{
const p = await browser.newPage(), said = []