3D engine that AI agents drive
Add touch, device motion, smoothing and input replay to the input layer (part of #773)
Nguyen Xuan An86cb44c257af
diff --git a/README.md b/README.mdindex 43a5cca3..5331e4e6 100644--- a/README.md+++ b/README.md@@ -139,7 +139,7 @@ Each addon is a separate import, so a page loads only what it uses.| Vehicles | `67js/addons/vehicle` | [Wheeled vehicles on a path](https://67js.codepawl.com/vehicle.md) || Physics | `67js/addons/physics` | [Rapier contacts and joints](docs/physics.md) recorded into a timeline || Live games | `67js/addons/runtime` | [Game loop](docs/runtime.md) with fixed or variable steps and pluggable systems |-| Live games | `67js/addons/input` | [Input](docs/input.md): keyboard, mouse, gamepad and custom sources mapped to named actions and axes |+| Live games | `67js/addons/input` | [Input](docs/input.md): keyboard, mouse, gamepad, touch, device motion, replay and custom sources mapped to named actions and axes || Modeling | `67js/addons/mesh`, `67js/addons/modeling`, `67js/addons/sculpt` | [Mesh editing](docs/mesh.md), [modeling](docs/modeling.md) and [sculpt brushes](https://67js.codepawl.com/sculpt.md) || Offline light | `67js/addons/path-tracer`, `67js/addons/light-bake` | [Path-traced stills](https://67js.codepawl.com/path-tracer.md) and [baked lightmaps](https://67js.codepawl.com/light-bake.md) || Post passes | `67js/addons/user-pass` | [Your own full-screen GLSL passes](docs/user-passes.md): dither, pixel downscale, quantise |diff --git a/changes/773-touch-replay.md b/changes/773-touch-replay.mdnew file mode 100644index 00000000..9032a269--- /dev/null+++ b/changes/773-touch-replay.md@@ -0,0 +1 @@+- `67js/addons/input` adds `touchSource` (virtual sticks, tap, swipe, pinch), `deviceMotionSource`, axis `smoothing`, and `recordInput` with `replaySource` so one demo runs from keyboard, gamepad, touch or a replay (#773)diff --git a/docs/input.md b/docs/input.mdindex 46b78ade..8f0f1c88 100644--- a/docs/input.md+++ b/docs/input.md@@ -36,11 +36,13 @@ Events change the live state at any time, but game code reads a snapshot taken b| --- | --- || `isHeld(action)` | Any bound control is down || `wasPressed(action)`, `wasReleased(action)` | Went down or up since the previous sample |-| `getAxis(axis)` | Sum of `value * scale` over the bindings, clamped to -1 to 1, then dead zone applied |+| `getAxis(axis)` | Sum of `value * scale` over the bindings, clamped to -1 to 1, dead zone applied, then smoothing |## Actions, axes and rebinding-`bindAction(name, controls)` and `bindAxis(name, { bindings, deadZone })` add or replace a mapping at any time, which is rebinding. A binding is a control name or `{ control, scale }`. `unbindAction` and `unbindAxis` remove one. Reading a name that was never bound throws an error that shows the `bindAction` call to add.+`bindAction(name, controls)` and `bindAxis(name, { bindings, deadZone })` add or replace a mapping at any time, which is rebinding. A binding is a control name or `{ control, scale }`. `unbindAction` and `unbindAxis` remove one. `bindAxis` also takes `smoothing` from 0 (default, none) up to but not including 1: each sample the axis moves `1 - smoothing` of the way to its target, so `0.5` halves the gap per fixed step. It is per sample, so it stays deterministic for a replay. Rebinding resets it.++Reading a name that was never bound throws an error that shows the `bindAction` call to add.## Built-in sources@@ -48,7 +50,10 @@ Events change the live state at any time, but game code reads a snapshot taken b- `mouseSource({ target, pointerLockElement })`: `mouse/button0..`, `mouse/x`, `mouse/y` and `mouse/deltaX`, `mouse/deltaY`, which sum movement since the last sample. Pointer events cover touch and pen as `button0`. Pass `pointerLockElement` to request pointer lock on press.- `gamepadSource({ getGamepads })`: polled each sample, `gamepad/<index>.button<n>` and `gamepad/<index>.axis<n>`. An unplugged pad reads 0.-Keyboard and mouse release everything held when the window loses focus.+- `touchSource({ target, sticks })`: virtual sticks and gestures from touch pointers (mouse and pen are ignored). A stick is `{ name, area: { x, y, width, height }, radius }`, the area as fractions of the target (`{ x: 0, y: 0, width: 0.5, height: 1 }` is the left half) and `radius` in pixels (default 60). The finger's first position is the centre; `touch/<name>X`, `touch/<name>Y` read -1 to 1 (down is +1) and `touch/<name>Active` is 1 while held. Touches outside every free stick are gestures: `touch/tap`, `touch/swipeLeft|Right|Up|Down` pulse for one sample, and `touch/pinch` sums the spread ratio change of two fingers since the last sample. Tune with `tapMaximumMilliseconds`, `tapMaximumDistance`, `swipeMinimumDistance`, `swipeMaximumMilliseconds`. Up to 4 sticks.+- `deviceMotionSource({ target })`: `motion/accelX|Y|Z` (m/s², with gravity), `motion/rotationAlpha|Beta|Gamma` (deg/s), `motion/alpha|beta|gamma` (degrees) and `motion/tiltX`, `motion/tiltY` (gamma and beta over 45°, clamped to -1 to 1). On iOS call `await source.requestPermission()` from a tap handler; it returns false when the user refuses.++Keyboard, mouse and touch release everything held when the window loses focus.## Custom and scripted input@@ -65,8 +70,24 @@ input.bindAction('kick', ['midi/note36']);Tests, replays and agents drive a game through the same path with `input.press('script/start')`, `input.release(...)` or `input.setControl(name, value)`.+## Recording and replay++`recordInput(input)` logs every control change with the sample number it belongs to. `stop()` returns plain JSON: `{ version: 1, samples, truncated, events: [[sample, "keyboard/Space", 1], ...] }`. `replaySource(recording, { loop })` writes those changes back during the same sample numbers under their original control names, so the game's bindings and code run unchanged, a press and release inside one sample included.++```js+const recorder = recordInput(input); // while a player plays+const recording = recorder.stop(); // save with JSON.stringify+const replay = replaySource(recording);+const headless = createInput({ sources: [replay], actions, axes }); // same bindings, no live sources+// headless.fixedUpdate() each step; replay.finished turns true after the last sample+```++Use an input without live sources for the replay, or live devices mix in. When the recording ends the replay releases what it set. A recording holds up to 100000 events; past that `truncated` is true. Recording is only as deterministic as the game: it stores input, not game state.++`input.onControlChange(listener)` and `input.onSample(listener)` are the hooks it uses; each returns a function that removes the listener. A custom source can write any full control name with `emit.control("keyboard/Space", 1)`.+## Limits-16 sources, 512 controls, 128 actions, 64 axes, 16 bindings each (`INPUT_LIMITS`). Errors name the field and say what to pass.+16 sources, 512 controls, 128 actions, 64 axes, 16 bindings each (`INPUT_LIMITS`), 4 touch sticks and 100000 recorded events (`INPUT_SOURCE_LIMITS`). Errors name the field and say what to pass.A runnable sample is `bench/runtime-loop/index.html`: A/D or the arrow keys slide the cube, Space kicks the ball.diff --git a/src/input-sources.js b/src/input-sources.jsnew file mode 100644index 00000000..6e987c0d--- /dev/null+++ b/src/input-sources.js@@ -0,0 +1,335 @@+// Input sources beyond the basics: touch sticks and gestures, device motion, and input recording with replay.+// Each source follows the contract in input.js: { name, attach(emit), poll(emit), endSample(emit), dispose() }.+// This file does not import input.js, so input.js can re-export it. See docs/input.md.++export const INPUT_SOURCE_LIMITS = Object.freeze({+ maximumTouchSticks: 4,+ maximumRecordedEvents: 100000,+});++function describe(value) {+ return value === null ? 'null' : typeof value;+}++function requireName(value, label) {+ if (typeof value !== 'string' || value === '') {+ throw new TypeError(`${label} must be a non-empty string, got ${describe(value)}.`);+ }+ return value;+}++function requirePositive(value, label) {+ if (!Number.isFinite(value) || value <= 0) {+ throw new RangeError(`${label} must be a number above 0, got ${String(value)}.`);+ }+ return value;+}++function assertTarget(target, label) {+ if (!target || typeof target.addEventListener !== 'function' || typeof target.removeEventListener !== 'function') {+ throw new TypeError(`${label} must have addEventListener and removeEventListener, such as window or a canvas.`);+ }+}++function clampUnit(value) {+ return Math.max(-1, Math.min(1, value));+}++function normalizeStick(stick, index) {+ if (stick === null || typeof stick !== 'object') {+ throw new TypeError(`touchSource sticks[${index}] must be { name, area: { x, y, width, height }, radius }, got ${describe(stick)}.`);+ }+ const name = requireName(stick.name, `touchSource sticks[${index}] name`);+ const area = stick.area;+ for (const key of ['x', 'y', 'width', 'height']) {+ if (area === null || typeof area !== 'object' || !Number.isFinite(area[key])) {+ throw new TypeError(`touchSource stick "${name}" area needs numbers x, y, width and height as fractions of the target, such as { x: 0, y: 0, width: 0.5, height: 1 } for the left half.`);+ }+ }+ return { name, area, radius: requirePositive(stick.radius ?? 60, `touchSource stick "${name}" radius`) };+}++// Touch: virtual sticks and gestures from pointer events with pointerType "touch" (mouse and pen are ignored).+// A stick is a part of the target (area, as fractions of its size). The first touch that starts inside it sets the centre;+// dragging up to `radius` pixels gives "touch/<stick>X" and "touch/<stick>Y" from -1 to 1 (down is +1 on Y), and+// "touch/<stick>Active" is 1 while held. A touch outside every free stick is a gesture touch:+// "touch/tap" pulses 1 on a short, small touch; "touch/swipeLeft|Right|Up|Down" pulse on a fast, long drag;+// "touch/pinch" sums the ratio change between two gesture touches since the last sample (0.1 = spread 10 percent).+export function touchSource(options = {}) {+ const target = options.target ?? globalThis;+ const blurTarget = options.blurTarget ?? globalThis;+ const sticks = (options.sticks ?? []).map(normalizeStick);+ if (sticks.length > INPUT_SOURCE_LIMITS.maximumTouchSticks) {+ throw new RangeError(`touchSource takes up to ${INPUT_SOURCE_LIMITS.maximumTouchSticks} sticks, got ${sticks.length}.`);+ }+ const tapMaximumMilliseconds = requirePositive(options.tapMaximumMilliseconds ?? 250, 'touchSource tapMaximumMilliseconds');+ const tapMaximumDistance = requirePositive(options.tapMaximumDistance ?? 10, 'touchSource tapMaximumDistance');+ const swipeMinimumDistance = requirePositive(options.swipeMinimumDistance ?? 50, 'touchSource swipeMinimumDistance');+ const swipeMaximumMilliseconds = requirePositive(options.swipeMaximumMilliseconds ?? 400, 'touchSource swipeMaximumMilliseconds');+ const stickTouches = new Map(); // pointerId -> { stick, originX, originY }+ const gestureTouches = new Map(); // pointerId -> { startX, startY, startTime, x, y }+ let pulses = new Set();+ let pinchTotal = 0;+ let handlers = null;+ const eventTypes = ['pointerdown', 'pointermove', 'pointerup', 'pointercancel'];++ function bounds() {+ const rectangle = target.getBoundingClientRect?.();+ if (rectangle) return rectangle;+ return { left: 0, top: 0, width: target.innerWidth ?? 1, height: target.innerHeight ?? 1 };+ }++ function findFreeStick(x, y) {+ const { left, top, width, height } = bounds();+ return sticks.find(({ area, name }) => {+ const minimumX = left + area.x * width;+ const minimumY = top + area.y * height;+ const inside = x >= minimumX && x <= minimumX + area.width * width && y >= minimumY && y <= minimumY + area.height * height;+ const held = [...stickTouches.values()].some(touch => touch.stick.name === name);+ return inside && !held;+ });+ }++ function gestureDistance() {+ const [first, second] = [...gestureTouches.values()];+ return Math.hypot(first.x - second.x, first.y - second.y);+ }++ function pulse(emit, name) {+ pulses.add(name);+ emit(name, 1);+ }++ function finishGesture(emit, touch, event) {+ const deltaX = event.clientX - touch.startX;+ const deltaY = event.clientY - touch.startY;+ const distance = Math.hypot(deltaX, deltaY);+ const elapsed = (event.timeStamp ?? 0) - touch.startTime;+ if (distance <= tapMaximumDistance && elapsed <= tapMaximumMilliseconds) {+ pulse(emit, 'tap');+ return;+ }+ if (distance < swipeMinimumDistance || elapsed > swipeMaximumMilliseconds) return;+ if (Math.abs(deltaX) >= Math.abs(deltaY)) {+ pulse(emit, deltaX < 0 ? 'swipeLeft' : 'swipeRight');+ } else {+ pulse(emit, deltaY < 0 ? 'swipeUp' : 'swipeDown');+ }+ }++ function handlePointerDown(emit, event) {+ if (event.pointerType !== 'touch') return;+ const stick = findFreeStick(event.clientX, event.clientY);+ if (stick) {+ stickTouches.set(event.pointerId, { stick, originX: event.clientX, originY: event.clientY });+ emit(`${stick.name}Active`, 1);+ return;+ }+ gestureTouches.set(event.pointerId, {+ startX: event.clientX,+ startY: event.clientY,+ startTime: event.timeStamp ?? 0,+ x: event.clientX,+ y: event.clientY,+ });+ }++ function handlePointerMove(emit, event) {+ const stickTouch = stickTouches.get(event.pointerId);+ if (stickTouch) {+ emit(`${stickTouch.stick.name}X`, clampUnit((event.clientX - stickTouch.originX) / stickTouch.stick.radius));+ emit(`${stickTouch.stick.name}Y`, clampUnit((event.clientY - stickTouch.originY) / stickTouch.stick.radius));+ return;+ }+ const gestureTouch = gestureTouches.get(event.pointerId);+ if (!gestureTouch) return;+ const distanceBefore = gestureTouches.size === 2 ? gestureDistance() : 0;+ gestureTouch.x = event.clientX;+ gestureTouch.y = event.clientY;+ if (distanceBefore > 0) {+ pinchTotal += gestureDistance() / distanceBefore - 1;+ emit('pinch', pinchTotal);+ }+ }++ function handlePointerUp(emit, event) {+ const stickTouch = stickTouches.get(event.pointerId);+ if (stickTouch) {+ stickTouches.delete(event.pointerId);+ emit(`${stickTouch.stick.name}Active`, 0);+ emit(`${stickTouch.stick.name}X`, 0);+ emit(`${stickTouch.stick.name}Y`, 0);+ return;+ }+ const gestureTouch = gestureTouches.get(event.pointerId);+ if (!gestureTouch) return;+ gestureTouches.delete(event.pointerId);+ if (gestureTouches.size === 0) finishGesture(emit, gestureTouch, event);+ }++ return {+ name: 'touch',+ attach(emit) {+ assertTarget(target, 'touchSource target');+ handlers = {+ pointerdown: event => handlePointerDown(emit, event),+ pointermove: event => handlePointerMove(emit, event),+ pointerup: event => handlePointerUp(emit, event),+ pointercancel: event => handlePointerUp(emit, event),+ blur: () => {+ stickTouches.clear();+ gestureTouches.clear();+ emit.releaseAll();+ },+ };+ for (const type of eventTypes) target.addEventListener(type, handlers[type]);+ blurTarget.addEventListener?.('blur', handlers.blur);+ },+ endSample(emit) {+ for (const name of pulses) emit(name, 0);+ pulses = new Set();+ if (pinchTotal !== 0) emit('pinch', 0);+ pinchTotal = 0;+ },+ dispose() {+ if (handlers === null) return;+ for (const type of eventTypes) target.removeEventListener(type, handlers[type]);+ blurTarget.removeEventListener?.('blur', handlers.blur);+ handlers = null;+ },+ };+}++// Device motion: "motion/accelX|Y|Z" (acceleration with gravity, m/s^2), "motion/rotationAlpha|Beta|Gamma" (deg/s),+// "motion/alpha|beta|gamma" (orientation, degrees) and "motion/tiltX", "motion/tiltY" (gamma and beta over 45 degrees,+// clamped to -1 to 1: a phone held flat reads 0, tipped right or toward you reads +1).+// iOS needs a user gesture to allow sensors: call source.requestPermission() from a tap handler.+export function deviceMotionSource(options = {}) {+ const target = options.target ?? globalThis;+ let handlers = null;+ return {+ name: 'motion',+ attach(emit) {+ assertTarget(target, 'deviceMotionSource target');+ const emitFinite = (name, value) => {+ if (Number.isFinite(value)) emit(name, value);+ };+ handlers = {+ devicemotion: event => {+ const acceleration = event.accelerationIncludingGravity ?? {};+ emitFinite('accelX', acceleration.x);+ emitFinite('accelY', acceleration.y);+ emitFinite('accelZ', acceleration.z);+ const rotation = event.rotationRate ?? {};+ emitFinite('rotationAlpha', rotation.alpha);+ emitFinite('rotationBeta', rotation.beta);+ emitFinite('rotationGamma', rotation.gamma);+ },+ deviceorientation: event => {+ emitFinite('alpha', event.alpha);+ emitFinite('beta', event.beta);+ emitFinite('gamma', event.gamma);+ if (Number.isFinite(event.gamma)) emit('tiltX', clampUnit(event.gamma / 45));+ if (Number.isFinite(event.beta)) emit('tiltY', clampUnit(event.beta / 45));+ },+ };+ for (const type of Object.keys(handlers)) target.addEventListener(type, handlers[type]);+ },+ async requestPermission() {+ const eventClasses = [globalThis.DeviceMotionEvent, globalThis.DeviceOrientationEvent]+ .filter(eventClass => typeof eventClass?.requestPermission === 'function');+ for (const eventClass of eventClasses) {+ if (await eventClass.requestPermission() !== 'granted') return false;+ }+ return true;+ },+ dispose() {+ if (handlers === null) return;+ for (const type of Object.keys(handlers)) target.removeEventListener(type, handlers[type]);+ handlers = null;+ },+ };+}++// Recording: logs every control change with the number of the sample it belongs to, as plain JSON:+// { version: 1, samples, truncated, events: [[sampleIndex, "source/control", value], ...] }.+// Call stop() to get the recording. Past maximumRecordedEvents it stops logging and sets truncated.+export function recordInput(input) {+ if (input === null || typeof input !== 'object' || typeof input.onControlChange !== 'function' || typeof input.onSample !== 'function') {+ throw new TypeError('recordInput needs an input made by createInput.');+ }+ const events = [];+ let sampleIndex = 0;+ let truncated = false;+ const stopListening = input.onControlChange((controlName, value) => {+ if (events.length >= INPUT_SOURCE_LIMITS.maximumRecordedEvents) {+ truncated = true;+ return;+ }+ events.push([sampleIndex, controlName, value]);+ });+ const stopSampling = input.onSample(() => {+ sampleIndex += 1;+ });+ return {+ get sampleCount() {+ return sampleIndex;+ },+ stop() {+ stopListening();+ stopSampling();+ return { version: 1, samples: sampleIndex, truncated, events: events.map(event => [...event]) };+ },+ };+}++function validateRecording(recording) {+ const valid = recording !== null && typeof recording === 'object' && recording.version === 1+ && Array.isArray(recording.events) && Number.isInteger(recording.samples) && recording.samples >= 0;+ if (!valid) throw new TypeError('replaySource needs a recording from recordInput(input).stop(): { version: 1, samples, events }.');+ if (recording.events.length > INPUT_SOURCE_LIMITS.maximumRecordedEvents) {+ throw new RangeError(`A recording holds up to ${INPUT_SOURCE_LIMITS.maximumRecordedEvents} events, got ${recording.events.length}.`);+ }+ let previousSample = 0;+ recording.events.forEach((event, index) => {+ const wellFormed = Array.isArray(event) && event.length === 3 && Number.isInteger(event[0]) && event[0] >= previousSample+ && typeof event[1] === 'string' && event[1] !== '' && Number.isFinite(event[2]);+ if (!wellFormed) {+ throw new TypeError(`Recording event ${index} must be [sampleIndex, "source/control", value] with sample indexes that do not go down, got ${JSON.stringify(event)}.`);+ }+ previousSample = event[0];+ });+}++// Replay: writes the recorded changes back during the same sample numbers, under their original control names, so the+// game's bindings and code run unchanged. Use it with an input that has no live sources, or live input mixes in.+// When the recording ends the replay releases what it set; `finished` turns true, and `loop: true` starts over.+export function replaySource(recording, options = {}) {+ validateRecording(recording);+ const loop = options.loop === true;+ const touched = new Set();+ let nextEvent = 0;+ let sampleIndex = 0;+ return {+ name: options.name ?? 'replay',+ get finished() {+ return !loop && sampleIndex > recording.samples;+ },+ poll(emit) {+ if (sampleIndex >= recording.samples) {+ for (const controlName of touched) emit.control(controlName, 0);+ touched.clear();+ sampleIndex = loop ? 0 : recording.samples + 1;+ nextEvent = 0;+ return;+ }+ while (nextEvent < recording.events.length && recording.events[nextEvent][0] <= sampleIndex) {+ const [, controlName, value] = recording.events[nextEvent];+ emit.control(controlName, value);+ touched.add(controlName);+ nextEvent += 1;+ }+ sampleIndex += 1;+ },+ };+}diff --git a/src/input.js b/src/input.jsindex 50cb1ce2..44ef70cd 100644--- a/src/input.js+++ b/src/input.js@@ -4,6 +4,8 @@// Events write the live control state at any time; the snapshot game logic reads changes only when the runtime samples,// so a fixed-step game sees the same state for the whole step. See docs/input.md.+export { deviceMotionSource, INPUT_SOURCE_LIMITS, recordInput, replaySource, touchSource } from './input-sources.js';+export const INPUT_LIMITS = Object.freeze({maximumSources: 16,maximumControls: 512,@@ -61,6 +63,8 @@ export function createInput(options = {}) {let started = false;let disposed = false;let snapshot = { actions: new Map(), axes: new Map() };+ const controlListeners = new Set();+ const sampleListeners = new Set();function assertAlive() {if (disposed) throw new Error('This input was disposed. Call createInput again.');@@ -85,6 +89,13 @@ export function createInput(options = {}) {if (isDown && !wasDown) control.pressedLatch = true;if (!isDown && wasDown) control.releasedLatch = true;control.value = value;+ for (const listener of [...controlListeners]) listener(controlName, value);+ }++ function addListener(listeners, listener, label) {+ if (typeof listener !== 'function') throw new TypeError(`${label} needs a function, got ${describe(listener)}.`);+ listeners.add(listener);+ return () => listeners.delete(listener);}function releaseControls(sourceName) {@@ -121,6 +132,7 @@ export function createInput(options = {}) {}const emit = (controlName, value) => setControl(`${name}/${controlName}`, value);emit.releaseAll = () => releaseControls(name);+ emit.control = setControl; // writes any full control name, for sources such as a replay that restore recorded controlsconst entry = { source, emit, attached: false };sources.set(name, entry);if (started) attach(entry);@@ -154,13 +166,17 @@ export function createInput(options = {}) {requireName(name, 'bindAxis name');const bindings = Array.isArray(definition) ? definition : definition?.bindings;const deadZone = (Array.isArray(definition) ? undefined : definition?.deadZone) ?? INPUT_LIMITS.defaultDeadZone;+ const smoothing = (Array.isArray(definition) ? undefined : definition?.smoothing) ?? 0;+ if (!Number.isFinite(smoothing) || smoothing < 0 || smoothing >= 1) {+ throw new RangeError(`Axis "${name}" smoothing must be a number from 0 (none) up to but not including 1, got ${String(smoothing)}. Each sample moves the axis (1 - smoothing) of the way to its target.`);+ }if (!Number.isFinite(deadZone) || deadZone < 0 || deadZone >= 1) {throw new RangeError(`Axis "${name}" deadZone must be a number from 0 up to but not including 1, got ${String(deadZone)}.`);}if (!axisBindings.has(name) && axisBindings.size >= INPUT_LIMITS.maximumAxes) {throw new RangeError(`Input already has ${INPUT_LIMITS.maximumAxes} axes. Reuse a name or unbindAxis first.`);}- axisBindings.set(name, { bindings: normalizeBindings(bindings, `Axis "${name}" bindings`), deadZone });+ axisBindings.set(name, { bindings: normalizeBindings(bindings, `Axis "${name}" bindings`), deadZone, smoothing, smoothed: 0 });return api;}@@ -180,12 +196,14 @@ export function createInput(options = {}) {actions.set(name, state);}const axes = new Map();- for (const [name, { bindings, deadZone }] of axisBindings) {+ for (const [name, axis] of axisBindings) {let total = 0;- for (const { control: controlName, scale } of bindings) {+ for (const { control: controlName, scale } of axis.bindings) {total += (controls.get(controlName)?.value ?? 0) * scale;}- axes.set(name, applyDeadZone(Math.max(-1, Math.min(1, total)), deadZone));+ const target = applyDeadZone(Math.max(-1, Math.min(1, total)), axis.deadZone);+ axis.smoothed = axis.smoothing === 0 ? target : axis.smoothed + (target - axis.smoothed) * (1 - axis.smoothing);+ axes.set(name, axis.smoothed);}for (const control of controls.values()) {control.pressedLatch = false;@@ -193,6 +211,7 @@ export function createInput(options = {}) {}for (const entry of sources.values()) entry.source.endSample?.(entry.emit);snapshot = { actions, axes };+ for (const listener of [...sampleListeners]) listener();}function actionState(name) {@@ -221,6 +240,8 @@ export function createInput(options = {}) {actionBindings.clear();axisBindings.clear();snapshot = { actions: new Map(), axes: new Map() };+ controlListeners.clear();+ sampleListeners.clear();disposed = true;}@@ -237,6 +258,10 @@ export function createInput(options = {}) {release: controlName => setControl(controlName, 0),releaseControls,sample,+ // Observers for recording: the first hears every control change, the second the end of every sample.+ // Each returns a function that removes the listener.+ onControlChange: listener => addListener(controlListeners, listener, 'onControlChange'),+ onSample: listener => addListener(sampleListeners, listener, 'onSample'),isHeld: name => actionState(name).held,wasPressed: name => actionState(name).pressed,wasReleased: name => actionState(name).released,diff --git a/tests/input-touch-replay.test.js b/tests/input-touch-replay.test.jsnew file mode 100644index 00000000..235c729c--- /dev/null+++ b/tests/input-touch-replay.test.js@@ -0,0 +1,299 @@+import assert from 'node:assert/strict';+import test from 'node:test';+import {+ createInput,+ deviceMotionSource,+ gamepadSource,+ keyboardSource,+ recordInput,+ replaySource,+ touchSource,+} from '../src/input.js';++function createFakeTarget(size = { width: 800, height: 400 }) {+ const listeners = new Map();+ return {+ innerWidth: size.width,+ innerHeight: size.height,+ addEventListener(type, handler) {+ if (!listeners.has(type)) listeners.set(type, new Set());+ listeners.get(type).add(handler);+ },+ removeEventListener(type, handler) {+ listeners.get(type)?.delete(handler);+ },+ dispatch(type, event = {}) {+ for (const handler of [...(listeners.get(type) ?? [])]) handler(event);+ },+ listenerCount() {+ let count = 0;+ for (const set of listeners.values()) count += set.size;+ return count;+ },+ };+}++const leftStick = { name: 'left', area: { x: 0, y: 0, width: 0.5, height: 1 }, radius: 100 };++function touchInput(extra = {}) {+ const target = createFakeTarget();+ const blurTarget = createFakeTarget();+ const input = createInput({+ sources: [touchSource({ target, blurTarget, sticks: [leftStick], ...extra })],+ actions: { tap: ['touch/tap'], swipeRight: ['touch/swipeRight'], swipeUp: ['touch/swipeUp'] },+ axes: {+ moveX: { bindings: ['touch/leftX'], deadZone: 0 },+ moveY: { bindings: ['touch/leftY'], deadZone: 0 },+ zoom: { bindings: ['touch/pinch'], deadZone: 0 },+ },+ });+ input.init();+ return { target, blurTarget, input };+}++function touch(pointerId, clientX, clientY, timeStamp = 0) {+ return { pointerType: 'touch', pointerId, clientX, clientY, timeStamp };+}++test('a touch stick reads from where the finger first landed and releases on lift', () => {+ const { target, input } = touchInput();+ target.dispatch('pointerdown', touch(1, 100, 200));+ target.dispatch('pointermove', touch(1, 150, 100));+ input.sample();+ assert.equal(input.getAxis('moveX'), 0.5);+ assert.equal(input.getAxis('moveY'), -1);++ target.dispatch('pointermove', touch(1, 900, 200));+ input.sample();+ assert.equal(input.getAxis('moveX'), 1, 'clamped to the radius');++ target.dispatch('pointerup', touch(1, 900, 200));+ input.sample();+ assert.equal(input.getAxis('moveX'), 0);+});++test('mouse and pen pointers do not drive touch', () => {+ const { target, input } = touchInput();+ target.dispatch('pointerdown', { pointerType: 'mouse', pointerId: 1, clientX: 100, clientY: 200 });+ target.dispatch('pointermove', { pointerType: 'mouse', pointerId: 1, clientX: 180, clientY: 200 });+ input.sample();+ assert.equal(input.getAxis('moveX'), 0);+});++test('a touch outside the stick area is a gesture: tap and swipes pulse for one sample', () => {+ const { target, input } = touchInput();+ target.dispatch('pointerdown', touch(2, 600, 200, 1000));+ target.dispatch('pointerup', touch(2, 602, 200, 1100));+ input.sample();+ assert.equal(input.wasPressed('tap'), true);+ input.sample();+ assert.equal(input.wasPressed('tap'), false);+ assert.equal(input.isHeld('tap'), false);++ target.dispatch('pointerdown', touch(3, 500, 300, 2000));+ target.dispatch('pointermove', touch(3, 600, 300, 2100));+ target.dispatch('pointerup', touch(3, 650, 300, 2150));+ input.sample();+ assert.equal(input.wasPressed('swipeRight'), true);+ assert.equal(input.wasPressed('tap'), false);++ target.dispatch('pointerdown', touch(4, 500, 350, 3000));+ target.dispatch('pointerup', touch(4, 500, 250, 3100));+ input.sample();+ assert.equal(input.wasPressed('swipeUp'), true);+});++test('a slow drag is neither a tap nor a swipe', () => {+ const { target, input } = touchInput();+ target.dispatch('pointerdown', touch(5, 500, 300, 0));+ target.dispatch('pointerup', touch(5, 700, 300, 2000));+ input.sample();+ assert.equal(input.wasPressed('tap'), false);+ assert.equal(input.wasPressed('swipeRight'), false);+});++test('two gesture touches spreading apart report a pinch that clears each sample', () => {+ const { target, input } = touchInput();+ target.dispatch('pointerdown', touch(6, 500, 200));+ target.dispatch('pointerdown', touch(7, 600, 200));+ target.dispatch('pointermove', touch(7, 700, 200));+ input.sample();+ assert.equal(input.getAxis('zoom'), 1, 'distance went 100 to 200');+ input.sample();+ assert.equal(input.getAxis('zoom'), 0);+});++test('a second finger inside a held stick area becomes a gesture touch, and blur releases everything', () => {+ const { target, blurTarget, input } = touchInput();+ target.dispatch('pointerdown', touch(1, 100, 200));+ target.dispatch('pointermove', touch(1, 200, 200));+ target.dispatch('pointerdown', touch(2, 120, 220));+ input.sample();+ assert.equal(input.getAxis('moveX'), 1);+ blurTarget.dispatch('blur');+ input.sample();+ assert.equal(input.getAxis('moveX'), 0);+});++test('touch source options and removal are validated', () => {+ assert.throws(() => touchSource({ sticks: [{ name: 'left' }] }), /area needs numbers x, y, width and height/);+ assert.throws(() => touchSource({ sticks: [{ ...leftStick, radius: 0 }] }), /radius must be a number above 0/);+ assert.throws(() => touchSource({ sticks: Array.from({ length: 5 }, (_, index) => ({ ...leftStick, name: `s${index}` })) }), /up to 4 sticks/);+ const { target, input } = touchInput();+ assert.ok(target.listenerCount() > 0);+ input.dispose();+ assert.equal(target.listenerCount(), 0);+});++test('device motion reports tilt and raw sensors, ignoring missing values', () => {+ const target = createFakeTarget();+ const input = createInput({+ sources: [deviceMotionSource({ target })],+ axes: { steer: { bindings: ['motion/tiltX'], deadZone: 0 }, pitch: { bindings: ['motion/tiltY'], deadZone: 0 } },+ actions: { shake: ['motion/accelX'] },+ });+ input.init();+ target.dispatch('deviceorientation', { alpha: 10, beta: -90, gamma: 22.5 });+ target.dispatch('devicemotion', { accelerationIncludingGravity: { x: 3, y: null, z: 9.8 }, rotationRate: {} });+ input.sample();+ assert.equal(input.getAxis('steer'), 0.5);+ assert.equal(input.getAxis('pitch'), -1, 'clamped');+ assert.equal(input.isHeld('shake'), true);+ input.dispose();+ assert.equal(target.listenerCount(), 0);+});++test('axis smoothing eases toward the target each sample and is validated', () => {+ const target = createFakeTarget();+ const input = createInput({+ sources: [keyboardSource({ target, blurTarget: createFakeTarget() })],+ axes: { moveX: { bindings: ['keyboard/KeyD'], deadZone: 0, smoothing: 0.5 } },+ });+ input.init();+ target.dispatch('keydown', { code: 'KeyD' });+ const readings = [];+ for (let step = 0; step < 3; step += 1) {+ input.sample();+ readings.push(input.getAxis('moveX'));+ }+ assert.deepEqual(readings, [0.5, 0.75, 0.875]);+ target.dispatch('keyup', { code: 'KeyD' });+ input.sample();+ assert.equal(input.getAxis('moveX'), 0.4375);+ assert.throws(() => input.bindAxis('bad', { bindings: ['keyboard/KeyA'], smoothing: 1 }), /smoothing must be a number from 0/);+});++// One tiny game, driven only through named actions and axes.+function playDemo(input, steps) {+ const state = { x: 0, jumps: 0 };+ for (let step = 0; step < steps; step += 1) {+ input.fixedUpdate();+ state.x += input.getAxis('moveX');+ if (input.wasPressed('jump')) state.jumps += 1;+ }+ return state;+}++const demoBindings = {+ actions: { jump: ['keyboard/Space', 'gamepad/0.button0', 'touch/tap'] },+ axes: {+ moveX: { bindings: ['keyboard/KeyD', 'gamepad/0.axis0', 'touch/leftX'], deadZone: 0 },+ },+};++test('the same demo plays from keyboard, gamepad, touch and a scripted replay', () => {+ const keyboardTarget = createFakeTarget();+ const keyboardInput = createInput({ sources: [keyboardSource({ target: keyboardTarget, blurTarget: createFakeTarget() })], ...demoBindings });+ keyboardInput.init();+ keyboardTarget.dispatch('keydown', { code: 'KeyD' });+ keyboardTarget.dispatch('keydown', { code: 'Space' });+ const fromKeyboard = playDemo(keyboardInput, 3);++ const pad = { index: 0, buttons: [{ pressed: true, value: 1 }], axes: [1] };+ const gamepadInput = createInput({ sources: [gamepadSource({ getGamepads: () => [pad] })], ...demoBindings });+ gamepadInput.init();+ const fromGamepad = playDemo(gamepadInput, 3);++ const touchTarget = createFakeTarget();+ const touchInputInstance = createInput({ sources: [touchSource({ target: touchTarget, blurTarget: createFakeTarget(), sticks: [leftStick] })], ...demoBindings });+ touchInputInstance.init();+ touchTarget.dispatch('pointerdown', touch(1, 100, 200));+ touchTarget.dispatch('pointermove', touch(1, 200, 200));+ const fromTouch = playDemo(touchInputInstance, 3);++ assert.deepEqual(fromKeyboard, { x: 3, jumps: 1 });+ assert.deepEqual(fromGamepad, fromKeyboard);+ assert.equal(fromTouch.x, 3);++ const recordingTarget = createFakeTarget();+ const recordedInput = createInput({ sources: [keyboardSource({ target: recordingTarget, blurTarget: createFakeTarget() })], ...demoBindings });+ recordedInput.init();+ const recorder = recordInput(recordedInput);+ recordingTarget.dispatch('keydown', { code: 'KeyD' });+ recordingTarget.dispatch('keydown', { code: 'Space' });+ recordingTarget.dispatch('keyup', { code: 'Space' }); // pressed and released inside one sample+ const live = playDemo(recordedInput, 2);+ recordingTarget.dispatch('keyup', { code: 'KeyD' });+ playDemo(recordedInput, 2);+ const recording = JSON.parse(JSON.stringify(recorder.stop()));++ const replayInput = createInput({ sources: [replaySource(recording)], ...demoBindings });+ replayInput.init();+ const replayed = playDemo(replayInput, 2);+ assert.deepEqual(replayed, live);+ assert.deepEqual(live, { x: 2, jumps: 1 });+});++test('replay releases what it held when the recording ends, can loop, and rejects bad recordings', () => {+ const recording = { version: 1, samples: 2, truncated: false, events: [[0, 'script/go', 1], [1, 'script/go', 0]] };+ const source = replaySource(recording);+ const input = createInput({ sources: [source], actions: { go: ['script/go'] } });+ input.init();+ input.sample();+ assert.equal(input.isHeld('go'), true);+ input.sample();+ assert.equal(input.isHeld('go'), false);+ assert.equal(source.finished, false);+ input.sample();+ assert.equal(source.finished, true);++ const heldForever = replaySource({ version: 1, samples: 1, events: [[0, 'script/go', 1]] });+ const heldInput = createInput({ sources: [heldForever], actions: { go: ['script/go'] } });+ heldInput.init();+ heldInput.sample();+ assert.equal(heldInput.isHeld('go'), true);+ heldInput.sample();+ assert.equal(heldInput.isHeld('go'), false, 'released at the end');++ const looping = replaySource(recording, { loop: true });+ const loopInput = createInput({ sources: [looping], actions: { go: ['script/go'] } });+ loopInput.init();+ const held = [];+ for (let step = 0; step < 6; step += 1) {+ loopInput.sample();+ held.push(loopInput.isHeld('go'));+ }+ assert.deepEqual(held, [true, false, false, true, false, false]);++ assert.throws(() => replaySource({}), /needs a recording from recordInput/);+ assert.throws(() => replaySource({ version: 1, samples: 1, events: [[0, 'a/b', 'x']] }), /Recording event 0 must be/);+ assert.throws(() => replaySource({ version: 1, samples: 2, events: [[1, 'a/b', 1], [0, 'a/b', 0]] }), /do not go down/);+ assert.throws(() => recordInput({}), /needs an input made by createInput/);+});++test('recording stops logging after stop and truncates past the event limit', () => {+ const input = createInput({});+ const recorder = recordInput(input);+ input.press('script/a');+ input.sample();+ const first = recorder.stop();+ input.release('script/a');+ assert.deepEqual(first.events, [[0, 'script/a', 1]]);+ assert.equal(first.samples, 1);+ assert.equal(first.truncated, false);+ assert.throws(() => input.onControlChange('nope'), /onControlChange needs a function/);++ const busy = recordInput(input);+ for (let index = 0; index < 100001; index += 1) input.setControl('script/n', index % 2);+ assert.equal(busy.stop().truncated, true);+});