Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
JavaScript 3d-generationai-agentsanimationavatardigital-humangenerative-aiglbgltfimage-to-3dllmmcpmcp-servermodel-context-protocolsolanatext-to-3dthreejsweb-componentswebglwebxrx402
// Motion signatures: what a baked clip actually does, measured from its keyframes.
//
// Every clip in public/animations/clips/*.json is a THREE.AnimationClip with
// named bone tracks on the canonical skeleton (src/glb-canonicalize.js maps any
// rig onto it). That means the motion is not opaque: the rotation of every bone
// over time is right there, and questions people keep answering by eye can be
// answered by arithmetic instead.
//
// • Where does this clip move? (head, arms, torso, legs, root)
// • How energetic is it, and does it have a beat?
// • Does it travel across the floor, or stay on the spot?
// • Does its last frame meet its first, i.e. can it loop without a snap?
// • Can it overlay a walk cycle, or does it need the whole body?
//
// Those answers drive real behaviour, not decoration: the /walk gesture layer
// table is checked against measured overlay-safety in tests/motion-signature
// .test.js, and the /gestures override picker refuses to quietly hand a slot a
// clip that cannot do the job that slot is for.
//
// Dependency-free on purpose (no Three.js, no DOM, no fs) for the same reason
// animation-slots.js is: the build script, the tests and the browser all import
// this one file, so they can never disagree about what a signature means.
/** Bumped when a field changes meaning. The build script stamps it into the index. */
export const SIGNATURE_VERSION = 1;
/** Body regions, in head-to-toe order (the order the UI renders them in). */
export const REGIONS = ['head', 'arms', 'torso', 'root', 'legs'];
/** Human labels for the regions, so every surface names them identically. */
export const REGION_LABELS = {
head: 'Head',
arms: 'Arms',
torso: 'Torso',
root: 'Hips',
legs: 'Legs',
};
/**
* Fewer frames than this is a pose, not an animation. Single-frame clips do
* exist in the library (they were exported from a static pose) and they must be
* flagged rather than silently scored as "perfectly calm".
*/
const MIN_ANIMATED_FRAMES = 2;
/** Below this mean angular speed (rad/s) nothing visible is happening. */
const STATIC_SPEED = 0.005;
/** Saturation constant for the 0..1 energy scale. A brisk walk lands near 0.7. */
const ENERGY_SCALE = 1.2;
/** Autocorrelation search window for a beat, in seconds. */
const MIN_BEAT_PERIOD = 0.3;
const MAX_BEAT_PERIOD = 2.5;
/** A peak below this is noise, not a beat. */
const BEAT_FLOOR = 0.25;
/**
* Minimum relative spread (standard deviation over mean) of the motion-energy
* envelope before a beat is claimed at all. A clip moving at a steady rate has
* a flat envelope: it is continuous, not rhythmic, and any lag will correlate
* with any other. Without this gate a constant-speed ramp reports whatever the
* shortest searched lag happens to be.
*/
const BEAT_MIN_PULSE = 0.3;
/**
* A candidate lag this close to the strongest peak wins if it is shorter.
* Autocorrelation peaks at every multiple of the true period, and the estimator
* gets noisier as the overlap shrinks, so the largest peak is regularly 2x or
* 3x the real one. Preferring the shortest near-equal peak is the standard
* octave-error guard from pitch detection.
*/
const BEAT_HARMONIC_TOLERANCE = 0.88;
/** Loop-seam thresholds: mean bone angle (rad) and root offset (metres). */
const SEAM_ANGLE_LIMIT = 0.2;
const SEAM_ROOT_LIMIT = 0.06;
/**
* Share of motion that must survive an upper-body strip for a clip to still
* read as itself when played as an overlay. AnimationManager._buildOverlayClip
* drops every hips/leg/foot track before making the clip additive, so a clip
* whose motion is mostly in the legs becomes a near no-op on the `upper` layer.
*/
const OVERLAY_UPPER_SHARE = 0.6;
/**
* Net root displacement (metres) past which a one-shot visibly relocates the
* avatar. Agent slots play full-body through AnimationManager.play(), root
* track included, so a clip that ends 30 cm from where it started slides the
* avatar sideways and snaps back when it crossfades out.
*/
const ANCHOR_TRAVEL = 0.08;
/** Resample grid rate. Every clip in the library is baked at 30fps. */
const GRID_FPS = 30;
const MAX_GRID = 4096;
/**
* Map a canonical bone name to a region, or null for bones that are excluded.
*
* Finger bones are deliberately excluded. Only some clips carry finger tracks
* (51 vs 156 tracks across the library), so counting them would make two clips
* with identical body motion score differently purely on export settings.
*
* @param {string} bone
* @returns {string|null}
*/
export function regionOf(bone) {
if (bone === 'Hips') return 'root';
if (bone === 'Neck' || bone === 'Head') return 'head';
if (/^Spine\d*$/.test(bone)) return 'torso';
if (/^(Left|Right)(Shoulder|Arm|ForeArm|Hand)$/.test(bone)) return 'arms';
if (/^(Left|Right)(UpLeg|Leg|Foot|ToeBase)$/.test(bone)) return 'legs';
return null;
}
/** Which side of the body a bone is on, for the balance measure. */
function sideOf(bone) {
if (bone.startsWith('Left')) return 'left';
if (bone.startsWith('Right')) return 'right';
return null;
}
/**
* Visual mass of a bone: roughly how much of the silhouette its own rotation
* moves. A shoulder rotation swings the whole arm; a wrist rotation swings a
* hand. Without this, region shares answer "how many joints rotated" instead of
* "where did the body move", and a wave reads as head-led because the two neck
* bones out-average the one arm that is doing the work.
*
* Every clip in the library carries the full canonical bone set, so these
* weights are comparable across clips without further normalisation.
*/
const BONE_MASS = {
Hips: 3,
Spine: 1.6,
Spine1: 1.3,
Spine2: 1,
Neck: 0.5,
Head: 1,
Shoulder: 0.8,
Arm: 1.2,
ForeArm: 0.8,
Hand: 0.4,
UpLeg: 1.4,
Leg: 1,
Foot: 0.5,
ToeBase: 0.2,
};
/** Mass of a canonical bone, side prefix stripped. */
function massOf(bone) {
return BONE_MASS[bone.replace(/^(Left|Right)/, '')] ?? 1;
}
const round = (n, dp = 3) => {
const f = 10 ** dp;
return Math.round(n * f) / f;
};
/** Shortest-arc angle between two unit quaternions, in radians. */
function quatAngle(v, a, b) {
const dot = v[a] * v[b] + v[a + 1] * v[b + 1] + v[a + 2] * v[b + 2] + v[a + 3] * v[b + 3];
return 2 * Math.acos(Math.min(1, Math.abs(dot)));
}
/**
* Spread a segment's rotation angle across a uniform time grid.
* Segments are proportioned by overlap, so a clip sampled off-grid still lands
* its energy in the right bins instead of being snapped to the nearest one.
*/
function accumulate(grid, gridDt, t0, t1, angle) {
const span = t1 - t0;
if (!(span > 0)) return;
const first = Math.max(0, Math.floor(t0 / gridDt));
const last = Math.min(grid.length - 1, Math.floor((t1 - 1e-9) / gridDt));
for (let i = first; i <= last; i++) {
const lo = Math.max(t0, i * gridDt);
const hi = Math.min(t1, (i + 1) * gridDt);
if (hi > lo) grid[i] += angle * ((hi - lo) / span);
}
}
/** Three-tap moving average. Frame-to-frame jitter autocorrelates; a beat does not need it. */
function smooth(signal) {
const n = signal.length;
const out = new Float64Array(n);
for (let i = 0; i < n; i++) {
const a = signal[Math.max(0, i - 1)];
const b = signal[i];
const c = signal[Math.min(n - 1, i + 1)];
out[i] = (a + 2 * b + c) / 4;
}
return out;
}
/**
* Find the clip's beat by autocorrelating its motion-energy envelope.
*
* Only local maxima count. Autocorrelation decays with lag, so taking the
* largest value in the window always returns the shortest lag searched, which
* is how a nod ends up reported at 257 BPM. A real periodic motion puts a peak
* at its period with troughs either side; that is what this looks for.
*
* @returns {{period:number|null, strength:number}} period in seconds
*/
function findBeat(signal, dt, duration) {
const n = signal.length;
const maxLagS = Math.min(MAX_BEAT_PERIOD, duration / 2);
if (n < 12 || maxLagS <= MIN_BEAT_PERIOD) return { period: null, strength: 0 };
const env = smooth(signal);
let mean = 0;
for (let i = 0; i < n; i++) mean += env[i];
mean /= n;
const dev = new Float64Array(n);
let energy = 0;
for (let i = 0; i < n; i++) {
dev[i] = env[i] - mean;
energy += dev[i] * dev[i];
}
if (energy <= 0) return { period: null, strength: 0 };
if (mean > 0 && Math.sqrt(energy / n) / mean < BEAT_MIN_PULSE) {
return { period: null, strength: 0 };
}
const minLag = Math.max(2, Math.round(MIN_BEAT_PERIOD / dt));
const maxLag = Math.min(n - 2, Math.round(maxLagS / dt));
if (maxLag <= minLag + 1) return { period: null, strength: 0 };
// Normalised autocorrelation across the whole window, then peak-pick on it.
const acf = new Float64Array(maxLag + 2);
for (let lag = minLag - 1; lag <= maxLag + 1 && lag < n - 1; lag++) {
let sum = 0;
for (let i = 0; i + lag < n; i++) sum += dev[i] * dev[i + lag];
acf[lag] = (sum / (n - lag)) / (energy / n);
}
let bestLag = 0;
let best = 0;
const peaks = [];
for (let lag = minLag; lag <= maxLag; lag++) {
if (acf[lag] <= acf[lag - 1] || acf[lag] < acf[lag + 1]) continue;
peaks.push(lag);
if (acf[lag] > best) {
best = acf[lag];
bestLag = lag;
}
}
// Octave-error guard: the shortest peak within tolerance of the strongest
// one is the true period; the strongest is regularly a multiple of it.
for (const lag of peaks) {
if (lag < bestLag && acf[lag] >= best * BEAT_HARMONIC_TOLERANCE) {
bestLag = lag;
best = acf[lag];
break;
}
}
if (bestLag === 0 || best < BEAT_FLOOR) {
return { period: null, strength: round(Math.max(0, best)) };
}
return { period: bestLag * dt, strength: round(Math.min(1, best)) };
}
/**
* Measure one baked clip.
*
* @param {{name:string, duration:number, tracks:Array}} clip — the parsed
* contents of public/animations/clips/<name>.json
* @returns {object} signature (see the README table in docs/animations.md)
*/
export function analyzeClip(clip) {
const name = String(clip?.name || '');
const duration = Number(clip?.duration) || 0;
const tracks = Array.isArray(clip?.tracks) ? clip.tracks : [];
const gridN = Math.min(MAX_GRID, Math.max(2, Math.round(duration * GRID_FPS)));
const gridDt = duration > 0 ? duration / gridN : 1;
const grid = new Float64Array(gridN);
const regionSum = { head: 0, arms: 0, torso: 0, root: 0, legs: 0 };
const sideSum = { left: 0, right: 0 };
let seamAngle = 0;
let seamBones = 0;
let totalAngle = 0;
let boneCount = 0;
let frames = 0;
for (const track of tracks) {
const [bone, prop] = String(track?.name || '').split('.');
if (prop !== 'quaternion') continue;
const region = regionOf(bone);
if (!region) continue;
const times = track.times;
const values = track.values;
if (!times?.length || !values?.length) continue;
frames = Math.max(frames, times.length);
let sum = 0;
for (let i = 0; i < times.length - 1; i++) {
const angle = quatAngle(values, i * 4, (i + 1) * 4);
sum += angle;
accumulate(grid, gridDt, times[i], times[i + 1], angle);
}
const perSecond = duration > 0 ? sum / duration : 0;
const mass = massOf(bone);
regionSum[region] += mass * perSecond;
totalAngle += sum;
boneCount += 1;
const side = sideOf(bone);
if (side) sideSum[side] += mass * perSecond;
seamAngle += quatAngle(values, 0, (times.length - 1) * 4);
seamBones += 1;
}
const meanSpeed = boneCount > 0 && duration > 0 ? totalAngle / boneCount / duration : 0;
// Mass-weighted shares: what fraction of the body's visible movement each
// region accounts for.
let regionTotal = 0;
for (const r of REGIONS) regionTotal += regionSum[r];
const regions = {};
for (const r of REGIONS) regions[r] = round(regionTotal > 0 ? regionSum[r] / regionTotal : 0);
// Root translation. Hips.position is in metres on the canonical rig.
let travel = 0;
let path = 0;
let rise = 0;
const hips = tracks.find((t) => t?.name === 'Hips.position');
if (hips?.values?.length >= 3) {
const v = hips.values;
const n = Math.floor(v.length / 3);
let yMin = Infinity;
let yMax = -Infinity;
for (let i = 0; i < n; i++) {
const y = v[i * 3 + 1];
if (y < yMin) yMin = y;
if (y > yMax) yMax = y;
if (i > 0) path += Math.hypot(v[i * 3] - v[(i - 1) * 3], v[i * 3 + 2] - v[(i - 1) * 3 + 2]);
}
travel = Math.hypot(v[(n - 1) * 3] - v[0], v[(n - 1) * 3 + 2] - v[2]);
rise = yMax - yMin;
}
const speedSignal = new Float64Array(gridN);
for (let i = 0; i < gridN; i++) speedSignal[i] = grid[i] / gridDt;
const beat = findBeat(speedSignal, gridDt, duration);
const seam = seamBones > 0 ? seamAngle / seamBones : 0;
const seamRoot = hips?.values?.length >= 3
? (() => {
const v = hips.values;
const n = Math.floor(v.length / 3);
return Math.hypot(v[(n - 1) * 3] - v[0], v[(n - 1) * 3 + 1] - v[1], v[(n - 1) * 3 + 2] - v[2]);
})()
: 0;
const balance = sideSum.left + sideSum.right > 0
? 1 - Math.abs(sideSum.left - sideSum.right) / (sideSum.left + sideSum.right)
: 1;
const isStatic = frames < MIN_ANIMATED_FRAMES || meanSpeed < STATIC_SPEED;
const upperShare = round(regions.head + regions.arms + regions.torso);
return {
clip: name,
duration: round(duration, 2),
frames,
energy: round(1 - Math.exp(-meanSpeed / ENERGY_SCALE)),
speed: round(meanSpeed),
regions,
lead: leadRegion(regions),
upperShare,
balance: round(balance),
travel: round(travel),
path: round(path),
rise: round(rise),
tempo: beat.period ? Math.round(60 / beat.period) : 0,
beat: beat.strength,
seam: round(seam),
seamRoot: round(seamRoot),
loopClean: !isStatic && seam < SEAM_ANGLE_LIMIT && seamRoot < SEAM_ROOT_LIMIT,
anchored: travel < ANCHOR_TRAVEL,
overlay: !isStatic && upperShare >= OVERLAY_UPPER_SHARE,
static: isStatic,
};
}
/** The region carrying the most motion. */
export function leadRegion(regions) {
let best = REGIONS[0];
for (const r of REGIONS) if ((regions[r] || 0) > (regions[best] || 0)) best = r;
return best;
}
/**
* Distance between two signatures in the descriptor space, 0 (identical) to 1.
* Region shares dominate because "where the motion is" is what people mean by
* "a clip like this one"; energy and tempo refine the ordering inside a family.
*/
export function distance(a, b) {
if (!a || !b) return 1;
let regionDelta = 0;
for (const r of REGIONS) regionDelta += Math.abs((a.regions?.[r] || 0) - (b.regions?.[r] || 0));
// Region shares each sum to 1, so the L1 delta is at most 2.
const regionTerm = regionDelta / 2;
const energyTerm = Math.abs((a.energy || 0) - (b.energy || 0));
const tempoTerm = a.tempo && b.tempo
? Math.min(1, Math.abs(a.tempo - b.tempo) / 120)
: (a.tempo || b.tempo ? 0.5 : 0);
const travelTerm = Math.min(1, Math.abs((a.path || 0) - (b.path || 0)) / 2);
return round(Math.min(1, 0.55 * regionTerm + 0.25 * energyTerm + 0.12 * tempoTerm + 0.08 * travelTerm));
}
/** Rank an index of signatures by closeness to one clip. */
export function similarTo(clipName, index, limit = 6) {
const target = index?.[clipName];
if (!target) return [];
return Object.values(index)
.filter((s) => s.clip !== clipName)
.map((s) => ({ clip: s.clip, distance: distance(target, s), signature: s }))
.sort((a, b) => a.distance - b.distance)
.slice(0, limit);
}
/** Plain-language energy band, so the UI and the docs use the same words. */
export function energyBand(sig) {
if (!sig || sig.static) return 'still';
if (sig.energy < 0.15) return 'calm';
if (sig.energy < 0.4) return 'gentle';
if (sig.energy < 0.65) return 'lively';
return 'explosive';
}
/**
* One sentence describing a signature, for card subtitles and screen readers.
* @param {object} sig
* @returns {string}
*/
export function describe(sig) {
if (!sig) return '';
if (sig.static) return 'A single held pose. Nothing moves.';
const parts = [`${energyBand(sig)} motion led by the ${REGION_LABELS[sig.lead].toLowerCase()}`];
if (sig.tempo) parts.push(`on a ${sig.tempo} BPM beat`);
if (!sig.anchored) parts.push(`ending ${sig.travel.toFixed(2)} m from where it started`);
else parts.push('anchored on the spot');
if (sig.overlay) parts.push('readable as an upper-body overlay');
if (!sig.loopClean) parts.push('with a visible seam if looped');
return `${parts.join(', ')}.`;
}
/**
* Slots the walk state machine plays on its `upper` layer, where
* AnimationManager._buildOverlayClip strips every hips/leg/foot track before
* blending. tests/motion-signature.test.js holds this list to the real GESTURES
* table in src/animation-state-machine.js.
*/
export const DEFAULT_OVERLAY_SLOTS = ['wave', 'nod', 'shake', 'point', 'shrug', 'sign'];
/** Slots that run as a continuous loop rather than a one-shot. */
export const DEFAULT_LOOP_SLOTS = ['idle', 'fidget', 'patience'];
/**
* Whether a clip can actually do the job a slot is for.
*
* Three failure modes, each measured rather than guessed:
*
* 1. A held pose in an animated slot freezes the avatar (four clips in the
* library are single-frame exports).
* 2. A leg-led clip in an overlay slot nearly vanishes, because the overlay
* path strips the lower body before blending.
* 3. A clip whose root ends away from where it began slides the avatar across
* the floor and snaps it back on crossfade out. Agent slots play full-body
* through AnimationManager.play(), root track included, so this is visible
* on every agent page.
*
* @param {string} slot
* @param {object} sig
* @param {{overlaySlots?: string[], loopSlots?: string[]}} [rules]
* @returns {{level:'ok'|'warn', message:string}|null} null when nothing to say
*/
export function slotFit(slot, sig, rules = {}) {
if (!sig) return null;
const overlaySlots = rules.overlaySlots || DEFAULT_OVERLAY_SLOTS;
const loopSlots = rules.loopSlots || DEFAULT_LOOP_SLOTS;
if (sig.static) {
return {
level: 'warn',
message: 'This clip is a single held pose, so the slot would freeze instead of animating.',
};
}
if (overlaySlots.includes(slot) && !sig.overlay) {
return {
level: 'warn',
message: `On /walk, ${slot} plays as an upper-body overlay with the hips and legs stripped out, and ${Math.round((1 - sig.upperShare) * 100)}% of this clip's motion is below the waist. Most of it would not survive the strip.`,
};
}
if (loopSlots.includes(slot) && !sig.loopClean) {
return {
level: 'warn',
message: `${slot} loops continuously, and this clip's last frame does not meet its first. Expect a visible snap every cycle.`,
};
}
if (!sig.anchored) {
return {
level: 'warn',
message: `This clip ends ${sig.travel.toFixed(2)} m from where it started, so the avatar slides that far and snaps back when the gesture finishes.`,
};
}
return { level: 'ok', message: describe(sig) };
}