Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
Skill Spec v0.1
A skill is a portable, content-addressed capability bundle that any compatible agent can install at runtime. Skills are the npm of embodied AI: publish once (IPFS or HTTPS), install anywhere.
A skill bundles three things:
- Instructions (
SKILL.md) — when and how to use it, written for the LLM. - Tool schema (
tools.json) — the tool-use functions the LLM can call. - Assets (optional) — animation clips, morph target presets, shaders, audio, prompts.
Bundle layout
skill-name/
├── SKILL.md # instructions (required)
├── tools.json # tool schema (required if skill exposes tools)
├── handlers.js # tool implementations (optional; omitted = declarative-only)
├── manifest.json # metadata, deps, version (required)
├── clips/ # animation GLBs (optional)
│ ├── wave-casual.glb
│ └── wave-enthusiastic.glb
├── morphs/ # morph target presets (optional)
│ └── happy.json
├── prompts/ # prompt templates referenced by SKILL.md (optional)
│ └── greeting.txt
├── assets/ # arbitrary other files (optional)
└── LICENSE
manifest.json
{
"spec": "skill/0.1",
"name": "wave",
"version": "0.1.0",
"description": "Wave at the user with context-appropriate enthusiasm.",
"author": "0xDeadBeef...", // optional wallet
"license": "MIT",
"tags": ["greeting", "gesture", "mixamo-compatible"],
// Compatibility
"requires": {
"rig": ["mixamo", "any"], // rigs this skill works with
"runtime": ">=0.1.0",
"tools": ["play_clip"], // built-in tools this skill depends on
},
// Dependencies on other skills (composition)
"dependencies": {
"ipfs://bafy.../gesture-base/": "^1.0.0",
},
// What this skill adds
"provides": {
"tools": ["wave"], // declared in tools.json
"triggers": ["greeting"], // semantic tags for skill discovery
},
// Content-addressed integrity (optional but recommended)
"integrity": {
"SKILL.md": "sha256-...",
"tools.json": "sha256-...",
"clips/wave-casual.glb": "sha256-...",
},
}
SKILL.md
Frontmatter + markdown. The LLM sees this as a system-prompt fragment whenever the skill is loaded.
---
name: wave
description: Wave at the user with context-appropriate enthusiasm.
triggers:
- user_greeting
- user_farewell
- introduction
cost: low
---
# Wave
When the user greets you, says goodbye, or when you are introduced to
someone, wave at them. Pick the clip based on the vibe:
- **wave-casual** — default, for most greetings
- **wave-enthusiastic** — when the user seems excited, or after a win
- **wave-subtle** — in a professional context or mid-conversation
Call `wave({ style: "casual" | "enthusiastic" | "subtle" })`.
Do not wave more than once per turn. Do not wave if you just waved.
Frontmatter fields:
| Field | Type | Purpose |
|---|---|---|
name |
string | Must match manifest.json |
description |
string | One-line, used in skill discovery UIs |
triggers |
string[] | Semantic tags — when LLM router sees these, it surfaces the skill |
cost |
low | medium | high |
Latency/token hint for the runtime to decide eager vs. lazy load |
tools.json
Standard tool-use schema (Anthropic / OpenAI compatible). This is what the LLM sees as callable functions.
{
"tools": [
{
"name": "wave",
"description": "Wave at the user.",
"input_schema": {
"type": "object",
"properties": {
"style": {
"type": "string",
"enum": ["casual", "enthusiastic", "subtle"],
"default": "casual",
},
"duration_ms": {
"type": "integer",
"minimum": 500,
"maximum": 5000,
"default": 1500,
},
},
"required": [],
},
},
],
}
handlers.js (optional)
ES module exporting one function per tool. Receives (args, context) where context exposes the live scene, viewer, memory, and built-in tools.
// skills/wave/handlers.js
export async function wave(args, ctx) {
const { style = 'casual', duration_ms = 1500 } = args;
const clipUri = new URL(`clips/wave-${style}.glb`, ctx.skillBaseURI).href;
const clip = await ctx.loadClip(clipUri);
ctx.viewer.play(clip, { duration: duration_ms, blend: 0.2 });
// Optional — annotate memory so the LLM doesn't double-wave
ctx.memory.note('just_waved', { at: Date.now(), style });
return { ok: true, played: style };
}
Handlers run in a sandboxed scope with a fixed context API. They cannot:
- Import arbitrary modules (only
ctx.*is available) - Touch
windowdirectly - Make network calls outside
ctx.fetch(which enforces CORS + IPFS gateway policy) - Persist state outside
ctx.memory
The context API
interface SkillContext {
// Scene control
viewer: {
play(clip: AnimationClip, opts?: PlayOptions): void;
stop(clipName?: string): void;
setExpression(preset: string): void;
lookAt(target: Vector3 | 'user' | 'camera'): void;
moveTo(position: Vector3, opts?: MoveOptions): void;
scene: Scene; // read-only reference
};
// LLM
llm: {
complete(prompt: string, opts?: CompleteOpts): Promise<string>;
embed(text: string): Promise<Float32Array>;
};
// Memory
memory: {
read(key: string): any;
write(key: string, value: any): void;
note(type: string, data: any): void; // appends to a timeline
recall(query: string): Promise<MemoryEntry[]>;
};
// Asset loading
loadClip(uri: string): Promise<AnimationClip>;
loadGLB(uri: string): Promise<GLTF>;
loadJSON(uri: string): Promise<any>;
skillBaseURI: string; // base for relative asset resolution
// Safe network
fetch(uri: string, opts?: FetchOpts): Promise<Response>;
// Cross-skill calls
call(toolName: string, args: any): Promise<any>;
// User interaction
speak(text: string): Promise<void>;
listen(opts?: ListenOpts): Promise<string>;
}
Resolution & loading
manifest.json: skills: [{ uri: "ipfs://.../wave/", version: "0.1.0" }]
│
▼
fetch {uri}/manifest.json
│
▼
verify integrity hashes (if present)
│
▼
check requires.rig matches agent body.rig
│
▼
parallel fetch: SKILL.md, tools.json, handlers.js (if any)
│
▼
merge tools into runtime tool-use schema
│
▼
inject SKILL.md into system prompt
│
▼
skill is live; LLM can now call its tools
Declarative-only skills
A skill without handlers.js is valid — it's pure prompt engineering + assets. Example: a "british-accent" skill that only contains SKILL.md ("Speak with a British accent. Use terms like 'brilliant', 'cheers', etc.") — no tools, no code, just context injection.
Skill discovery
The runtime class is SkillRegistry (src/skills/index.js). Skills are installed explicitly with registry.install({ uri, version }) and removed with registry.uninstall(name); there is no built-in search method. Discovery over the installed set is caller-side iteration — e.g. filter registry.all() by manifest fields such as tags:
const greeters = registry.all().filter((s) => s.manifest.tags?.includes("greeting"));
registry.allTools() merges every installed skill's tool schemas into the runtime tool-use schema, registry.findSkillForTool(name) resolves a tool name back to its skill, and registry.systemPrompt() concatenates the installed skills' SKILL.md fragments for the system prompt.
Install specs come from:
- The local manifest's
skillsarray. - Optional HTTPS skill index (community-hosted).
- On-chain skill registry (if deployed) — same ERC-8004-style pattern, agents can publish + own skills.
Security model
- Declarative skills (no
handlers.js) are always safe to load. - Code skills execute in a shared Web Worker (
src/skills/sandbox-worker.js). The worker has no DOM, nowindow, nodocument, and nolocalStorageorcookieaccess. Handlers cannot read or write page state. - The worker receives handler source as text and loads it via a
blob:URL dynamicimport(). This prevents handlers from importing external modules at load time. - Each
ctx.*method call from inside the worker becomes a structured-cloneablepostMessageround-trip to the main thread. The main thread validates and dispatches the call, then returns the result. Non-serializable objects (e.g.AnimationClip,GLTF) are stored in a host-side handle registry and represented to the worker as opaque"@h:N"strings. ctx.fetchandctx.loadJSONuse the worker's nativefetch— no main-thread proxy is needed. Standard CORS rules apply (same-origin and credentialed-CORS requests work; cross-origin without CORS headers are blocked by the browser as usual).- Individual
ctx.*calls time out after 30 seconds. A timed-out or errored handler surfaces as{ error: "..." }to the runtime without crashing the worker. - Agent owners set a skill trust policy per agent:
trust: "any" | "whitelist" | "owned-only".owned-only(default) — only run skills signed by the agent's owner wallet.whitelist— allow a list of publisher wallets.any— wild west, allowed for kiosks/demos.
- Integrity hashes are verified before handlers execute.
Opt-out: sandboxPolicy: "trusted-main-thread"
Owner-signed skills that require main-thread capabilities (Three.js direct access, window.* APIs, or performance-sensitive frame-tick work) can bypass the worker by setting in manifest.json:
{
"sandboxPolicy": "trusted-main-thread",
}
When this flag is honored, handlers.js is loaded via a direct import() in the main thread and called synchronously, the same as the pre-sandbox behavior. The flag is a request, not a grant: the manifest is written by whoever hosts the bundle. The registry honors it only when the bundle is served from an origin the host trusts (for <agent-3d>, the page's own origin or the origin that served the library) and the trust mode is not any. Any other bundle that sets it still installs, but runs in the worker sandbox.
Rationale for the name: it describes where the code runs (main-thread) and the precondition (trusted), making the security trade-off explicit to manifest authors.
What sandboxed handlers cannot do
- Read
document.cookie,window.localStorage,sessionStorage, or any DOM API. - Access
window,navigator, orlocationfrom handler code. - Import external modules (no
importstatements in handler source — the blob context has no base URL for relative imports). - Make arbitrary network calls outside
ctx.fetch/ctx.loadJSON(CORS still applies).
What sandboxed handlers can still do
All ctx.* capabilities listed in the API section above — animation, memory, speech, LLM, cross-skill calls — work identically via the postMessage proxy.
Worked example: the wave skill
skills/wave/
├── manifest.json
├── SKILL.md
├── tools.json
├── handlers.js
├── clips/
│ ├── wave-casual.glb
│ ├── wave-enthusiastic.glb
│ └── wave-subtle.glb
└── LICENSE
Files above — manifest.json, SKILL.md, tools.json, handlers.js — are exactly as shown in the sections above. A publisher can ipfs add -r skills/wave/ and hand out the resulting CID; any agent can then install via:
{ "uri": "ipfs://bafy.../", "version": "0.1.0" }
Versioning
spec: skill/0.1— skill format itself.versioninmanifest.json— semver, author-controlled.- Agents can pin
"version": "0.1.0"or range"^0.1.0". - Version resolution prefers highest-compatible published version within range.
See also
- AGENT_MANIFEST.md — how agents reference skills
- EMBED_SPEC.md — how skills are surfaced in the web component