Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
Memory Spec v0.1
Agent memory is a file-based, human-readable system modeled on Claude Code's memory. It is intentionally the same shape — if you know how to read a Claude MEMORY.md, you can read an agent's memory.
Memory travels with the agent. Move your agent from one site to another — the memory goes with it, because memory is just files in the manifest bundle.
Bundle layout
agent/
└── memory/
├── MEMORY.md # index (always loaded into context)
├── user_role.md # one memory per file
├── user_preferences.md
├── feedback_tone.md
├── project_launch.md
├── reference_discord.md
└── timeline/ # append-only event log (optional)
├── 2026-04-14.jsonl
└── 2026-04-13.jsonl
MEMORY.md — the index
Always loaded into the LLM's system context. Lines beyond 200 are truncated — keep it concise.
No frontmatter. One line per memory: - [Title](file.md) — one-line hook.
# Coach Leo's Memory
## User
- [Role](user_role.md) — Argentina fan, plays weekly 5-a-side on Saturdays
- [Preferences](user_preferences.md) — terse feedback, no emojis
## Feedback
- [Tone](feedback_tone.md) — stay warm but don't coddle; user asked for direct critique
- [Drill pacing](feedback_pacing.md) — 3 drills per session max
## Project
- [Season goal](project_season.md) — user is training for a tournament in June 2026
- [Recovery](project_recovery.md) — knee injury from March; avoid high-impact drills until May
## Reference
- [Highlight reel](reference_reel.md) — shared Drive folder where user stores match clips
Individual memory files
Each file has frontmatter + body.
---
name: Tone
description: user wants direct critique, not encouragement-first
type: feedback
created: 2026-03-22
updated: 2026-04-14
---
User prefers direct critique over encouragement-first framing. "Just tell me
what's wrong" — their words after session #4.
**Why:** user explicitly corrected the first two sessions' overly warm tone.
**How to apply:** lead with the issue, follow with the fix. Save warmth for
genuine wins, not every message.
Frontmatter fields
| Field | Required | Purpose |
|---|---|---|
name |
yes | Human-readable title |
description |
yes | One-line, used by the retrieval layer to judge relevance |
type |
yes | user | feedback | project | reference |
created |
yes | ISO date |
updated |
yes | ISO date — bump on every edit |
source |
no | What conversation/event produced this memory |
decay |
no | never | 30d | 90d — retrieval-layer decay hint |
Types
Same taxonomy as Claude Code memory:
- user — who the user is, their role, goals, knowledge level. Durable.
- feedback — corrections or validations of the agent's own behavior. Include Why and How to apply.
- project — state of ongoing work, deadlines, stakeholders. Decays fast; convert relative dates to absolute (
Thursday→2026-03-05). - reference — pointers to external systems (Drive folders, Slack channels, Linear projects).
timeline/ — append-only event log
Optional. Skills write ephemeral events here via ctx.memory.note(type, data). One JSONL file per day. Read by the retrieval layer as short-term context.
{"ts":"2026-04-14T12:03:12Z","type":"waved","style":"enthusiastic"}
{"ts":"2026-04-14T12:03:45Z","type":"user_said","text":"how's my form?"}
{"ts":"2026-04-14T12:04:10Z","type":"played_clip","name":"demo-kick"}
Timeline entries auto-decay: default retention is 30 days, configurable per agent in manifest.json → memory.timelineRetentionDays.
Storage modes
Set in manifest.json → memory.mode:
local (default)
Persisted in browser localStorage under agent:{agentId}:memory:*. Fast, private to the device, ephemeral (cleared if the user clears site data).
ipfs
Each write pins the full memory directory to IPFS. The new CID is recorded:
- In
localStoragefor fast recovery. - Optionally in the manifest's
id.memoryCIDfield (requires a chain write — opt-in).
Slow on write, durable, portable across devices. Use when the agent's memory is valuable and should survive device loss.
encrypted-ipfs
Same as ipfs but the content is encrypted to the owner's wallet pubkey (ECIES / libsodium sealed box) before pinning. Only the owner can decrypt. Use when memory contains user PII.
none
Agent is stateless across sessions. Useful for demos, kiosks, one-shot interactions.
Custom modes
Any memory.mode value that is not one of the five built-ins resolves to a backend you registered via Memory.registerBackend — see Custom backends. If no backend matches, the runtime warns once and falls back to local, so a typo can never break the embed.
Custom backends
You can plug an arbitrary store — a vector database, an episodic event log, your own API — behind a named mode without forking the Memory class. Register it once at startup; from then on memory.mode: "<name>" (or <agent-3d memory="<name>">) routes through it.
import { Memory } from 'https://three.ws/agent-3d/latest/agent-3d.js';
Memory.registerBackend('vector', {
// REQUIRED. Hydrate persisted state. Receives the full Memory.load opts
// (namespace, manifestURI, fetchFn, plus anything extra you pass through).
// Return any subset of { files, timeline, index }.
async load({ namespace }) {
const rows = await myVectorStore.fetchAll(namespace);
const files = Object.fromEntries(rows.map((r) => [r.filename, r.markdown]));
return { files };
},
// OPTIONAL. Called after every write() and note(). Inspect memory.files
// (a Map) and memory.timeline (an array) and ship them to your store.
async persist(memory) {
for (const [filename, markdown] of memory.files) {
await myVectorStore.upsert(memory.namespace, { filename, markdown });
}
await myEpisodicLog.replace(memory.namespace, memory.timeline);
},
// OPTIONAL. Real semantic search. Return ranked { file, meta, body, score }
// hits. If it throws, Memory silently falls back to substring matching, so
// recall never hard-fails mid-conversation.
async recall(query, memory, { limit = 5 } = {}) {
const matches = await myVectorStore.search(memory.namespace, query, limit);
return matches.map((m) => ({ file: m.filename, meta: m.meta, body: m.body, score: m.score }));
},
});
Contract notes:
loadis the only required hook. A backend with justloadis read-through; addpersistto make it durable,recallto make search semantic.persistis fire-and-forget onwrite/note(so those stay synchronous, matchinglocal/remote); it is awaited onagent.memory.save(). Make it idempotent.- Built-in mode names (
none,local,remote,ipfs,encrypted-ipfs) are reserved —registerBackendthrows if you try to shadow one. Memory.backends()lists registered names;Memory.unregisterBackend(name)removes one.
The lighter-weight extension point — keeping built-in storage but swapping only the ranker — is manifest.json → memory.retriever (see Retrieval). Reach for a full backend when you also own where the bytes live.
Retrieval
The runtime budgets memory.maxTokens (default 8192) per turn and fills it with:
- Always:
MEMORY.md(the index — forces global awareness). - Relevance-ranked: memories selected by matching the current user message against each memory's
descriptionvia embedding similarity. - Recency-weighted: the last N timeline entries (N computed from remaining budget).
Retrieval is pluggable — manifest.json → memory.retriever can point to a skill that implements a custom ranker.
Writing memories
Skills and the runtime both write via ctx.memory.*:
ctx.memory.write('feedback_tone', {
name: 'Tone',
description: 'user wants direct critique, not encouragement-first',
type: 'feedback',
body: 'User prefers direct critique...\n\n**Why:** ...\n**How to apply:** ...',
});
ctx.memory.note('waved', { style: 'casual' }); // timeline
ctx.memory.read('feedback_tone'); // read single
await ctx.memory.recall('how does the user prefer feedback'); // semantic search
What NOT to save
Copied verbatim from Claude Code memory guidance — applies here too:
- Information derivable from code/state (skill code, manifest contents).
- Ephemeral conversation context ("we're currently discussing X").
- Anything already in
SKILL.mdorinstructions.md. - Secrets, API keys, tokens (ever).
Sync semantics
- Single-device: straightforward — localStorage read-through.
- Multi-device (ipfs mode): last-write-wins, with a 3-way merge on conflict (LLM-mediated). Conflicts are rare because most writes are additive.
- Multi-runtime: if the same agent is live in two tabs, writes lock via a BroadcastChannel mutex.
Forgetting
Explicit user request ("forget that I..."): delete the file, update MEMORY.md, append a forgot entry to timeline for audit.
Automatic decay (per frontmatter decay): retrieval layer down-weights; files are not deleted without user confirmation.
Snapshot contract
agent.memory.snapshot() returns a stable, JSON-safe object — the memory/0.1 contract — that fully describes an agent's memory at a point in time. It is synchronous, so embedded widgets can serialize/deserialize across page reloads, postMessage between frames, or stash state in sessionStorage.
{
"version": "memory/0.1",
"mode": "local", // the storage mode this snapshot came from
"namespace": "agent-uuid",
"index": "# Memory\n\n## User\n- [Role](user_role.md) — …",
"files": { "user_role.md": "---\nname: Role\n…", /* … */ },
"timeline": [ { "ts": "2026-04-14T12:03:12Z", "type": "waved", "style": "casual" } ]
}
Rehydrate with Memory.fromSnapshot. Overrides win over the snapshot's own values, so you can restore into a fresh agent id:
// Embedded widget — survive a reload:
sessionStorage.setItem('agent-mem', JSON.stringify(agent.memory.snapshot()));
// …after reload:
const restored = Memory.fromSnapshot(JSON.parse(sessionStorage.getItem('agent-mem')));
// or graft onto a new identity:
const forked = Memory.fromSnapshot(snap, { mode: 'remote', namespace: newAgentId });
If a snapshot omits index but carries files, fromSnapshot rebuilds the index automatically.
Export / import
export() is the async alias for snapshot() and returns the same memory/0.1 shape; import() merges another agent's snapshot in (last-write-wins on the timeline, additive on files):
const blob = await agent.memory.export(); // === agent.memory.snapshot()
await otherAgent.memory.import(blob); // merge (default)
await otherAgent.memory.import(blob, { strategy: 'replace' }); // overwrite
Enables memory-as-inheritance: fork an agent, carry the memories forward.
See also
- AGENT_MANIFEST.md —
memoryfield in manifest - SKILL_SPEC.md — skill
ctx.memoryAPI