Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.
Python agent-orchestrationai-agentai-coding-assistantclaude-codecoding-agentcomputer-usedesktop-applocal-firstmcppersonal-jarvisvoice-assistantvoice-coding
docs(society): define the agent, its hands and the wiki as shared memory
rubenluetke10-beepc8382b528ac3
diff --git a/docs/agent-society/MASTERPLAN.md b/docs/agent-society/MASTERPLAN.mdindex 063c718792..f061033511 100644--- a/docs/agent-society/MASTERPLAN.md+++ b/docs/agent-society/MASTERPLAN.md@@ -348,6 +348,13 @@ Decided 2026-09-01:society — backend and UI — is built in-house (§7).8. **Build start:** frontend card work may proceed on clearly-labeled sample data while M1 lands(`components/society/data.ts` is the single swap point).+9. **Agent definition & ecosystem wiring:** see [`agent-definition.md`](agent-definition.md)+ (binding). Highlights: full customization per agent (brain, grant/focus/deny over ONE+ capability catalog of plugins + CLIs + MCPs + skills, approval rules, budget, workspace);+ a "Gmail agent" is `grant_mode=all` + `focus=[plugin:gmail]` derived from its description;+ the **Obsidian wiki is the society's shared memory** (namespaced `society/<agent>/` writes,+ reviewed promotion to `society/shared/`) — the `knowledge` table of §3.1 becomes the+ staging/provenance layer, not the truth. Grok Bot re-analyzed there as the product floor.Still open:diff --git a/docs/agent-society/agent-definition.md b/docs/agent-society/agent-definition.mdnew file mode 100644index 0000000000..29282fbedd--- /dev/null+++ b/docs/agent-society/agent-definition.md@@ -0,0 +1,313 @@+# Agent Definition — what a society agent IS and how it plugs into Jarvis++Status: **binding design, 2026-09-01.** Subordinate to [`MASTERPLAN.md`](MASTERPLAN.md); it+refines §3.1 (roster), §3.2 (runners), §4.2 (model card), §6 (safety) and answers the questions+the build plan left open: how an agent gets its hands (plugins, CLIs, MCPs, skills), how it knows+the Jarvis ecosystem, and how the Obsidian wiki becomes the society's shared memory.++The reference product is Grok Bot (§1). The whole thing is built in-house (MASTERPLAN §10.7).++---++## 1. Grok Bot, analyzed (docs.x.ai, 2026-09-01)++What xAI ships, in one table, with our verdict per row.++| Grok Bot | What it means | Ours |+|---|---|---|+| A Bot = one persistent, named teammate with **name, title, description**; the description holds *durable operating rules*, the chat holds one-off instructions | Three fields, no wizard. "Focused Bots build more useful context than one catch-all Bot." | **Adopt** exactly: `name`, `title` (job line), `description` (standing instructions = the agent's own AGENTS.md). |+| Bots **message each other**, share context in threads / group chats, `@mention`, **pass ownership**; xAI recommends "message one owner Bot" rather than parallel threads | Coordinator-over-specialists; noisy group chats are a known failure ("repeat each other, start unnecessary loops") | **Adopt the shape, harden the mechanism:** owner = our `orchestrator` tier; typed envelopes instead of prose; rooms capped 2–6 / ≤3 rounds / ≤10 messages. |+| A **handoff** records: output path, what was completed, evidence used, what remains unresolved, which Bot owns the next step | The one durable coordination artifact | **Adopt as the `RESULT` payload schema** (§4.3). |+| **Skills** = reusable procedure (steps, decision rules, expected output, safety boundaries), invoked with `/`, created from chat, manually, or by *teaching* (recorded browser demo → draft skill) | Their self-improvement loop | We already have `SKILL.md` skills with `draft` lifecycle (AP-15), `/` typeahead, `create-skill` + `run-skill`. **Adopt** per-agent skill enablement; teach-by-demo = M6 stretch. |+| **Routines** = one Bot + schedule or event trigger; results post into the Bot's chat; 50 per Bot, 20 run records kept; "test the skill on a real one-time task first" | Automations | Our Automations scheduler already does schedules, event triggers, approvals. **Adopt** `[agent:<name>]` routines with results into the canonical chat. |+| **Connectors / Plugins / MCP servers** are account-wide ("not isolated to one Bot"); prefer a connector over clicking through a website | Capability surface is shared, bots differ by role | **Adopt with a twist:** the capability catalog is global, but each agent has a *grant* (all / allow-list) and a *focus* (§3.2). |+| One **shared cloud computer** per account; every Bot has its own screen; "do not use separate Bots as a security boundary" | Cheap handoff via shared files/logins | **Do differently:** local-first. Files = the agent's workspace dir; screens = `agent_screen` leases (M6); security boundary = the risk tiers + per-agent grant, not the bot. |+| **Memory**: stable preferences, role context, summaries of prior work; "context and memory are not the same as a database"; durable work goes to `/workspace` or external systems | Memory is per bot, sharing is by files/messages | **Do differently:** the Obsidian wiki is the shared memory (§5); per-agent memory is a wiki page, not a hidden store. |+| **Approvals**: Auto-review rules `Require Approval` (always stop) vs `Always Allow` (only if nothing else objects); Require wins; high-risk classes: sending, publishing, purchases, deleting, permissions, production changes, legal terms; "Allow once / Deny"; approval binds to the exact action | Model-based review complementing least privilege | We have `safe / monitor / ask / block` with blacklist > whitelist. **Adopt** per-agent *Require-approval rules* on top of the ceiling (§3.4) and the "binds to the exact action" wording. |+| **No model picker**, no local models, cloud only, Linux desktop unsupported | Their biggest gap | **Our edge:** provider + model + effort per agent from the existing catalog, local models included, every OS. |+| 50 Bots + group chats per account; duplicate a Bot copies profile/skills/routines but not memory; hide ≠ pause; share = config only, "strip secrets" | Housekeeping rules | **Adopt** duplicate/hide/archive semantics; sharing = M6 marketplace item with the same "no secrets" rule. |+| First run: asks which tools you use, then suggests teammates ("Piper — product performance investigator") | Onboarding by tool inventory | **Adopt:** seed agents proposed from *connected* capabilities (a Gmail agent only if the Gmail plugin is connected). |++Net: Grok Bot's product shape is the right floor. What we add is per-agent models, local-first,+structural safety, the wiki as shared memory, and the world.++---++## 2. The agent, field by field (roster row = model card)++`society_agents` grows from the MASTERPLAN sketch to this. Every field is user-visible on the+card; nothing hidden drives behavior.++**Identity**+- `agent_id` (slug, stable), `name` (UNIQUE, the display name), `title` (one job line,+ "Gmail assistant"), `description` (standing instructions, Markdown; the agent's own AGENTS.md:+ goal, ownership, working style, approval boundary, durable rules). `tier`+ (`lead | orchestrator | specialist`), `parent_agent_id`, `state` (`active | paused | archived`).+- `avatar` (parts + palette now; GLB/texture uri in M3), `checkpoint` (world place).++**Brain**+- `provider`, `model`, `effort` — from `agent_chat.catalog.rows_for("society")`: API families,+ keyless local rows (`ollama`, `local-openai`). CLI seats (Claude Code, Codex, …) are NOT+ society brains: a society agent runs on Jarvis' own harness (`brain_runner=True`,+ `cli_seats=False`, like the `jarvis` surface). CLIs are *tools* of the agent (§3.1), not its mind.+- `fallback_policy` = the global capability-gated chain (AP-21/22); no per-agent fallback pins.++**Capabilities** (§3)+- `grant_mode`: `all` (default — everything the global tiers allow) or `allowlist`.+- `grants` JSON: capability ids from the catalog (only read when `grant_mode = allowlist`).+- `focus` JSON: capability ids the agent should reach for FIRST (derived from `description`,+ editable). A "Gmail agent" gets `focus = ["plugin:gmail"]` without losing everything else.+- `denies` JSON: capability ids never offered to this agent, even in `all` mode.+- `skills` JSON: enabled skill slugs (default: all `active` skills; `draft` never).+- `workspace_dir`: the agent's folder (default `DATA_DIR/society/<agent_id>/workspace`).++**Knowledge & memory** (§5)+- `wiki_namespace`: `society/<agent_id>/` in the vault (created lazily).+- `memory_page`: `society/<agent_id>/memory.md` — the agent's own durable notes.+- `knowledge_scope`: `shared` (reads the whole vault) or `own` (only its namespace + the+ society hub pages). Default `shared`.++**Safety & economy** (§3.4)+- `permission_ceiling` (`safe | monitor | ask`; block is block, never a bypass).+- `approval_rules` JSON: Grok-style rules — `require_approval: [patterns]`,+ `always_allow: [patterns]`; require wins; patterns are capability ids or `capability:verb`+ (`plugin:gmail:send`). Global blacklist and `always_confirm_tiers` sit above all of it.+- `daily_budget_usd`, `max_concurrent_runs` (default 1 for specialists, 3 for orchestrators).++**Automation & stats**+- routines live in the Automations scheduler tagged `agent:<agent_id>` (no column);+- `stats` are derived from `society_events` (runs, cost, last active) — never stored twice.++---++## 3. Hands: how an agent gets tools from the Jarvis ecosystem++### 3.1 One capability catalog++Everything a Jarvis brain can call already exists in four registries. The society adds ONE+read-only view over them — `jarvis/society/capabilities.py` — that returns typed rows+`CapabilityRow(id, kind, label, one_liner, risk_tier, connected: bool, source)`:++| kind | id form | comes from | connected means |+|---|---|---|---|+| `plugin` | `plugin:<entry-point name>` (`plugin:gmail`, `plugin:google-calendar`, `plugin:spotify`) | `jarvis.tool` entry points (`jarvis/core/registry.py`), marketplace-installed plugins included | its credential probe is green (`brain/app_control.is_credential_present`) |+| `cli` | `cli:<name>` (`cli:gcloud`, `cli:gh`) | `jarvis.clis` registry — the `cli_<name>` tools the CLI Test Hub already drives | binary found + connected in the CLIs section |+| `mcp` | `mcp:<server>` (whole server) and `mcp:<server>/<tool>` | `jarvis/mcp/adapter.py` (`MCPToolAdapter` rows in the tool registry) | server verified (`mcp/bootstrap.verify_server`) |+| `skill` | `skill:<slug>` | `jarvis/skills/registry.py`, `active` lifecycle only | always (skills are text) |+| `core` | `core:<tool>` (`core:search-web`, `core:wiki-recall`, `core:run-shell`, `core:computer-use`, `core:read/write/edit` folder tools) | the built-in tool set minus the router-only dispatch tools | always |++Never in any society tool set, structurally (AP-5/AP-14): `spawn-worker`, `spawn-subagents`,+`multi-spawn`, `dispatch-*`, `create-artifact`, `navigate`, `switch-provider`,+`manage-mcp-server`, `app-command`. Dispatch is the scheduler's privilege; app control stays+with Jarvis.++The catalog is what the model card's "Tools" tab lists (grouped by kind, connected first, with+the same brand marks the Plugins / CLIs / MCPs sections use), what the Create dialog's Advanced+disclosure offers, and what `capability_epoch` fingerprints so an agent notices when its hands+change.++### 3.2 Grant, focus, deny — and "it just knows it is the Gmail agent"++Per turn, the society surface kit builds the agent's tool dict in this order:++1. **Start** from the full brain tool set the global safety tiers allow (what the `jarvis`+ surface gets today) — that is `grant_mode = all`. With `allowlist`, keep only `grants`.+2. **Remove** `denies`, everything unconnected (a disconnected plugin is not offered — no+ "tool not available" refusals), every dispatch tool, and every skill not in `skills`.+3. **Order** the schema list: `focus` first, then the rest alphabetically (models attend to the+ head of the tool list; prompt caching is safe because the order is deterministic per agent).+4. **Brief** the model (system_extra, §3.3): a "Your hands" section listing focus tools with+ their one-liners, then a compact "Also available" line of names by kind.++`focus` is derived ONCE at creation (and on every description edit) by+`jarvis/society/focus.py`: deterministic keyword matching of the `title` + `description`+against catalog labels, one-liners and plugin aliases ("Gmail", "mail", "inbox" → `plugin:gmail`;+"calendar", "Termin" → `plugin:google-calendar` <!-- i18n-allow: alias example -->; "GitHub", "PR" → `cli:gh` / `mcp:github`).+No LLM call for this — it must work with any single key and cost nothing. The card shows the+derived focus as chips the user can add to / remove; a wrong guess is a one-click fix.++A "Gmail agent" therefore: keeps every tool (so it can still look something up in the wiki or+the calendar), gets Gmail listed first and briefed as its primary hand, and its description+tells it what it owns. That is the Grok behavior ("the description stores durable rules") on top+of our tiers.++### 3.3 Prompt assembly — how the agent knows the ecosystem++The society kit's `system_extra` is assembled per turn from cached, byte-stable parts (prompt-+cache friendly, prefix-stable), in this order:++1. **Society persona frame** (constant): you are `<name>`, `<title>`, tier `<tier>`, in a team+ led by Jarvis; how to hand off (§4.3); what you never do (dispatch, secrets, outbound at+ scale).+2. **The agent's `description`** verbatim (its standing instructions).+3. **Capability brief** (§3.2) — focus tools with one-liners; "also available" line; the+ capability epoch fingerprint.+4. **Ecosystem card** (constant, generated from code, ~40 lines): what Jarvis is (voice lead,+ sections, Automations, wiki, missions, artifacts), which of those the agent reaches how+ (routines via the scheduler tag; wiki via `wiki-recall` / `wiki-page-read` / `wiki-ingest`;+ files via its workspace; other agents via `message_agent`), and the house rules (turn+ language, no secrets in chat, ask-tier actions queue for the human).+5. **Identity card** of the user (`brain/identity_card.py`, already cached) and the user's+ agent-instructions file — the agent knows whom it works for.+6. **Wiki context** for this turn (`brain/wiki_context`) + the agent's `memory_page` head.+7. **Roster line**: the other active agents with their titles, so the agent knows whom to+ message ("Archivist — knowledge curator").++Turn language is decided once by `turn_language.py` and passed through; agents never re-derive+it.++### 3.4 Safety composition (unchanged chokepoint, three new layers)++`ToolExecutor.execute()` stays the only executor. Per call, in order:++1. Global **blacklist** (config) — raises before anything else, every stance.+2. Agent **denies** and grant (§3.2) — a tool not in the dict cannot be called at all.+3. Agent **approval rules**: `require_approval` match → queue (unattended) or card (attended);+ else `always_allow` match → run if the tool's own tier ≤ `monitor`; else fall through.+4. Tool **risk tier** vs the agent's `permission_ceiling` (`ask` ceiling: ask-tier queues;+ `monitor` ceiling: ask-tier is refused with a typed reason, never silently skipped).+5. Society **budget** (`daily_budget_usd`, global `BudgetTracker`) and the kill switch.++The anti-harm blacklist class (mass outbound, credential probing, repeated actions against+non-consenting endpoints) is seeded at tier `block` and is not overridable by any agent field.++---++## 4. Talking: agents, Jarvis and the user++### 4.1 Surfaces+- **User ↔ agent:** the canonical chat (card's right column), `society:<agent_id>` session on+ the agent_chat store; the composer's `/` lists the agent's enabled skills, `@` the roster.+- **Jarvis ↔ agent (voice):** the `delegate-to-agent` router tool (M4): ACK < 5 s, work goes+ through the scheduler as `ASSIGN`, completion re-enters voice through `scrub_for_voice`.+- **Agent ↔ agent:** exactly one tool, `message_agent(target, text, kind)` → one `SAY` /+ `QUERY` / `PROPOSE` envelope to ONE teammate; the scheduler wakes the target's chat.+ `@name` in a canonical chat by the USER also produces a `SAY` from the user's identity.+- **Group:** a room (`ROOM_OPEN → SAY* → ROOM_SETTLE`), opened by Jarvis or an orchestrator,+ rendered as one thread in the society feed and as the meeting pavilion in the world.++### 4.2 Envelope payload (typed, compact)+`{ "type": "SAY", "text": "...", "refs": ["wiki:society/scout/2026-09-01-brief.md", "file:..."],+"lang": "de" }` — text is for humans, `refs` are what the receiver opens. Every envelope carries+`trace_id`; a reply carries `parent_event_id`.++### 4.3 Handoff = `RESULT` payload (Grok's record, made mandatory)+```+{ "type": "RESULT", "status": "done | partial | blocked",+ "output": ["file:...", "wiki:..."], # where the work is+ "done": "one paragraph", # what was completed+ "evidence": ["url", "file:...", "event:seq"], # what it was based on+ "open": ["..."], # what remains unresolved+ "next_owner": "archivist" | null, # who owns the next step+ "cost_usd": 0.12 }+```+The scheduler refuses a `RESULT` without `done` and `output`/`open`; the world plays "walks to+the archive" on it; the ledger shows the five fields.++### 4.4 Delegation rules (scheduler-enforced)+- `ASSIGN` only from `lead` / `orchestrator`; depth ≤ 2; no recursion; specialist `ASSIGN` →+ typed refusal `tier_not_allowed`.+- One `ASSIGN` = one worker under the target's identity, model, grants and ceiling; the+ worker's tool set is the target's §3.2 dict minus every dispatch tool (already absent).+- Per-trace message cap (default 24) and room caps end every conversation deterministically.++---++## 5. Memory: the Obsidian wiki is the society's shared store++Maintainer direction (2026-09-01): the agents continue and use the wiki we built as their+common memory. This refines MASTERPLAN §3.1 / §6.6 — the `knowledge` table becomes a staging+and provenance layer; the vault is where knowledge lives.++**Layout in the vault**+```+society/+ README.md # hub: what this folder is, who writes here+ <agent_id>/+ memory.md # the agent's durable notes (its "memory page")+ YYYY-MM-DD-<slug>.md # work notes, one per RESULT that produced knowledge+ shared/+ <topic>.md # curated cross-agent knowledge (reviewed)+```+Every page written by an agent carries frontmatter `author: agent:<id>`, `origin:+user|tool|web|agent`, `trace: <trace_id>`, `reviewed: false`.++**Read path.** Agents use the existing `wiki-recall`, `wiki-page-read`, `wiki-list` tools+(safe tier). `knowledge_scope = shared` reads the whole vault; per-turn `wiki_context` also+serves them. Pages with `reviewed: false` and `origin: web|agent` are surfaced to agents with a+visible "unreviewed" marker in the tool result, and never enter another agent's *system prompt*+(confused-deputy defense) — they are available on request, not ambiently.++**Write path.** `wiki-ingest` (monitor tier) is namespaced for society sessions: an agent can+write only under `society/<own id>/`. Writing to `society/shared/` or anywhere else is a+curator action (M4, event-driven after `RESULT`, local model preferred) and lands in the+approvals queue as "promote to shared knowledge" when the source is web/agent-origin. The user's+own pages are never edited by an agent.++**Why not a separate knowledge DB as the truth:** the user already reads, edits and searches the+vault; an invisible second store would split memory. The `knowledge` table keeps only what the+vault cannot: taint state, curation cursors, FTS over agent notes — derived, rebuildable.++---++## 6. Lifecycle & housekeeping (Grok parity)++- **Create**: three fields + Advanced (brain, grants, ceiling, budget, avatar). The agent+ introduces itself as the first chat message, in the turn language, naming its focus tools.+- **Seed on first run**: propose teammates from CONNECTED capabilities (Gmail plugin connected →+ "Mail agent"; `gh` connected → "Repo agent"); ship a starter coordinator once the maintainer+ decides §10.6.+- **Duplicate**: copies profile, grants, rules, skills, routines, avatar; not the chat, not the+ memory page, not the workspace.+- **Pause**: no routine fires, no `ASSIGN` accepted, chat still answers. **Hide**: roster only.+ **Archive** (delete): profile + routines gone; wiki pages stay (they are the user's vault);+ workspace dir is kept for 30 days.+- **Edit description** → focus re-derived → card shows the diff of focus chips → the canonical+ session is re-seated (transcript kept).+- **Capability epoch**: the brief's fingerprint changes when plugins/CLIs/MCPs connect or drop;+ the next turn re-briefs, the card shows "hands changed".++---++## 7. What was missing, now defined++| Gap (from the brainstorm) | Answer here |+|---|---|+| "Default connected to all plugins, but knows it is the Gmail agent" | `grant_mode=all` + `focus` derived from the description (§3.2) |+| Explicit "use only X" | `grant_mode=allowlist` + `denies` (§2, §3.2) |+| Skills / MCPs / CLIs / CLI Test Hub tools reachable | one capability catalog over the four registries (§3.1) |+| Agent knows the whole Jarvis ecosystem | ecosystem card + roster line in the prompt (§3.3) |+| Wiki as shared memory | §5, namespaced writes, reviewed promotion |+| Context exchange between agents | `refs` in envelopes, `RESULT` handoff record (§4.2–4.3) |+| Per-agent approval rules like Grok | `approval_rules` under the ceiling (§3.4) |+| Onboarding | seed proposals from connected capabilities (§6) |++---++## 8. Roadmap — where this lands in the build plan++Maps onto [`build-plan-m1-m2.md`](build-plan-m1-m2.md); new items are marked **+**.++| Wave | Adds from this document |+|---|---|+| 2 | roster schema with §2 fields (`title`, `description`, `grant_mode`, `grants`, `focus`, `denies`, `skills`, `approval_rules`, `workspace_dir`, `wiki_namespace`, `knowledge_scope`, `max_concurrent_runs`) + five-layer parity for the new enums |+| **2b +** | `jarvis/society/capabilities.py` (catalog over plugins / CLIs / MCPs / skills / core) + `focus.py` (deterministic derivation) + tests with fakes for every registry |+| 4 | scheduler enforces §4.3 `RESULT` schema and §4.4 rules; per-trace cap |+| 5 | REST: `GET /api/society/capabilities`, `POST /agents` derives focus, `PATCH` re-derives on description change; CLI `jarvis society capabilities` |+| 6 | society surface kit implements §3.2 tool dict + §3.3 prompt assembly; `message_agent` with `refs`; wiki tools namespaced (§5 write path) |+| **6b +** | ecosystem card generator (from code, cached, byte-stable) + `capability_epoch` wiring |+| 7 | card "Tools" tab reads the catalog with brand marks; focus chips; approval-rules editor |+| 9 | approvals queue applies §3.4 order; routines tagged `agent:<id>`; seed proposals from connected capabilities |+| **9b +** | vault `society/` layout, `memory.md` per agent, unreviewed marker in wiki tool results, "promote to shared" approval item |+| M4 | `delegate-to-agent` voice tool; curator writes `society/shared/`; rooms live |+| M6 | teach-by-demonstration → draft skill; agent screens; sharing without secrets |++Exit test for this document (part of the M2 contract test): create "Mailbox" with a German+sample description meaning "You are my Gmail agent. You read, sort and answer my mail; external+mail only after approval" — literally: <!-- i18n-allow: German sample description -->+"Du bist mein Gmail-Agent. Du liest, sortierst und beantwortest meine Mails; externe Mails nur nach Freigabe." <!-- i18n-allow -->+Assert `focus == ["plugin:gmail"]`, `approval_rules.require_approval` contains+`plugin:gmail:send` (derived from the "only after approval" clause), the first chat message+names Gmail as its hand, a `wiki-ingest` outside+`society/mailbox/` is refused with a typed reason, and a `send` call lands in the approvals+queue while the card shows the pending item.