Skip to content

Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.

feat(self-build): add the build ledger the self-building loop records into (#752)

Ruben Lütkeccece4c8abc3
diff --git a/docs/self-build.md b/docs/self-build.md
new file mode 100644
index 0000000000..48014f573a
--- /dev/null
+++ b/docs/self-build.md
@@ -0,0 +1,112 @@
+# Self-building loop
+
+Dodaios notices what the person keeps doing and builds what would take that
+work off them: a memory, a skill, a routine, a specialised agent or a code
+change in one of their projects. It does this without being asked, keeps
+only what proves useful, and lets the person see and undo everything.
+
+```
+ observe ──> decide ──> build ──> prove ──> trial ──> keep / archive
+ (free) (1 call) (sandbox) (judge) (live, (measured)
+ undoable)
+```
+
+- **Observe** counts repetition in the person's own actions. No model is asked
+ and untrusted content (web pages, tool results, other agents) is never
+ evidence.
+- **Decide** picks the smallest artefact that meets the need, in this order:
+ memory, skill, routine, agent, code. "Build nothing" is a valid answer.
+- **Build** goes through the existing creation paths (proposals, the skill
+ creator, routines, the roster, coding threads), never a new back door. No
+ agent gets a tool that creates agents.
+- **Prove**: a judge the builder cannot edit checks the build before it counts
+ (replay of the turns that triggered it, shadow runs, tests and CI).
+- **Trial**: the build is live, labelled "built by Dodaios" and undoable with
+ one click.
+- **Keep or archive** is decided from use, corrections and cost. Archived
+ builds are restorable, never deleted.
+
+## The ledger: `dodaios/self_build/`
+
+Every build has one row in `self_build.db` (in the data directory). Each stage
+writes there and nowhere else.
+
+| Module | Holds |
+|---|---|
+| `records.py` | `Kind`, `State`, `Outcome`, `BuildRecord`, the allowed state moves |
+| `ledger.py` | `BuildLedger`: `open_case`, `set_state`, `attach_proof`, `attach_artefact`, `record_use`, reads, rejections, meta |
+| `policy.py` | `admit` (may a build open, and how far may it go), `trial_over`, `settle`, the kill switch, the silent first days |
+| `undo.py` | `UndoRegistry`: one undoer per kind |
+| `cost.py` | `attributed(build_id)` tags model calls; `spent(record)` sums them from the cost ledger |
+| `service.py` | `SelfBuild`, started at boot; `current()` returns it; `register_builder(kind, fn)` |
+
+States:
+
+```
+proposed → building → proving → trial → kept | archived | undone
+ │ └──────────┴─────────┴──> failed
+ └─> rejected
+```
+
+`kept` and `archived` can still be undone; `archived` can be kept again.
+
+### How a builder plugs in
+
+1. Register an undoer for the kind: `service.undo.register(Kind.SKILL, fn)`. A
+ kind without an undoer is never built, only proposed.
+2. Register a builder: `service.register_builder(Kind.SKILL, fn)`. It is
+ called with the record when a proposal is accepted.
+3. To open a build, call `service.admit(kind, signature)`. `start` is `None`
+ (do not open), `PROPOSED` (open as a card) or `BUILDING` (build now).
+4. Open it with `ledger.open_case(kind, signature, title, case=..., evidence=...,
+ state=admission.start)`. `title` is one sentence the person reads.
+5. Run every model call inside `cost.attributed(record.id)`.
+6. Record what you created with `attach_artefact(id, artefact, undo)`, the
+ judge's verdict with `attach_proof`, then move the record to `trial` or
+ `failed`.
+7. Report each use with `record_use(id, Outcome.ACCEPTED | CORRECTED | USED |
+ IGNORED)`.
+
+A `signature` is a stable key for the need (for example
+`routine:weekly-sales-report`). Only one build per signature can be open at a
+time, and a signature the person rejected or undid `max_rejections` times is
+not offered again.
+
+## Settings: `[self_build]`
+
+| Key | Default | Meaning |
+|---|---|---|
+| `enabled` | `true` | master switch |
+| `silent_days` | `7` | the first days after the loop first ran only record cases |
+| `memory`, `skill`, `routine`, `agent`, `code` | `trial`, `suggest`, `suggest`, `suggest`, `suggest` | autonomy per kind: `off`, `suggest`, `trial`, `full` |
+| `trial_days`, `trial_uses` | `14`, `10` | a trial ends at whichever comes first |
+| `max_builds_per_day` | `3` | builds past the proposal stage per day |
+| `max_agents_per_week` | `1` | |
+| `max_open_trials` | `5` | |
+| `max_code_runs_per_day` | `1` | |
+| `max_rejections` | `2` | |
+
+At `suggest` a card asks first. At `trial` the build goes live on trial; an
+unused trial is archived on its own, a used one waits for the person's keep.
+At `full` a used trial is kept on its own too.
+
+## REST: `/api/self-build`
+
+`GET ""` (builds and status), `GET /digest?hours=24`, `POST /pause`
+(`{"paused": true}` is the kill switch), `GET /{id}` (record, history, cost),
+`POST /{id}/accept`, `POST /{id}/keep`, `POST /{id}/undo`. Every change
+publishes `SelfBuildChanged` on the app bus, and the WebSocket forwards it.
+
+## Rules enforced in code
+
+1. Background work for the loop never bills a per-token key. It runs on
+ subscriptions or local models, or it waits.
+2. Only the person's own actions are evidence.
+3. The builder cannot edit the judge, the budgets, `[self_build]`, the kill
+ switch, `security.*`, secrets or the self-mod allowlist.
+4. No agent tool creates agents; only the trusted builder service does.
+5. A build without an undo path is not created.
+6. Hard caps per day and week (above).
+7. Nothing leaves the machine on the loop's initiative: no messages, emails or
+ posts, and no comments in other people's repos.
+8. Dodaios never speaks about its builds unprompted.
diff --git a/dodaios/core/config.py b/dodaios/core/config.py
index f6de8c903e..dc2b8bfb04 100644
--- a/dodaios/core/config.py
+++ b/dodaios/core/config.py
@@ -4671,6 +4671,38 @@ class VoiceEngineConfig(BaseModel):
return cleaned or ["de", "en"]
+class SelfBuildConfig(BaseModel):
+ """``[self_build]`` — the self-building loop (docs/self-build.md).
+
+ Dodaios notices what the person keeps doing and builds what would take it
+ off them: a memory, a skill, a routine, an agent or a code change. Each
+ kind has its own autonomy level: ``off``, ``suggest`` (a card, nothing is
+ built before the person accepts), ``trial`` (built, checked and live with
+ one-click undo) or ``full`` (as trial, and the trial ends by measure).
+ Every key is read by ``dodaios.self_build.policy`` (AP-31).
+ """
+
+ model_config = {"extra": "allow"}
+
+ enabled: bool = True
+ #: The first days after the loop first ran only record build cases.
+ silent_days: int = Field(default=7, ge=0, le=90)
+ memory: Literal["off", "suggest", "trial", "full"] = "trial"
+ skill: Literal["off", "suggest", "trial", "full"] = "suggest"
+ routine: Literal["off", "suggest", "trial", "full"] = "suggest"
+ agent: Literal["off", "suggest", "trial", "full"] = "suggest"
+ code: Literal["off", "suggest", "trial", "full"] = "suggest"
+ #: A trial ends after this many days or this many uses, whichever is first.
+ trial_days: int = Field(default=14, ge=1, le=90)
+ trial_uses: int = Field(default=10, ge=1, le=1_000)
+ max_builds_per_day: int = Field(default=3, ge=0, le=50)
+ max_agents_per_week: int = Field(default=1, ge=0, le=10)
+ max_open_trials: int = Field(default=5, ge=0, le=50)
+ max_code_runs_per_day: int = Field(default=1, ge=0, le=20)
+ #: A need the person rejected or undid this often is not offered again.
+ max_rejections: int = Field(default=2, ge=1, le=20)
+
+
class JarvisConfig(BaseModel):
"""Root config model."""
@@ -4690,6 +4722,7 @@ class JarvisConfig(BaseModel):
# Google key routing (AI Studio vs Vertex express) — see GoogleAuthConfig.
google: GoogleAuthConfig = Field(default_factory=GoogleAuthConfig)
memory: MemoryConfig = Field(default_factory=MemoryConfig)
+ self_build: SelfBuildConfig = Field(default_factory=SelfBuildConfig)
safety: SafetyConfig = Field(default_factory=SafetyConfig)
skills: SkillsConfig = Field(default_factory=SkillsConfig)
harness: HarnessConfig = Field(default_factory=HarnessConfig)
diff --git a/dodaios/core/events.py b/dodaios/core/events.py
index fe9c5e3e4b..1d79947334 100644
--- a/dodaios/core/events.py
+++ b/dodaios/core/events.py
@@ -2712,3 +2712,18 @@ class MarketplaceItemInstalled(Event):
item_id: str = ""
title: str = ""
ready: bool = False
+
+
+@dataclass(frozen=True, slots=True)
+class SelfBuildChanged(Event):
+ """A build in the self-building ledger was opened or changed state.
+
+ Published by :class:`dodaios.self_build.service.SelfBuild` after the row
+ was committed. The WebSocket forwards it; the "Built for you" view re-reads
+ the ledger instead of waiting for its next poll.
+ """
+
+ build_id: str = ""
+ kind: str = ""
+ state: str = ""
+ title: str = ""
diff --git a/dodaios/self_build/__init__.py b/dodaios/self_build/__init__.py
new file mode 100644
index 0000000000..9d2978f7f3
--- /dev/null
+++ b/dodaios/self_build/__init__.py
@@ -0,0 +1,31 @@
+"""The self-building loop: Dodaios builds what the person keeps needing.
+
+Doc: docs/self-build.md. This package holds the parts every stage
+shares: the build ledger (:mod:`.ledger`), the lifecycle (:mod:`.records`),
+the hard limits (:mod:`.policy`), one undo for every kind (:mod:`.undo`), cost
+attribution (:mod:`.cost`) and the in-app entry point (:mod:`.service`).
+"""
+
+from .ledger import BuildLedger, DuplicateBuild
+from .policy import Admission, Autonomy
+from .records import BuildRecord, Kind, Outcome, State, TransitionError
+from .service import NoBuilder, SelfBuild, current, start
+from .undo import UndoRegistry, UndoUnavailable
+
+__all__ = [
+ "Admission",
+ "Autonomy",
+ "BuildLedger",
+ "BuildRecord",
+ "DuplicateBuild",
+ "Kind",
+ "NoBuilder",
+ "Outcome",
+ "SelfBuild",
+ "State",
+ "TransitionError",
+ "UndoRegistry",
+ "UndoUnavailable",
+ "current",
+ "start",
+]
diff --git a/dodaios/self_build/cost.py b/dodaios/self_build/cost.py
new file mode 100644
index 0000000000..92e8f8d61b
--- /dev/null
+++ b/dodaios/self_build/cost.py
@@ -0,0 +1,37 @@
+"""What a build cost: every model call for it carries its caller tag.
+
+Builders run their model calls inside :func:`attributed`; the cost ledger
+then stores ``caller = "self_build:<id>"`` and :func:`spent` sums it.
+"""
+
+from __future__ import annotations
+
+import contextlib
+from collections.abc import Iterator
+from pathlib import Path
+
+from dodaios.costs.ledger import ledger_path, read_usage, usage_context
+
+from .ledger import now_ms
+from .records import BuildRecord, caller_for
+
+
+def attributed(build_id: str) -> Iterator[None]:
+ """Tag every model call inside the block with the build's caller."""
+ with usage_context(caller_for(build_id)):
+ yield
+
+
+def spent(record: BuildRecord, *, path: Path | None = None) -> dict[str, float]:
+ """Tokens and dollars recorded for ``record`` since it was opened."""
+ source = path or ledger_path()
+ totals = {"tokens": 0.0, "usd": 0.0}
+ if not source.is_file():
+ return totals
+ caller = record.caller
+ for row in read_usage(source, record.created_ms, now_ms()):
+ if row.caller == caller:
+ totals["tokens"] += row.tokens_in + row.tokens_out
+ totals["usd"] += row.cost_usd
+ return totals
diff --git a/dodaios/self_build/ledger.py b/dodaios/self_build/ledger.py
new file mode 100644
index 0000000000..82662cd681
--- /dev/null
+++ b/dodaios/self_build/ledger.py
@@ -0,0 +1,393 @@
+"""``self_build.db``: one row per self-built thing, the loop's only record.
+
+Every stage of the self-building loop writes here and nowhere else: the
+observer opens a case, builders attach what they created and how to undo it,
+the judge attaches its verdict, use and outcome are counted, and the person's
+keep / undo decisions land here too. The "Built for you" view reads it.
+
+Plain ``sqlite3`` in WAL mode behind one lock: calls are short, so async
+callers wrap them in ``asyncio.to_thread``. Only the server process writes.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import sqlite3
+import threading
+import time
+import uuid
+from collections.abc import Callable, Iterable
+from pathlib import Path
+from typing import Any, Final
+
+from .records import (
+ OPEN_STATES,
+ BuildRecord,
+ Kind,
+ Outcome,
+ State,
+ TransitionError,
+ can_move,
+ dumps,
+)
+
+log = logging.getLogger(__name__)
+
+DB_NAME: Final[str] = "self_build.db"
+
+_SCHEMA: Final[str] = """
+CREATE TABLE IF NOT EXISTS builds (
+ id TEXT PRIMARY KEY,
+ kind TEXT NOT NULL,
+ state TEXT NOT NULL,
+ signature TEXT NOT NULL,
+ title TEXT NOT NULL DEFAULT '',
+ case_text TEXT NOT NULL DEFAULT '',
+ evidence_json TEXT NOT NULL DEFAULT '[]',
+ proof_json TEXT NOT NULL DEFAULT '{}',
+ artefact_json TEXT NOT NULL DEFAULT '{}',
+ undo_json TEXT NOT NULL DEFAULT '{}',
+ uses INTEGER NOT NULL DEFAULT 0,
+ outcomes_json TEXT NOT NULL DEFAULT '{}',
+ note TEXT NOT NULL DEFAULT '',
+ created_ms INTEGER NOT NULL,
+ updated_ms INTEGER NOT NULL,
+ trial_started_ms INTEGER NOT NULL DEFAULT 0,
+ last_used_ms INTEGER NOT NULL DEFAULT 0
+);
+CREATE INDEX IF NOT EXISTS builds_state ON builds(state);
+CREATE INDEX IF NOT EXISTS builds_signature ON builds(signature);
+CREATE TABLE IF NOT EXISTS history (
+ seq INTEGER PRIMARY KEY AUTOINCREMENT,
+ build_id TEXT NOT NULL,
+ ts_ms INTEGER NOT NULL,
+ src TEXT NOT NULL,
+ dst TEXT NOT NULL,
+ note TEXT NOT NULL DEFAULT ''
+);
+CREATE INDEX IF NOT EXISTS history_build ON history(build_id);
+CREATE TABLE IF NOT EXISTS meta (
+ key TEXT PRIMARY KEY,
+ value TEXT NOT NULL
+);
+CREATE TABLE IF NOT EXISTS rejections (
+ signature TEXT PRIMARY KEY,
+ count INTEGER NOT NULL DEFAULT 0,
+ last_ms INTEGER NOT NULL DEFAULT 0
+);
+"""
+
+_COLUMNS: Final[str] = (
+ "id, kind, state, signature, title, case_text, evidence_json, proof_json, "
+ "artefact_json, undo_json, uses, outcomes_json, note, created_ms, updated_ms, "
+ "trial_started_ms, last_used_ms"
+)
+
+#: Called after every committed change with the record as it now stands.
+ChangeHook = Callable[[BuildRecord], None]
+
+
+def now_ms() -> int:
+ return int(time.time() * 1000)
+
+
+def _row(row: sqlite3.Row | tuple[Any, ...]) -> BuildRecord:
+ return BuildRecord(
+ id=str(row[0]),
+ kind=Kind(str(row[1])),
+ state=State(str(row[2])),
+ signature=str(row[3]),
+ title=str(row[4]),
+ case=str(row[5]),
+ evidence=tuple(json.loads(row[6] or "[]")),
+ proof=json.loads(row[7] or "{}"),
+ artefact=json.loads(row[8] or "{}"),
+ undo=json.loads(row[9] or "{}"),
+ uses=int(row[10]),
+ outcomes={str(k): int(v) for k, v in json.loads(row[11] or "{}").items()},
+ note=str(row[12]),
+ created_ms=int(row[13]),
+ updated_ms=int(row[14]),
+ trial_started_ms=int(row[15]),
+ last_used_ms=int(row[16]),
+ )
+
+
+class DuplicateBuild(ValueError):
+ """An open build already answers this signature."""
+
+
+class BuildLedger:
+ def __init__(self, path: Path, *, on_change: ChangeHook | None = None) -> None:
+ self._path = Path(path)
+ self._lock = threading.RLock()
+ self._on_change = on_change
+ self._path.parent.mkdir(parents=True, exist_ok=True)
+ self._conn = sqlite3.connect(str(self._path), check_same_thread=False, timeout=5.0)
+ self._conn.execute("PRAGMA journal_mode=WAL")
+ self._conn.execute("PRAGMA busy_timeout=5000")
+ self._conn.executescript(_SCHEMA)
+ self._conn.commit()
+
+ @property
+ def path(self) -> Path:
+ return self._path
+
+ def set_change_hook(self, hook: ChangeHook | None) -> None:
+ self._on_change = hook
+
+ def close(self) -> None:
+ with self._lock:
+ self._conn.close()
+
+ # ------------------------------------------------------------ reads
+
+ def get(self, build_id: str) -> BuildRecord | None:
+ with self._lock:
+ row = self._conn.execute(
+ f"SELECT {_COLUMNS} FROM builds WHERE id = ?", (build_id,) # noqa: S608 — fixed identifiers, bound values
+ ).fetchone()
+ return _row(row) if row else None
+
+ def list(
+ self,
+ *,
+ states: Iterable[State] | None = None,
+ kinds: Iterable[Kind] | None = None,
+ limit: int = 200,
+ ) -> list[BuildRecord]:
+ sql = f"SELECT {_COLUMNS} FROM builds" # noqa: S608 — fixed identifiers, bound values
+ where: list[str] = []
+ args: list[Any] = []
+ if states is not None:
+ values = [State(s).value for s in states]
+ where.append(f"state IN ({','.join('?' * len(values))})" if values else "0")
+ args += values
+ if kinds is not None:
+ values = [Kind(k).value for k in kinds]
+ where.append(f"kind IN ({','.join('?' * len(values))})" if values else "0")
+ args += values
+ if where:
+ sql += " WHERE " + " AND ".join(where)
+ sql += " ORDER BY updated_ms DESC LIMIT ?"
+ args.append(max(1, int(limit)))
+ with self._lock:
+ rows = self._conn.execute(sql, args).fetchall()
+ return [_row(r) for r in rows]
+
+ def open_for(self, signature: str) -> BuildRecord | None:
+ """The open build answering ``signature``, if any."""
+ states = [s.value for s in OPEN_STATES]
+ with self._lock:
+ row = self._conn.execute(
+ f"SELECT {_COLUMNS} FROM builds WHERE signature = ? AND state IN " # noqa: S608 — fixed identifiers, bound values
+ f"({','.join('?' * len(states))}) LIMIT 1",
+ (signature, *states),
+ ).fetchone()
+ return _row(row) if row else None
+
+ def ever_for(self, signature: str) -> list[BuildRecord]:
+ with self._lock:
+ rows = self._conn.execute(
+ f"SELECT {_COLUMNS} FROM builds WHERE signature = ? ORDER BY created_ms", # noqa: S608 — fixed identifiers, bound values
+ (signature,),
+ ).fetchall()
+ return [_row(r) for r in rows]
+
+ def count_since(
+ self, since_ms: int, *, kind: Kind | None = None, built_only: bool = False
+ ) -> int:
+ """Builds opened since ``since_ms`` (``built_only``: past the proposal)."""
+ sql = "SELECT COUNT(*) FROM builds WHERE created_ms >= ?"
+ args: list[Any] = [since_ms]
+ if kind is not None:
+ sql += " AND kind = ?"
+ args.append(Kind(kind).value)
+ if built_only:
+ sql += " AND state NOT IN ('proposed', 'rejected')"
+ with self._lock:
+ return int(self._conn.execute(sql, args).fetchone()[0])
+
+ def count_in(self, states: Iterable[State], *, kind: Kind | None = None) -> int:
+ values = [State(s).value for s in states]
+ if not values:
+ return 0
+ sql = f"SELECT COUNT(*) FROM builds WHERE state IN ({','.join('?' * len(values))})" # noqa: S608 — fixed identifiers, bound values
+ args: list[Any] = list(values)
+ if kind is not None:
+ sql += " AND kind = ?"
+ args.append(Kind(kind).value)
+ with self._lock:
+ return int(self._conn.execute(sql, args).fetchone()[0])
+
+ def history(self, build_id: str) -> list[dict[str, Any]]:
+ with self._lock:
+ rows = self._conn.execute(
+ "SELECT ts_ms, src, dst, note FROM history WHERE build_id = ? ORDER BY seq",
+ (build_id,),
+ ).fetchall()
+ return [{"ts_ms": r[0], "from": r[1], "to": r[2], "note": r[3]} for r in rows]
+
+ def rejections(self, signature: str) -> int:
+ """How often the person rejected or undid a build for ``signature``."""
+ with self._lock:
+ row = self._conn.execute(
+ "SELECT count FROM rejections WHERE signature = ?", (signature,)
+ ).fetchone()
+ return int(row[0]) if row else 0
+
+ # ------------------------------------------------------------ writes
+
+ def open_case(
+ self,
+ kind: Kind,
+ signature: str,
+ title: str,
+ *,
+ case: str = "",
+ evidence: Iterable[str] = (),
+ state: State = State.PROPOSED,
+ ) -> BuildRecord:
+ """Record a new build case. One open build per signature."""
+ kind = Kind(kind)
+ state = State(state)
+ if state not in (State.PROPOSED, State.BUILDING):
+ raise TransitionError(f"a build starts as proposed or building, not {state}")
+ signature = signature.strip()
+ if not signature:
+ raise ValueError("signature is required")
+ build_id = uuid.uuid4().hex[:16]
+ ts = now_ms()
+ with self._lock:
+ if self.open_for(signature) is not None:
+ raise DuplicateBuild(signature)
+ self._conn.execute(
+ f"INSERT INTO builds ({_COLUMNS}) VALUES " # noqa: S608 — fixed identifiers, bound values
+ "(?, ?, ?, ?, ?, ?, ?, '{}', '{}', '{}', 0, '{}', '', ?, ?, 0, 0)",
+ (
+ build_id,
+ kind.value,
+ state.value,
+ signature,
+ title.strip()[:300],
+ case.strip()[:4000],
+ dumps([str(e)[:300] for e in evidence][:50]),
+ ts,
+ ts,
+ ),
+ )
+ self._log(build_id, "", state.value, "opened", ts)
+ self._conn.commit()
+ record = self.get(build_id)
+ assert record is not None
+ self._emit(record)
+ return record
+
+ def set_state(self, build_id: str, state: State, note: str = "") -> BuildRecord:
+ state = State(state)
+ ts = now_ms()
+ with self._lock:
+ record = self._require(build_id)
+ if record.state == state:
+ return record
+ if not can_move(record.state, state):
+ raise TransitionError(f"{record.state} -> {state} is not allowed")
+ trial = ts if state == State.TRIAL else record.trial_started_ms
+ self._conn.execute(
+ "UPDATE builds SET state = ?, note = ?, updated_ms = ?, trial_started_ms = ?"
+ " WHERE id = ?",
+ (state.value, note.strip()[:500], ts, trial, build_id),
+ )
+ if state in (State.REJECTED, State.UNDONE):
+ self._conn.execute(
+ "INSERT INTO rejections (signature, count, last_ms) VALUES (?, 1, ?)"
+ " ON CONFLICT(signature) DO UPDATE SET count = count + 1, last_ms = ?",
+ (record.signature, ts, ts),
+ )
+ self._log(build_id, record.state.value, state.value, note, ts)
+ self._conn.commit()
+ updated = self._require(build_id)
+ self._emit(updated)
+ return updated
+
+ def attach_proof(
+ self, build_id: str, *, passed: bool, judge: str, detail: str = ""
+ ) -> BuildRecord:
+ proof = {"passed": bool(passed), "judge": judge[:120], "detail": detail[:2000]}
+ return self._update(build_id, proof_json=dumps(proof))
+
+ def attach_artefact(
+ self, build_id: str, artefact: dict[str, Any], undo: dict[str, Any]
+ ) -> BuildRecord:
+ """Record what was created and what its undoer needs to remove it."""
+ return self._update(build_id, artefact_json=dumps(artefact), undo_json=dumps(undo))
+
+ def record_use(self, build_id: str, outcome: Outcome = Outcome.USED) -> BuildRecord:
+ outcome = Outcome(outcome)
+ ts = now_ms()
+ with self._lock:
+ record = self._require(build_id)
+ outcomes = dict(record.outcomes)
+ outcomes[outcome.value] = outcomes.get(outcome.value, 0) + 1
+ uses = record.uses + (1 if outcome != Outcome.IGNORED else 0)
+ self._conn.execute(
+ "UPDATE builds SET uses = ?, outcomes_json = ?, last_used_ms = ?, updated_ms = ?"
+ " WHERE id = ?",
+ (uses, dumps(outcomes), ts, ts, build_id),
+ )
+ self._conn.commit()
+ updated = self._require(build_id)
+ self._emit(updated)
+ return updated
+
+ def get_meta(self, key: str, default: str = "") -> str:
+ with self._lock:
+ row = self._conn.execute("SELECT value FROM meta WHERE key = ?", (key,)).fetchone()
+ return str(row[0]) if row else default
+
+ def set_meta(self, key: str, value: str) -> None:
+ with self._lock:
+ self._conn.execute(
+ "INSERT INTO meta (key, value) VALUES (?, ?)"
+ " ON CONFLICT(key) DO UPDATE SET value = excluded.value",
+ (key, value),
+ )
+ self._conn.commit()
+
+ # ------------------------------------------------------------ internals
+
+ def _require(self, build_id: str) -> BuildRecord:
+ record = self.get(build_id)
+ if record is None:
+ raise KeyError(build_id)
+ return record
+
+ def _update(self, build_id: str, **columns: Any) -> BuildRecord:
+ ts = now_ms()
+ sets = ", ".join(f"{name} = ?" for name in columns)
+ with self._lock:
+ self._require(build_id)
+ self._conn.execute(
+ f"UPDATE builds SET {sets}, updated_ms = ? WHERE id = ?", # noqa: S608 — fixed identifiers, bound values
+ (*columns.values(), ts, build_id),
+ )
+ self._conn.commit()
+ updated = self._require(build_id)
+ self._emit(updated)
+ return updated
+
+ def _log(self, build_id: str, src: str, dst: str, note: str, ts: int) -> None:
+ self._conn.execute(
+ "INSERT INTO history (build_id, ts_ms, src, dst, note) VALUES (?, ?, ?, ?, ?)",
+ (build_id, ts, src, dst, note.strip()[:500]),
+ )
+
+ def _emit(self, record: BuildRecord) -> None:
+ hook = self._on_change
+ if hook is None:
+ return
+ try:
+ hook(record)
+ except Exception: # noqa: BLE001 — a listener must never undo a committed write
+ log.warning("self_build: change hook failed", exc_info=True)
diff --git a/dodaios/self_build/policy.py b/dodaios/self_build/policy.py
new file mode 100644
index 0000000000..845bc7e434
--- /dev/null
+++ b/dodaios/self_build/policy.py
@@ -0,0 +1,175 @@
+"""The hard limits of the self-building loop, decided in Python.
+
+Nothing here asks a model. Every builder calls :func:`admit` before it opens a
+build and :func:`settle` when a trial ends, so the autonomy level per kind, the
+silent first days, the kill switch and the caps hold no matter what a prompt
+says (docs/self-build.md, "Rules enforced in code").
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from enum import StrEnum
+from typing import Any, Final
+
+from .ledger import BuildLedger, now_ms
+from .records import BuildRecord, Kind, Outcome, State
+
+_DAY_MS: Final[int] = 86_400_000
+
+#: Meta keys in ``self_build.db``.
+META_PAUSED: Final[str] = "paused"
+META_FIRST_SEEN: Final[str] = "first_seen_ms"
+
+
+class Autonomy(StrEnum):
+ #: Nothing of this kind is built or proposed.
+ OFF = "off"
+ #: A card the person accepts or rejects; nothing exists before that.
+ SUGGEST = "suggest"
+ #: Built, proven and live on trial with one-click undo.
+ TRIAL = "trial"
+ #: As trial, and the trial ends by measure without a card.
+ FULL = "full"
+
+
+def section(config: Any) -> Any:
+ """The ``[self_build]`` section, or ``None`` on a bare config."""
+ return getattr(config, "self_build", None)
+
+
+def autonomy(config: Any, kind: Kind) -> Autonomy:
+ cfg = section(config)
+ if cfg is None or not cfg.enabled:
+ return Autonomy.OFF
+ value = {
+ Kind.MEMORY: cfg.memory,
+ Kind.SKILL: cfg.skill,
+ Kind.ROUTINE: cfg.routine,
+ Kind.AGENT: cfg.agent,
+ Kind.CODE: cfg.code,
+ }[Kind(kind)]
+ try:
+ return Autonomy(str(value))
+ except ValueError: # an unknown level falls back to the safe one: suggest
+ return Autonomy.SUGGEST
+
+
+def paused(ledger: BuildLedger) -> bool:
+ """The kill switch: on, nothing is observed, built or settled."""
+ return ledger.get_meta(META_PAUSED) == "1"
+
+
+def set_paused(ledger: BuildLedger, value: bool) -> None:
+ ledger.set_meta(META_PAUSED, "1" if value else "0")
+
+
+def first_seen_ms(ledger: BuildLedger, *, now: int | None = None) -> int:
+ raw = ledger.get_meta(META_FIRST_SEEN)
+ if raw.isdigit():
+ return int(raw)
+ stamp = now if now is not None else now_ms()
+ ledger.set_meta(META_FIRST_SEEN, str(stamp))
+ return stamp
+
+
+def silent(ledger: BuildLedger, config: Any, *, now: int | None = None) -> bool:
+ """The first days after the loop first ran: cases are recorded, nothing is built."""
+ cfg = section(config)
+ days = int(getattr(cfg, "silent_days", 0) or 0) if cfg is not None else 0
+ if days <= 0:
+ return False
+ stamp = now if now is not None else now_ms()
+ return stamp - first_seen_ms(ledger, now=stamp) < days * _DAY_MS
+
+
+@dataclass(frozen=True, slots=True)
+class Admission:
+ #: The state the build may open in; ``None`` = do not open it at all.
+ start: State | None
+ reason: str
+
+ @property
+ def build(self) -> bool:
+ return self.start == State.BUILDING
+
+
+def admit(
+ ledger: BuildLedger,
+ config: Any,
+ kind: Kind,
+ signature: str,
+ *,
+ undoable: bool,
+ now: int | None = None,
+) -> Admission:
+ """May a build of ``kind`` for ``signature`` be opened, and how far may it go?
+
+ ``undoable`` is whether an undoer is registered for the kind: a build that
+ could not be undone is never made (rule 5).
+ """
+ kind = Kind(kind)
+ stamp = now if now is not None else now_ms()
+ cfg = section(config)
+ level = autonomy(config, kind)
+ if cfg is None or level == Autonomy.OFF:
+ return Admission(None, "off")
+ if paused(ledger):
+ return Admission(None, "paused")
+ if ledger.open_for(signature) is not None:
+ return Admission(None, "already open")
+ if ledger.rejections(signature) >= cfg.max_rejections:
+ return Admission(None, "rejected before")
+ if level == Autonomy.SUGGEST or silent(ledger, config, now=stamp):
+ reason = "suggest" if level == Autonomy.SUGGEST else "silent"
+ if ledger.count_in([State.PROPOSED]) >= cfg.max_open_trials * 2:
+ return Admission(None, "too many open proposals")
+ return Admission(State.PROPOSED, reason)
+ if not undoable:
+ return Admission(State.PROPOSED, "no undo for this kind")
+ if ledger.count_since(stamp - _DAY_MS, built_only=True) >= cfg.max_builds_per_day:
+ return Admission(State.PROPOSED, "daily build cap")
+ if ledger.count_in([State.TRIAL]) >= cfg.max_open_trials:
+ return Admission(State.PROPOSED, "trial cap")
+ if (
+ kind == Kind.AGENT
+ and ledger.count_since(stamp - 7 * _DAY_MS, kind=Kind.AGENT, built_only=True)
+ >= cfg.max_agents_per_week
+ ):
+ return Admission(State.PROPOSED, "weekly agent cap")
+ if (
+ kind == Kind.CODE
+ and ledger.count_since(stamp - _DAY_MS, kind=Kind.CODE, built_only=True)
+ >= cfg.max_code_runs_per_day
+ ):
+ return Admission(State.PROPOSED, "daily code cap")
+ return Admission(State.BUILDING, level.value)
+
+
+def trial_over(record: BuildRecord, config: Any, *, now: int | None = None) -> bool:
+ """Has a build on trial run long enough, or been used enough, to be judged?"""
+ if record.state != State.TRIAL or not record.trial_started_ms:
+ return False
+ cfg = section(config)
+ if cfg is None:
+ return False
+ stamp = now if now is not None else now_ms()
+ return (
+ stamp - record.trial_started_ms >= cfg.trial_days * _DAY_MS
+ or record.uses >= cfg.trial_uses
+ )
+
+
+def settle(record: BuildRecord) -> State:
+ """Keep or archive a build whose trial is over, from what it did.
+
+ Kept when it was used at least twice and more of its uses were accepted
+ as they were than corrected. Archived (never deleted) otherwise.
+ """
+ corrected = record.outcomes.get(Outcome.CORRECTED.value, 0)
+ accepted = record.outcomes.get(Outcome.ACCEPTED.value, 0) + record.outcomes.get(
+ Outcome.USED.value, 0
+ )
+ if record.uses >= 2 and accepted > corrected:
+ return State.KEPT
+ return State.ARCHIVED
diff --git a/dodaios/self_build/records.py b/dodaios/self_build/records.py
new file mode 100644
index 0000000000..4599dc7c9f
--- /dev/null
+++ b/dodaios/self_build/records.py
@@ -0,0 +1,146 @@
+"""What one self-built thing is, and the states it may move through.
+
+A *build* is anything Dodaios made on its own initiative: a memory entry, a
+skill, a routine, a specialised agent or a code change in one of the person's
+projects. Every build has one :class:`BuildRecord` in the ledger, so the
+person can always see why it exists, how it was checked, what it cost and how
+to undo it (docs/self-build.md).
+"""
+
+from __future__ import annotations
+
+import json
+from dataclasses import dataclass, field
+from enum import StrEnum
+from typing import Any, Final
+
+
+class Kind(StrEnum):
+ """The artefact ladder, cheapest first."""
+
+ MEMORY = "memory"
+ SKILL = "skill"
+ ROUTINE = "routine"
+ AGENT = "agent"
+ CODE = "code"
+
+
+class State(StrEnum):
+ #: A build case was written; nothing exists yet. With autonomy ``suggest``
+ #: this is the card the person accepts or rejects.
+ PROPOSED = "proposed"
+ BUILDING = "building"
+ PROVING = "proving"
+ #: Live, labelled, undoable; value is being measured.
+ TRIAL = "trial"
+ KEPT = "kept"
+ ARCHIVED = "archived"
+ UNDONE = "undone"
+ REJECTED = "rejected"
+ FAILED = "failed"
+
+
+class Outcome(StrEnum):
+ """What happened when a build was used."""
+
+ USED = "used"
+ ACCEPTED = "accepted"
+ CORRECTED = "corrected"
+ IGNORED = "ignored"
+
+
+#: States a build can still leave.
+OPEN_STATES: Final[frozenset[State]] = frozenset(
+ {State.PROPOSED, State.BUILDING, State.PROVING, State.TRIAL}
+)
+#: States in which an artefact exists and an undo has something to remove.
+LIVE_STATES: Final[frozenset[State]] = frozenset({State.TRIAL, State.KEPT})
+
+_ALLOWED: Final[dict[State, frozenset[State]]] = {
+ State.PROPOSED: frozenset({State.BUILDING, State.REJECTED, State.FAILED}),
+ State.BUILDING: frozenset({State.PROVING, State.TRIAL, State.FAILED, State.UNDONE}),
+ State.PROVING: frozenset({State.TRIAL, State.FAILED, State.UNDONE}),
+ State.TRIAL: frozenset({State.KEPT, State.ARCHIVED, State.UNDONE, State.FAILED}),
+ State.KEPT: frozenset({State.ARCHIVED, State.UNDONE}),
+ State.ARCHIVED: frozenset({State.KEPT, State.UNDONE}),
+ State.UNDONE: frozenset(),
+ State.REJECTED: frozenset(),
+ State.FAILED: frozenset(),
+}
+
+
+def can_move(src: State, dst: State) -> bool:
+ return dst in _ALLOWED[src]
+
+
+class TransitionError(ValueError):
+ """A state change the lifecycle does not allow."""
+
+
+@dataclass(frozen=True, slots=True)
+class BuildRecord:
+ id: str
+ kind: Kind
+ state: State
+ #: Stable key of the need this build answers; one open build per signature.
+ signature: str
+ #: One sentence the person reads: why this build exists.
+ title: str
+ #: The build case: need, expected value, expected cost, how it is checked.
+ case: str = ""
+ #: References to the person's own turns or actions that justify the build.
+ evidence: tuple[str, ...] = ()
+ #: Verdict of the judge (``{"passed": bool, "judge": str, "detail": str}``).
+ proof: dict[str, Any] = field(default_factory=dict)
+ #: What was created (``{"type": "skill", "id": "..."}``), empty before that.
+ artefact: dict[str, Any] = field(default_factory=dict)
+ #: Kind-specific data the registered undoer needs.
+ undo: dict[str, Any] = field(default_factory=dict)
+ uses: int = 0
+ outcomes: dict[str, int] = field(default_factory=dict)
+ note: str = ""
+ created_ms: int = 0
+ updated_ms: int = 0
+ trial_started_ms: int = 0
+ last_used_ms: int = 0
+
+ @property
+ def open(self) -> bool:
+ return self.state in OPEN_STATES
+
+ @property
+ def caller(self) -> str:
+ """The cost-ledger caller tag every model call for this build carries."""
+ return caller_for(self.id)
+
+ def to_json(self) -> dict[str, Any]:
+ return {
+ "id": self.id,
+ "kind": self.kind.value,
+ "state": self.state.value,
+ "signature": self.signature,
+ "title": self.title,
+ "case": self.case,
+ "evidence": list(self.evidence),
+ "proof": dict(self.proof),
+ "artefact": dict(self.artefact),
+ "uses": self.uses,
+ "outcomes": dict(self.outcomes),
+ "note": self.note,
+ "created_ms": self.created_ms,
+ "updated_ms": self.updated_ms,
+ "trial_started_ms": self.trial_started_ms,
+ "last_used_ms": self.last_used_ms,
+ "undoable": self.state in LIVE_STATES or self.state in OPEN_STATES,
+ }
+
+
+CALLER_PREFIX: Final[str] = "self_build:"
+
+
+def caller_for(build_id: str) -> str:
+ return f"{CALLER_PREFIX}{build_id}"
+
+
+def dumps(value: Any) -> str:
+ return json.dumps(value, ensure_ascii=False, sort_keys=True)
diff --git a/dodaios/self_build/service.py b/dodaios/self_build/service.py
new file mode 100644
index 0000000000..bcf32f878d
--- /dev/null
+++ b/dodaios/self_build/service.py
@@ -0,0 +1,236 @@
+"""The self-building loop's one entry point inside the running app.
+
+Built once at boot (:func:`start`), cheap: it opens ``self_build.db`` and
+nothing else. Observers and builders reach it through :func:`current`; the
+REST routes through ``app.state.self_build``.
+
+Builders plug in per kind:
+
+* ``register_builder(kind, fn)``: called with the record when a proposed build
+ is accepted (or admitted straight to building); it builds, proves and moves
+ the record on.
+* ``undo.register(kind, fn)``: removes what the builder created.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+from collections.abc import Awaitable, Callable
+from pathlib import Path
+from typing import Any
+
+from . import policy
+from .ledger import DB_NAME, BuildLedger, now_ms
+from .records import LIVE_STATES, BuildRecord, Kind, State
+from .undo import UndoRegistry
+
+log = logging.getLogger(__name__)
+
+Builder = Callable[[BuildRecord], Awaitable[None]]
+Publish = Callable[[Any], Awaitable[None]]
+
+_current: SelfBuild | None = None
+
+
+class NoBuilder(RuntimeError):
+ """Nothing is registered that can build this kind yet."""
+
+
+class SelfBuild:
+ def __init__(
+ self,
+ data_dir: Path,
+ config: Callable[[], Any],
+ *,
+ publish: Publish | None = None,
+ ) -> None:
+ self._config = config
+ self._publish = publish
+ self._loop: asyncio.AbstractEventLoop | None = None
+ self._builders: dict[Kind, Builder] = {}
+ self._tasks: set[asyncio.Task[None]] = set()
+ self._sweeper: asyncio.Task[None] | None = None
+ self.undo = UndoRegistry()
+ self.ledger = BuildLedger(Path(data_dir) / DB_NAME, on_change=self._changed)
+
+ # ------------------------------------------------------------ wiring
+
+ def bind_loop(self, loop: asyncio.AbstractEventLoop) -> None:
+ self._loop = loop
+
+ def config(self) -> Any:
+ return self._config()
+
+ def register_builder(self, kind: Kind, builder: Builder) -> None:
+ self._builders[Kind(kind)] = builder
+
+ def can_build(self, kind: Kind) -> bool:
+ return Kind(kind) in self._builders
+
+ # ------------------------------------------------------------ the gate
+
+ def admit(self, kind: Kind, signature: str) -> policy.Admission:
+ return policy.admit(
+ self.ledger, self.config(), kind, signature, undoable=self.undo.has(kind)
+ )
+
+ def status(self) -> dict[str, Any]:
+ cfg = self.config()
+ section = policy.section(cfg)
+ return {
+ "enabled": bool(section is not None and section.enabled),
+ "paused": policy.paused(self.ledger),
+ "silent": policy.silent(self.ledger, cfg),
+ "autonomy": {kind.value: policy.autonomy(cfg, kind).value for kind in Kind},
+ "builders": sorted(kind.value for kind in self._builders),
+ }
+
+ # ------------------------------------------------------------ person's actions
+
+ async def accept(self, build_id: str) -> BuildRecord:
+ """The person accepted a proposed build: hand it to its builder."""
+ record = await asyncio.to_thread(self.ledger.get, build_id)
+ if record is None:
+ raise KeyError(build_id)
+ if record.state != State.PROPOSED:
+ return record
+ builder = self._builders.get(record.kind)
+ if builder is None or not self.undo.has(record.kind):
+ raise NoBuilder(record.kind.value)
+ record = await asyncio.to_thread(
+ self.ledger.set_state, build_id, State.BUILDING, "accepted"
+ )
+ self.spawn(builder(record), f"build {build_id}")
+ return record
+
+ async def keep(self, build_id: str) -> BuildRecord:
+ return await asyncio.to_thread(self.ledger.set_state, build_id, State.KEPT, "kept")
+
+ async def undo_build(self, build_id: str, note: str = "") -> BuildRecord:
+ return await self.undo.undo(self.ledger, build_id, note)
+
+ def set_paused(self, value: bool) -> None:
+ policy.set_paused(self.ledger, value)
+
+ # ------------------------------------------------------------ trials
+
+ async def sweep(self) -> list[BuildRecord]:
+ """Settle trials that are over. Archiving is automatic (restorable);
+ keeping without a look from the person only at autonomy ``full``."""
+ if policy.paused(self.ledger):
+ return []
+ cfg = self.config()
+ settled: list[BuildRecord] = []
+ trials = await asyncio.to_thread(self.ledger.list, states=[State.TRIAL], limit=500)
+ for record in trials:
+ if not policy.trial_over(record, cfg):
+ continue
+ verdict = policy.settle(record)
+ if verdict == State.KEPT and policy.autonomy(cfg, record.kind) != policy.Autonomy.FULL:
+ continue
+ note = (
+ "kept: used and accepted"
+ if verdict == State.KEPT
+ else "archived: not used enough"
+ )
+ settled.append(
+ await asyncio.to_thread(self.ledger.set_state, record.id, verdict, note)
+ )
+ return settled
+
+ def start_sweeper(self, interval_s: float = 3_600.0) -> None:
+ """Settle finished trials once an hour; no model is ever asked."""
+ if self._sweeper is not None:
+ return
+
+ async def loop() -> None:
+ while True:
+ await asyncio.sleep(interval_s)
+ try:
+ await self.sweep()
+ except Exception: # noqa: BLE001 — the next pass retries
+ log.warning("self_build: sweep failed", exc_info=True)
+
+ try:
+ self._sweeper = asyncio.get_running_loop().create_task(loop())
+ except RuntimeError: # no loop yet (tests, CLI): sweep() can still be called
+ self._sweeper = None
+
+ def digest(self, since_ms: int) -> dict[str, Any]:
+ """What happened since ``since_ms``, for the daily card."""
+ rows = [r for r in self.ledger.list(limit=500) if r.updated_ms >= since_ms]
+ by_state: dict[str, int] = {}
+ for record in rows:
+ by_state[record.state.value] = by_state.get(record.state.value, 0) + 1
+ return {
+ "since_ms": since_ms,
+ "until_ms": now_ms(),
+ "counts": by_state,
+ "live": sum(1 for r in self.ledger.list(states=LIVE_STATES, limit=500)),
+ "items": [r.to_json() for r in rows[:20]],
+ }
+
+ # ------------------------------------------------------------ plumbing
+
+ def spawn(self, coro: Awaitable[None], label: str) -> None:
+ async def run() -> None:
+ try:
+ await coro
+ except Exception: # noqa: BLE001 — one failed build must not stop the loop
+ log.warning("self_build: %s failed", label, exc_info=True)
+
+ task = asyncio.ensure_future(run())
+ self._tasks.add(task)
+ task.add_done_callback(self._tasks.discard)
+
+ def _changed(self, record: BuildRecord) -> None:
+ publish, loop = self._publish, self._loop
+ if publish is None or loop is None or loop.is_closed():
+ return
+ from dodaios.core.events import SelfBuildChanged
+
+ event = SelfBuildChanged(
+ build_id=record.id,
+ kind=record.kind.value,
+ state=record.state.value,
+ title=record.title,
+ source_layer="self_build",
+ )
+
+ def fire() -> None:
+ self.spawn(publish(event), "publish")
+
+ try:
+ running = asyncio.get_running_loop()
+ except RuntimeError: # called from a worker thread: hop onto the bound loop
+ running = None
+ if running is loop:
+ fire()
+ else:
+ loop.call_soon_threadsafe(fire)
+
+
+def start(
+ data_dir: Path, config: Callable[[], Any], *, publish: Publish | None = None
+) -> SelfBuild:
+ """Open the ledger for this process (idempotent)."""
+ global _current
+ if _current is None:
+ _current = SelfBuild(data_dir, config, publish=publish)
+ try:
+ _current.bind_loop(asyncio.get_running_loop())
+ except RuntimeError: # started outside a loop: events stay off until bind_loop
+ pass
+ return _current
+
+
+def current() -> SelfBuild | None:
+ return _current
+
+
+def reset_for_tests() -> None:
+ global _current
+ if _current is not None:
+ _current.ledger.close()
+ _current = None
diff --git a/dodaios/self_build/undo.py b/dodaios/self_build/undo.py
new file mode 100644
index 0000000000..ce9f88c6b2
--- /dev/null
+++ b/dodaios/self_build/undo.py
@@ -0,0 +1,56 @@
+"""One undo for any build: each artefact kind registers how it is removed.
+
+A builder registers an undoer for its kind at startup. ``undo`` then works the
+same for a memory entry, a skill, a routine, an agent or a pull request: the
+undoer removes the artefact from the payload the builder stored with
+:meth:`BuildLedger.attach_artefact`, and the build is marked ``undone``.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import inspect
+import logging
+from collections.abc import Awaitable, Callable
+
+from .ledger import BuildLedger
+from .records import LIVE_STATES, OPEN_STATES, BuildRecord, Kind, State
+
+log = logging.getLogger(__name__)
+
+#: Removes the artefact a build created. Raise to report that it could not.
+Undoer = Callable[[BuildRecord], Awaitable[None] | None]
+
+
+class UndoUnavailable(RuntimeError):
+ """The build has an artefact, but nothing knows how to remove it."""
+
+
+class UndoRegistry:
+ def __init__(self) -> None:
+ self._undoers: dict[Kind, Undoer] = {}
+
+ def register(self, kind: Kind, undoer: Undoer) -> None:
+ self._undoers[Kind(kind)] = undoer
+
+ def has(self, kind: Kind) -> bool:
+ return Kind(kind) in self._undoers
+
+ async def undo(self, ledger: BuildLedger, build_id: str, note: str = "") -> BuildRecord:
+ record = await asyncio.to_thread(ledger.get, build_id)
+ if record is None:
+ raise KeyError(build_id)
+ if record.state == State.PROPOSED:
+ return await asyncio.to_thread(
+ ledger.set_state, build_id, State.REJECTED, note or "rejected"
+ )
+ if record.state not in LIVE_STATES | OPEN_STATES | {State.ARCHIVED}:
+ return record # already undone, rejected or failed: nothing exists
+ if record.artefact:
+ undoer = self._undoers.get(record.kind)
+ if undoer is None:
+ raise UndoUnavailable(record.kind.value)
+ result = undoer(record)
+ if inspect.isawaitable(result):
+ await result
+ return await asyncio.to_thread(ledger.set_state, build_id, State.UNDONE, note or "undone")
diff --git a/dodaios/ui/web/self_build_routes.py b/dodaios/ui/web/self_build_routes.py
new file mode 100644
index 0000000000..444811a5e7
--- /dev/null
+++ b/dodaios/ui/web/self_build_routes.py
@@ -0,0 +1,129 @@
+"""REST routes for the self-building ledger ("Built for you").
+
+Endpoints (all under ``/api/self-build``)::
+
+ GET "" Every build, newest change first, plus the loop's status
+ GET /digest What happened in the last ``hours`` (the daily card)
+ POST /pause The kill switch: ``{"paused": true}`` stops observing and building
+ GET /{build_id} One build with its history and what it cost
+ POST /{build_id}/accept Build a proposed build
+ POST /{build_id}/keep Keep a build on trial (or restore an archived one)
+ POST /{build_id}/undo Remove what the build created, or reject the proposal
+
+The ledger is opened at boot (``dodaios.self_build.service.start``).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+from typing import Annotated, Any
+
+from fastapi import APIRouter, HTTPException, Query, Request
+from pydantic import BaseModel
+
+from dodaios.self_build import (
+ Kind,
+ NoBuilder,
+ SelfBuild,
+ State,
+ TransitionError,
+ UndoUnavailable,
+)
+from dodaios.self_build import current as current_service
+from dodaios.self_build.cost import spent
+
+log = logging.getLogger(__name__)
+
+router = APIRouter(prefix="/api/self-build", tags=["self-build"])
+
+
+class PauseBody(BaseModel):
+ paused: bool
+
+
+class NoteBody(BaseModel):
+ note: str = ""
+
+
+def _service(request: Request) -> SelfBuild:
+ service = getattr(request.app.state, "self_build", None) or current_service()
+ if service is None:
+ raise HTTPException(status_code=503, detail="self-build ledger is not running")
+ return service
+
+
+def _parse(values: list[str] | None, enum: Any) -> list[Any] | None:
+ if not values:
+ return None
+ try:
+ return [enum(v) for v in values]
+ except ValueError as exc:
+ raise HTTPException(status_code=422, detail=str(exc)) from exc
+
+
+async def list_builds(
+ request: Request,
+ state: Annotated[list[str] | None, Query()] = None,
+ kind: Annotated[list[str] | None, Query()] = None,
+ limit: int = Query(default=200, ge=1, le=500),
+) -> dict[str, Any]:
+ service = _service(request)
+ states = _parse(state, State)
+ kinds = _parse(kind, Kind)
+ rows = await asyncio.to_thread(service.ledger.list, states=states, kinds=kinds, limit=limit)
+ status = await asyncio.to_thread(service.status)
+ return {"builds": [r.to_json() for r in rows], "status": status}
+
+
+async def digest(request: Request, hours: int = Query(default=24, ge=1, le=24 * 30)) -> Any:
+ from dodaios.self_build.ledger import now_ms
+
+ service = _service(request)
+ return await asyncio.to_thread(service.digest, now_ms() - hours * 3_600_000)
+
+
+async def pause(request: Request, body: PauseBody) -> dict[str, Any]:
+ service = _service(request)
+ await asyncio.to_thread(service.set_paused, body.paused)
+ return await asyncio.to_thread(service.status)
+
+
[email protected]("/{build_id}")
+async def get_build(request: Request, build_id: str) -> dict[str, Any]:
+ service = _service(request)
+ record = await asyncio.to_thread(service.ledger.get, build_id)
+ if record is None:
+ raise HTTPException(status_code=404, detail="no such build")
+ history = await asyncio.to_thread(service.ledger.history, build_id)
+ cost = await asyncio.to_thread(spent, record)
+ return {"build": record.to_json(), "history": history, "cost": cost}
+
+
+async def _act(coro: Any) -> dict[str, Any]:
+ try:
+ record = await coro
+ except KeyError as exc:
+ raise HTTPException(status_code=404, detail="no such build") from exc
+ except (TransitionError, NoBuilder, UndoUnavailable) as exc:
+ raise HTTPException(status_code=409, detail=f"{type(exc).__name__}: {exc}") from exc
+ return {"build": record.to_json()}
+
+
[email protected]("/{build_id}/accept")
+async def accept(request: Request, build_id: str) -> dict[str, Any]:
+ return await _act(_service(request).accept(build_id))
+
+
[email protected]("/{build_id}/keep")
+async def keep(request: Request, build_id: str) -> dict[str, Any]:
+ return await _act(_service(request).keep(build_id))
+
+
[email protected]("/{build_id}/undo")
+async def undo(request: Request, build_id: str, body: NoteBody | None = None) -> dict[str, Any]:
+ note = body.note if body is not None else ""
+ return await _act(_service(request).undo_build(build_id, note))
diff --git a/dodaios/ui/web/server.py b/dodaios/ui/web/server.py
index 9dd438d4e3..6bcd1b810c 100644
--- a/dodaios/ui/web/server.py
+++ b/dodaios/ui/web/server.py
@@ -581,6 +581,7 @@ class WebServer:
from .starter_plan_routes import router as starter_plan_router
from .sub_agents_routes import router as sub_agents_router
from .tasks_routes import router as tasks_router
+ from .self_build_routes import router as self_build_router
from .team_thread_routes import router as team_thread_router
from .telephony_routes import router as telephony_router
from .tool_model_routes import router as tool_model_router
@@ -715,6 +716,9 @@ class WebServer:
app.include_router(agentic_ide_git_router)
app.state.team_threads = None
app.include_router(team_thread_router)
+ # "Built for you": the self-building ledger (/api/self-build/*).
+ app.state.self_build = None
+ app.include_router(self_build_router)
# The Skills tab's library of saved Markdown prompts (/api/agentic-ide/skills/*).
app.include_router(ide_skills_router)
app.include_router(agentic_ide_router)
@@ -2947,6 +2951,21 @@ class WebServer:
logger.debug("wiki health.record_bootstrap(False) failed", exc_info=True)
_boot_mark("wiki_integration")
+ # The self-building loop's ledger: opens self_build.db, nothing else at
+ # boot. Observers and builders register themselves later.
+ try:
+ from dodaios.self_build.service import start as start_self_build
+
+ _self_build = start_self_build(
+ Path(self.cfg.memory.data_dir), lambda: self.cfg, publish=self.bus.publish
+ )
+ self.app.state.self_build = _self_build
+ _self_build.start_sweeper()
+ except Exception as exc: # noqa: BLE001
+ logger.opt(exception=exc).warning(
+ "self-build ledger init failed — nothing will be self-built this run"
+ )
+
# Jarvis' own self-learning loop: two bus subscriptions, nothing else
# at boot. Reviews run later in the background (dodaios/memory/learning).
try:
diff --git a/tests/unit/self_build/__init__.py b/tests/unit/self_build/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/unit/self_build/test_self_build_ledger.py b/tests/unit/self_build/test_self_build_ledger.py
new file mode 100644
index 0000000000..97322276c3
--- /dev/null
+++ b/tests/unit/self_build/test_self_build_ledger.py
@@ -0,0 +1,302 @@
+"""The build ledger, the lifecycle, the hard limits and one undo for every kind."""
+
+from __future__ import annotations
+
+import asyncio
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+from fastapi import FastAPI
+from fastapi.testclient import TestClient
+
+from dodaios.core.config import JarvisConfig, SelfBuildConfig
+from dodaios.self_build import (
+ BuildLedger,
+ DuplicateBuild,
+ Kind,
+ Outcome,
+ SelfBuild,
+ State,
+ TransitionError,
+ UndoUnavailable,
+ policy,
+)
+from dodaios.self_build.ledger import now_ms
+
+DAY = 86_400_000
+
+
+def _config(**overrides: object) -> SimpleNamespace:
+ values = {"silent_days": 0, **overrides}
+ return SimpleNamespace(self_build=SelfBuildConfig(**values))
+
+
+def ledger(tmp_path: Path) -> BuildLedger:
+ book = BuildLedger(tmp_path / "self_build.db")
+ yield book
+ book.close()
+
+
+# ------------------------------------------------------------------ ledger
+
+
+def test_open_case_records_evidence_and_history(ledger: BuildLedger) -> None:
+ record = ledger.open_case(
+ Kind.SKILL, "weekly-report", "Builds the Monday report", evidence=["turn:1", "turn:2"]
+ )
+ assert record.state == State.PROPOSED
+ assert record.evidence == ("turn:1", "turn:2")
+ assert record.caller == f"self_build:{record.id}"
+ assert [h["to"] for h in ledger.history(record.id)] == ["proposed"]
+
+
+def test_one_open_build_per_signature(ledger: BuildLedger) -> None:
+ ledger.open_case(Kind.SKILL, "sig", "first")
+ with pytest.raises(DuplicateBuild):
+ ledger.open_case(Kind.ROUTINE, "sig", "second")
+
+
+def test_closed_build_frees_its_signature(ledger: BuildLedger) -> None:
+ first = ledger.open_case(Kind.SKILL, "sig", "first")
+ ledger.set_state(first.id, State.REJECTED)
+ second = ledger.open_case(Kind.SKILL, "sig", "again")
+ assert second.id != first.id
+ assert ledger.rejections("sig") == 1
+
+
+def test_lifecycle_refuses_impossible_moves(ledger: BuildLedger) -> None:
+ record = ledger.open_case(Kind.MEMORY, "sig", "note")
+ with pytest.raises(TransitionError):
+ ledger.set_state(record.id, State.KEPT)
+ ledger.set_state(record.id, State.BUILDING)
+ trial = ledger.set_state(record.id, State.TRIAL)
+ assert trial.trial_started_ms > 0
+ ledger.set_state(record.id, State.UNDONE)
+ with pytest.raises(TransitionError):
+ ledger.set_state(record.id, State.TRIAL)
+
+
+def test_uses_and_outcomes_are_counted(ledger: BuildLedger) -> None:
+ record = ledger.open_case(Kind.SKILL, "sig", "skill", state=State.BUILDING)
+ ledger.record_use(record.id, Outcome.ACCEPTED)
+ ledger.record_use(record.id, Outcome.CORRECTED)
+ updated = ledger.record_use(record.id, Outcome.IGNORED)
+ assert updated.uses == 2
+ assert updated.outcomes == {"accepted": 1, "corrected": 1, "ignored": 1}
+
+
+def test_change_hook_sees_every_commit(tmp_path: Path) -> None:
+ seen: list[str] = []
+ book = BuildLedger(tmp_path / "db.sqlite", on_change=lambda r: seen.append(r.state.value))
+ record = book.open_case(Kind.SKILL, "sig", "skill")
+ book.set_state(record.id, State.BUILDING)
+ book.close()
+ assert seen == ["proposed", "building"]
+
+
+def test_hook_failure_never_loses_the_write(tmp_path: Path) -> None:
+ def boom(_: object) -> None:
+ raise RuntimeError("listener broke")
+
+ book = BuildLedger(tmp_path / "db.sqlite", on_change=boom)
+ record = book.open_case(Kind.SKILL, "sig", "skill")
+ assert book.get(record.id) is not None
+ book.close()
+
+
+# ------------------------------------------------------------------ policy
+
+
+def test_off_and_paused_admit_nothing(ledger: BuildLedger) -> None:
+ assert policy.admit(ledger, _config(skill="off"), Kind.SKILL, "s", undoable=True).start is None
+ disabled = _config(enabled=False)
+ assert policy.admit(ledger, disabled, Kind.MEMORY, "s", undoable=True).start is None
+ policy.set_paused(ledger, True)
+ verdict = policy.admit(ledger, _config(), Kind.MEMORY, "s", undoable=True)
+ assert (verdict.start, verdict.reason) == (None, "paused")
+
+
+def test_suggest_only_proposes(ledger: BuildLedger) -> None:
+ verdict = policy.admit(ledger, _config(skill="suggest"), Kind.SKILL, "s", undoable=True)
+ assert verdict.start == State.PROPOSED
+
+
+def test_trial_builds_only_when_undoable(ledger: BuildLedger) -> None:
+ cfg = _config(memory="trial")
+ assert policy.admit(ledger, cfg, Kind.MEMORY, "s", undoable=True).build
+ assert policy.admit(ledger, cfg, Kind.MEMORY, "s", undoable=False).start == State.PROPOSED
+
+
+def test_silent_days_only_propose(ledger: BuildLedger) -> None:
+ cfg = _config(memory="trial", silent_days=7)
+ start = now_ms()
+ assert policy.admit(ledger, cfg, Kind.MEMORY, "s", undoable=True, now=start).reason == "silent"
+ later = start + 8 * DAY
+ assert policy.admit(ledger, cfg, Kind.MEMORY, "s", undoable=True, now=later).build
+
+
+def test_rejected_need_is_not_offered_again(ledger: BuildLedger) -> None:
+ cfg = _config(max_rejections=2)
+ for _ in range(2):
+ record = ledger.open_case(Kind.SKILL, "s", "skill")
+ ledger.set_state(record.id, State.REJECTED)
+ assert policy.admit(ledger, cfg, Kind.SKILL, "s", undoable=True).reason == "rejected before"
+
+
+def test_daily_cap_and_weekly_agent_cap(ledger: BuildLedger) -> None:
+ cfg = _config(memory="trial", agent="trial", max_builds_per_day=1, max_agents_per_week=1)
+ ledger.open_case(Kind.AGENT, "a1", "agent", state=State.BUILDING)
+ assert policy.admit(ledger, cfg, Kind.MEMORY, "m", undoable=True).reason == "daily build cap"
+ cfg = _config(agent="trial", max_builds_per_day=10, max_agents_per_week=1)
+ verdict = policy.admit(ledger, cfg, Kind.AGENT, "a2", undoable=True)
+ assert verdict.reason == "weekly agent cap"
+
+
+def test_trial_settles_by_use(ledger: BuildLedger) -> None:
+ cfg = _config(trial_uses=3)
+ record = ledger.open_case(Kind.SKILL, "s", "skill", state=State.BUILDING)
+ ledger.set_state(record.id, State.TRIAL)
+ for _ in range(3):
+ record = ledger.record_use(record.id, Outcome.ACCEPTED)
+ assert policy.trial_over(record, cfg)
+ assert policy.settle(record) == State.KEPT
+
+
+def test_unused_trial_is_archived(ledger: BuildLedger) -> None:
+ cfg = _config(trial_days=14)
+ record = ledger.open_case(Kind.SKILL, "s", "skill", state=State.BUILDING)
+ record = ledger.set_state(record.id, State.TRIAL)
+ assert not policy.trial_over(record, cfg)
+ assert policy.trial_over(record, cfg, now=record.trial_started_ms + 15 * DAY)
+ assert policy.settle(record) == State.ARCHIVED
+
+
+def test_config_section_ships_with_safe_defaults() -> None:
+ cfg = JarvisConfig().self_build
+ assert cfg.enabled and cfg.silent_days == 7
+ assert (cfg.memory, cfg.skill, cfg.routine, cfg.agent, cfg.code) == (
+ "trial",
+ "suggest",
+ "suggest",
+ "suggest",
+ "suggest",
+ )
+
+
+# ------------------------------------------------------------------ service + undo
+
+
+def service(tmp_path: Path) -> SelfBuild:
+ svc = SelfBuild(tmp_path, lambda: _config(memory="trial", skill="full", trial_uses=2))
+ yield svc
+ svc.ledger.close()
+
+
+async def test_undo_rejects_a_proposal(service: SelfBuild) -> None:
+ record = service.ledger.open_case(Kind.SKILL, "s", "skill")
+ undone = await service.undo_build(record.id)
+ assert undone.state == State.REJECTED
+
+
+async def test_undo_calls_the_kind_undoer(service: SelfBuild) -> None:
+ removed: list[str] = []
+
+ async def remove(rec: object) -> None:
+ removed.append(rec.artefact["id"]) # type: ignore[attr-defined]
+
+ service.undo.register(Kind.SKILL, remove)
+ record = service.ledger.open_case(Kind.SKILL, "s", "skill", state=State.BUILDING)
+ service.ledger.attach_artefact(record.id, {"type": "skill", "id": "weekly"}, {"slug": "weekly"})
+ service.ledger.set_state(record.id, State.TRIAL)
+ undone = await service.undo_build(record.id)
+ assert undone.state == State.UNDONE
+ assert removed == ["weekly"]
+
+
+async def test_undo_without_undoer_is_refused(service: SelfBuild) -> None:
+ record = service.ledger.open_case(Kind.AGENT, "s", "agent", state=State.BUILDING)
+ service.ledger.attach_artefact(record.id, {"type": "agent", "id": "a"}, {})
+ with pytest.raises(UndoUnavailable):
+ await service.undo_build(record.id)
+ assert service.ledger.get(record.id).state == State.BUILDING # type: ignore[union-attr]
+
+
+async def test_accept_hands_the_build_to_its_builder(service: SelfBuild) -> None:
+ built = asyncio.Event()
+
+ async def builder(rec: object) -> None:
+ service.ledger.set_state(rec.id, State.TRIAL) # type: ignore[attr-defined]
+ built.set()
+
+ service.register_builder(Kind.SKILL, builder)
+ service.undo.register(Kind.SKILL, lambda rec: None)
+ record = service.ledger.open_case(Kind.SKILL, "s", "skill")
+ accepted = await service.accept(record.id)
+ assert accepted.state == State.BUILDING
+ await asyncio.wait_for(built.wait(), 2)
+ assert service.ledger.get(record.id).state == State.TRIAL # type: ignore[union-attr]
+
+
+async def test_sweep_keeps_only_at_full_and_archives_unused(service: SelfBuild) -> None:
+ used = service.ledger.open_case(Kind.SKILL, "used", "used", state=State.BUILDING)
+ service.ledger.set_state(used.id, State.TRIAL)
+ for _ in range(2):
+ service.ledger.record_use(used.id, Outcome.ACCEPTED)
+ memo = service.ledger.open_case(Kind.MEMORY, "memo", "memo", state=State.BUILDING)
+ service.ledger.set_state(memo.id, State.TRIAL)
+ for _ in range(2):
+ service.ledger.record_use(memo.id, Outcome.ACCEPTED)
+ await service.sweep()
+ assert service.ledger.get(used.id).state == State.KEPT # type: ignore[union-attr]
+ # memory runs at "trial": a good trial waits for the person's keep
+ assert service.ledger.get(memo.id).state == State.TRIAL # type: ignore[union-attr]
+
+
+async def test_changes_are_published_on_the_bus(tmp_path: Path) -> None:
+ events: list[object] = []
+
+ async def publish(event: object) -> None:
+ events.append(event)
+
+ svc = SelfBuild(tmp_path, lambda: _config(), publish=publish)
+ svc.bind_loop(asyncio.get_running_loop())
+ svc.ledger.open_case(Kind.SKILL, "s", "Weekly report")
+ for _ in range(20):
+ if events:
+ break
+ await asyncio.sleep(0.01)
+ svc.ledger.close()
+ assert type(events[0]).__name__ == "SelfBuildChanged"
+ assert events[0].title == "Weekly report" # type: ignore[attr-defined]
+
+
+# ------------------------------------------------------------------ routes
+
+
+def test_routes_list_pause_and_undo(tmp_path: Path) -> None:
+ from dodaios.ui.web.self_build_routes import router
+
+ svc = SelfBuild(tmp_path, lambda: _config())
+ app = FastAPI()
+ app.state.self_build = svc
+ app.include_router(router)
+ client = TestClient(app)
+ record = svc.ledger.open_case(Kind.SKILL, "s", "Weekly report")
+
+ listing = client.get("/api/self-build").json()
+ assert [b["id"] for b in listing["builds"]] == [record.id]
+ assert listing["status"]["paused"] is False
+
+ assert client.post("/api/self-build/pause", json={"paused": True}).json()["paused"] is True
+ detail = client.get(f"/api/self-build/{record.id}").json()
+ assert detail["build"]["title"] == "Weekly report"
+ assert detail["history"][0]["to"] == "proposed"
+
+ undone = client.post(f"/api/self-build/{record.id}/undo").json()
+ assert undone["build"]["state"] == "rejected"
+ assert client.post(f"/api/self-build/{record.id}/accept").status_code == 200
+ assert client.get("/api/self-build/nope").status_code == 404
+ svc.ledger.close()