Skip to content

Your lightweight open-source personal agent.

docs: streamline project instructions and engineering constraints

chengyongrud3d70f20655b
diff --git a/.agent/design.md b/.agent/design.md
index 1eb925ae8..e60039ead 100644
--- a/.agent/design.md
+++ b/.agent/design.md
@@ -1,6 +1,6 @@
# Design Constraints
-These rules govern architectural decisions. When adding a feature or fixing a bug, prefer paths that respect these boundaries.
+Consult this document when changing ownership boundaries, extension points, abstractions, or dynamic types. These constraints apply to implementation choices on those paths.
## Core stays small; extend at the edges
@@ -8,30 +8,18 @@ New capabilities should be added via `channels/`, `tools/`, skills, or MCP serve
Runtime state fan-out follows the same boundary. `MessageBus.publish` awaits local subscribers for turn/run/model/goal state changes; `MessageBus.publish_event` queues routed channel delivery without waiting for network sends. Both carry `AgentEvent` values. Runner hooks publish typed output through the turn's scoped `EventSink`; direct-call callbacks are adapted at the execution boundary. WebUI/WebSocket wire details, title refreshes, and goal-state sync belong in `nanobot.session.webui_turns.WebuiTurnCoordinator` or the relevant channel adapter.
-## Less structure, more intelligence
-
-Prefer simple, readable code over new framework layers and indirection. Add structure only when it removes real complexity, protects an important boundary, or matches an established local pattern. The best fix is often a smaller prompt, a tighter tool contract, a channel-local change, or one focused regression test.
-
## Prefer duplication over premature abstraction
Channels and providers are allowed to repeat similar logic (send retries, media handling, message splitting). Do not introduce complex base classes or shared helpers just to eliminate duplication across channel files. Each channel file should remain self-contained and readable on its own. The same applies to provider implementations.
-## Minimal change that solves the real problem
-
-Fix bugs by changing only what is necessary. Do not bundle unrelated refactors or clean-ups into a feature or bugfix PR. If a refactor is genuinely required, it should be a separate, clearly scoped PR.
-
-## Keep PRs reviewable
-
-A bugfix should make the protected invariant clear, change the smallest surface that enforces it, and add only the closest regression test. If a diff starts changing ownership boundaries or mixing behavior changes with clean-up, split it before it becomes hard to review.
-
## Type dynamic boundaries at the edge
Wire payloads, persisted records, and third-party SDK objects are untrusted dynamic boundaries. Prefer a parser or small normalizer at the owning edge, and use `TypedDict` for stable dictionary shapes, so validation happens once and internal code receives a concrete type. Do not spread raw dynamic dictionaries or SDK objects through the core.
Stable first-party dependencies must be typed where they are stored or passed. Do not declare an internal service, context field, or callback result as `Any` and then recover its real type with consumer-side casts. Use the concrete type or a narrow `Protocol`; reserve `Any` for genuinely dynamic boundaries.
-`typing.cast` performs no runtime validation. Every new cast must be supported by a runtime check on the same path or by an explicit invariant that is clear from construction and control flow (and documented locally when it is not obvious). If input can violate the claimed type, handle that invalid case before casting; never use `cast` only to silence BasedPyright.
+Every new cast must be supported by a boundary check or an invariant established by construction and control flow. Do not add redundant checks inside a validated path or use `cast` only to silence BasedPyright.
## Explicit over magical
-Configuration must be declared explicitly in `config/schema.py` Pydantic models. Error handling should raise clear exceptions rather than silently correcting bad input. Provider auto-detection exists, but every resolution path must be traceable from the factory to the concrete provider class.
+Configuration belongs in `config/schema.py` Pydantic models. Provider auto-detection must remain traceable from the factory to the concrete provider class.
diff --git a/.agent/gotchas.md b/.agent/gotchas.md
index 588d957a0..b024814b0 100644
--- a/.agent/gotchas.md
+++ b/.agent/gotchas.md
@@ -1,31 +1,33 @@
# Common Gotchas
-## Do not use `ruff format`
+## Channel dependencies for strict type checking
-`CONTRIBUTING.md` mentions `ruff format`, but **do not run it** — it destroys git blame history. Only `ruff check` should be used.
+Channel dependencies come from channel package manifests, outside the main extras. Reproduce CI's setup with `uv sync --all-extras --dev`, then `uv run --no-sync python -m scripts.install_channel_dependencies --all-channels`. Keep `--no-sync` on subsequent `uv run` commands so syncing does not remove those dependencies. See `.github/workflows/ci.yml` for checks.
-## Config `${VAR}` References
+## Channel-owned UI and WebUI transport
+
+Channels can own frontend code in `nanobot/channels/*/webui/` and tests in `nanobot/channels/*/tests/`, including `tests/webui/`. The WebUI's lint and test configurations include those frontend paths. Do not assume all channel tests live under the root `tests/` or all frontend code lives under `webui/`.
+
+Vite proxies `/api`, `/webui`, and `/auth`; the application's WebSocket connects directly to the gateway. `NANOBOT_API_URL` sets the HTTP proxy target, whose default is `http://127.0.0.1:8765`. See `webui/vite.config.ts`.
-`config/loader.py` resolves `${VAR}` patterns in `config.json` at load time. This is **not** a shell-like default-value syntax. If the environment variable is missing, `load_config` raises `ValueError` and the agent falls back to default configuration.
+## Config `${VAR}` References
-Example valid usage:
-```json
-{ "providers": { "openrouter": { "apiKey": "${OPENROUTER_KEY}" } } }
-```
+`nanobot/config/loader.py` loads and validates the config; `resolve_config_env_vars` resolves `${VAR}` references before runtime use. This is not shell-like default-value syntax. Missing referenced variables raise `ConfigLoadError` with field locations; invalid existing config files also raise `ConfigLoadError`. Defaults are created when the config file is absent, not as recovery from an invalid file.
## Windows Compatibility
nanobot explicitly supports Windows. Key differences to keep in mind:
- `ExecTool` defaults to PowerShell on Windows (`pwsh` when available, otherwise Windows PowerShell); pass `shell="cmd"` for cmd.exe syntax or cmd built-ins (`shell.py`).
-- `cli/commands.py` forces `sys.stdout`/`stderr` to UTF-8 on startup to handle emoji and multilingual input.
+- `nanobot/cli/entry.py` configures Windows console output as UTF-8 before dispatch to handle emoji and multilingual input.
- MCP stdio server commands are normalized for Windows path separators (`mcp.py`).
-- Always use `pathlib.Path` for path manipulation; do not assume `/` separators.
## Prompt Templates
-Agent system prompts and scenario-specific instructions live in `nanobot/templates/` as Jinja2 markdown files (`identity.md`, `platform_policy.md`, `HEARTBEAT.md`, `SOUL.md`, etc.). Changing these files alters agent behavior as directly as changing Python code. They are loaded by `utils/prompt_templates.py`.
+Agent system prompts and scenario-specific instructions live in `nanobot/templates/`, including `agent/identity.md` and `agent/platform_policy.md`; workspace defaults include `AGENTS.md`, `HEARTBEAT.md`, and `SOUL.md`. Prompt rendering is handled by `nanobot/utils/prompt_templates.py`. Changing these files alters agent behavior as directly as changing Python code.
+
+The repository's root `AGENTS.md` guides coding agents; `nanobot/templates/AGENTS.md` is a default for nanobot workspaces. `nanobot/agent/context.py` loads project `AGENTS.md` and global `SOUL.md`/`USER.md`, skipping unchanged bundled `AGENTS.md` and `USER.md` defaults. It does not implement Codex's ancestor-directory instruction chain.
-Tool descriptions, skills, and replayed session history also shape model behavior. Treat changes to those surfaces like runtime code: keep them narrow, add a focused regression test when possible, and avoid teaching the model to repeat internal markers, local paths, or tool-call text.
+Tool descriptions, skills, and replayed session history also shape model behavior. Treat changes to those surfaces like runtime code: keep them focused, verify the affected contract, and avoid teaching the model to repeat internal markers, local paths, or tool-call text. Add a regression test when it proves a reachable failure that existing coverage does not address.
## Context Pollution Persists
@@ -35,6 +37,8 @@ Anything written into memory, session history, or prompt inputs can be replayed
Built-in skills live in `nanobot/skills/` (markdown + YAML frontmatter format). Agent capabilities that are "know-how" rather than code should be added as skills, not hardcoded into the agent loop. External skills can be published to and installed from ClawHub.
-## Atomic Session Writes
+## Persistence and durability
+
+`nanobot/agent/memory.py` owns the memory journal `memory/history.jsonl`; ordinary entries are appended under a lock, without explicit fsync. Full journal rewrites use `atomic_write_lines` with its default file and directory fsync. Preserve that atomic replacement path rather than truncating the live journal in place.
-`agent/memory.py` writes `history.jsonl` atomically (temp file + fsync + rename + directory fsync). This guarantees durability across crashes. Do not replace this with a plain `open(..., "w")` write.
+Conversation history belongs to `nanobot/session/manager.py`. Session saves use atomic replacement, with fsync controlled by the caller; the default is `False`, while durable shutdown and other explicit durability paths request `True`. Preserve the distinction between atomic visibility and crash durability, and retain locks and explicit fsync on paths that require them.
diff --git a/.agent/security.md b/.agent/security.md
index faedd9e54..51139cf9b 100644
--- a/.agent/security.md
+++ b/.agent/security.md
@@ -1,6 +1,6 @@
# Security Boundaries
-The agent operates with significant power (file system, shell, web). The following guards must not be bypassed when modifying related code.
+Consult this document when changing path access, outbound HTTP/MCP, or shell execution. The agent operates with filesystem, shell, and network access; preserve the guards below on every affected path.
## Workspace Restriction
@@ -26,4 +26,4 @@ HTTP/SSE MCP transports are part of this boundary: validate configured MCP URLs
`tools/sandbox.py` provides optional command wrapping: `bwrap` (bubblewrap) on Linux and `seatbelt` (`sandbox-exec`) on macOS. Seatbelt must not expose shared host temporary directories; scratch stays in the workspace. On Windows a configured backend warns and leaves only the application guard. On Unix a configured backend that cannot start must fail, not silently execute without isolation.
-**Rule**: If adding a new sandbox backend, implement `_wrap_<name>(command, workspace, cwd) -> str` and register it in `_BACKENDS`.
+**Rule**: If adding a sandbox backend, match the backend callable contract in `nanobot/agent/tools/sandbox.py` and register it in `_BACKENDS`. Preserve read/write root handling and the configured backend's failure behavior.
diff --git a/.agent/simplify.md b/.agent/simplify.md
new file mode 100644
index 000000000..4568cec28
--- /dev/null
+++ b/.agent/simplify.md
@@ -0,0 +1,26 @@
+# Simplify with judgment
+
+Use this guidance when cleaning up code, choosing between a patch and a refactor, or deciding whether a test earns its maintenance cost.
+
+## Think before extending the code
+
+Work out the required behavior, the facts available at the boundary, who owns each piece of state, and what actually causes the failure. Then choose the simplest coherent implementation. Do not begin by asking where another condition can be inserted into the current code.
+
+Use your judgment. You are expected to understand the problem, not merely keep the current code limping along. If a solution needs another fallback, ask which broken contract it hides. If a helper needs more modes, ask whether it owns too many responsibilities. If the implementation is hard to explain, reconsider the design before adding another layer.
+
+## Refactor when the structure is the problem
+
+Refactor the owning path when it removes the cause: duplicated state, contradictory ownership, tangled control flow, or an abstraction that no longer fits its callers. Keep that refactor tied to the requested behavior and preserve unaffected contracts. A small diff is not a success if it leaves the defect embedded in the design.
+
+Remove redundant checks inside trusted, validated paths. Validate untrusted input at its boundary and let internal code use that contract. Keep fallbacks only for supported, reachable failure modes with a defined result. Do not silently recover from broken internal invariants, retain speculative compatibility paths, or introduce frameworks for hypothetical future requirements. Preserve deliberate security checks and documented recovery contracts.
+
+## Tests must earn their place
+
+Prefer evidence for the supported user path and the important reachable regression. Reuse existing coverage when it already proves the contract.
+
+- Do not add defensive cases for impossible internal states, unsupported combinations, or hypothetical failures without a reachable path and a material consequence.
+- Avoid exhaustive mock permutations, assertions that mirror private implementation steps, and tests that merely prove old text or symbols disappeared.
+- A prose edit, mechanical cleanup, or low-impact reversible change does not automatically need new tests. Use focused existing checks when sufficient.
+- Test invalid input where a real untrusted boundary accepts it, security properties where a real capability exposes them, and recovery where the product promises it. These are supported contracts, not an invitation to test every imagined failure.
+
+Choose each test because it could detect a meaningful contract failure. Once the relevant evidence is sufficient, stop adding cases.
diff --git a/.agent/workflow.md b/.agent/workflow.md
new file mode 100644
index 000000000..e88125140
--- /dev/null
+++ b/.agent/workflow.md
@@ -0,0 +1,5 @@
+# Reusing verification evidence
+
+After a content or base change, rerun checks whose supporting assumptions changed or whose relevance is uncertain. Unrelated changes do not require repeating all verification or review. Carry evidence forward with the intervening diff, the contract it proved, and the reason that contract is unaffected. Consider shared dependencies and configuration, not just overlapping files.
+
+Evaluate PR review and required CI against the current remote head. Inherited evidence retains its original identity and an explanation of its applicability; it is not a fresh execution. Keep raw verification artifacts outside tracked source files.
diff --git a/AGENTS.md b/AGENTS.md
index 8217f0ad5..8f098bd67 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1,86 +1,20 @@
-This file provides guidance to AI coding agents working with this repository.
+# Working on nanobot
-## Project Overview
+The Python gateway owns agent execution, sessions, tools, memory, and security policy. WebUI and TUI share that runtime; keep execution and policy out of the clients.
-nanobot is a lightweight, open-source AI agent framework written in Python with a React/TypeScript WebUI. It centers around a small agent loop that receives messages from chat channels, invokes an LLM provider, executes tools, and manages session memory.
+## Task-specific guidance
-## Development Commands
+| When working on | Read |
+| --- | --- |
+| Core boundaries, extensions, or internal types | [`.agent/design.md`](.agent/design.md) |
+| Refactoring, fallbacks, or test selection | [`.agent/simplify.md`](.agent/simplify.md) |
+| Path permissions, HTTP/MCP, or shell isolation | [`.agent/security.md`](.agent/security.md) |
+| Dependency setup, WebUI transport, config, Windows, prompts, or persistence | [`.agent/gotchas.md`](.agent/gotchas.md) |
+| Reusing verification evidence | [`.agent/workflow.md`](.agent/workflow.md) |
+| Contribution or publication | [`CONTRIBUTING.md`](CONTRIBUTING.md), [`docs/releasing.md`](docs/releasing.md) |
-```bash
-# Python: run single test / lint
-pytest tests/test_openai_api.py::test_function -v
-ruff check nanobot/
+## Development constraints
-# Strict type checking (matches CI)
-uv sync --all-extras --dev
-uv run --no-sync python -m scripts.install_channel_dependencies --all-channels
-uv run --no-sync basedpyright
-
-# WebUI: dev server (proxies API/WS to gateway :8765), build, test
-# Build outputs to ../nanobot/web/dist (bundled into the Python wheel)
-cd webui && bun run dev # or NANOBOT_API_URL=... bun run dev
-cd webui && bun run build
-cd webui && bun run test
-
-# Gateway
-nanobot gateway
-```
-
-## High-Level Architecture
-
-### Core Data Flow
-
-Messages flow through an async `MessageBus` (`nanobot/bus/queue.py`) that decouples chat channels from the agent core:
-
-1. **Channels** (`nanobot/channels/`) receive messages from external platforms and publish `InboundMessage` events to the bus.
-2. **`AgentLoop`** (`nanobot/agent/loop.py`) consumes inbound messages, builds context, and coordinates the turn.
-3. **`AgentRunner`** (`nanobot/agent/runner.py`) handles the actual LLM conversation loop: send messages to the provider, receive tool calls, execute tools, and stream responses.
-4. Responses are published as `OutboundMessage` events back to the appropriate channel.
-
-### Key Subsystems
-
-- **Agent Loop** (`nanobot/agent/loop.py`, `runner.py`): The core processing engine. `AgentLoop` manages session keys, hooks, and context building. `AgentRunner` executes the multi-turn LLM conversation with tool execution.
-- **LLM Providers** (`nanobot/providers/`): Provider implementations (Anthropic, OpenAI-compatible, OpenAI Responses API, Azure, Bedrock, GitHub Copilot, OpenAI Codex, etc.) built on a common base (`base.py`). Includes image generation (`image_generation.py`) and audio transcription (`transcription.py`). `factory.py` and `registry.py` handle instantiation and model discovery.
-- **Channels** (`nanobot/channels/`): Platform integrations (Telegram, Discord, Slack, Feishu, Matrix, WhatsApp, QQ, WeChat, WeCom, DingTalk, Email, MoChat, MS Teams, WebSocket, Mattermost). `manager.py` discovers and coordinates them. Channels are self-contained packages auto-discovered via `pkgutil` scanning.
-- **Tools** (`nanobot/agent/tools/`): Agent capabilities exposed to the LLM: filesystem (read/write/edit/list), shell execution (with sandbox backends), web search/fetch, MCP servers, cron, notebook editing, subagent spawning, long-running tasks / sustained goals (`long_task.py`), image generation, and self-modification. Tools are auto-discovered via `pkgutil` scan + entry-point plugins.
-- **Memory** (`nanobot/agent/memory.py`): Session history persistence with Dream two-phase memory consolidation. Uses atomic writes with fsync for durability.
-- **Session Management** (`nanobot/session/`): Per-session history, context compaction, TTL-based auto-compaction (`manager.py`), and sustained goal state tracking (`goal_state.py`).
-- **Config** (`nanobot/config/schema.py`, `loader.py`): Pydantic-based configuration loaded from `~/.nanobot/config.json`. Supports camelCase aliases for JSON compatibility.
-- **WebUI** (`webui/`): Vite-based React SPA that talks to the gateway over a WebSocket multiplex protocol. The dev server proxies `/api`, `/webui`, `/auth`, and WebSocket traffic to the gateway.
-- **API Server** (`nanobot/api/server.py`): OpenAI-compatible HTTP API (`/v1/chat/completions`, `/v1/models`) for programmatic access.
-- **Command Router** (`nanobot/command/`): Slash command routing and built-in command handlers.
-- **Heartbeat** (`nanobot/templates/HEARTBEAT.md`): Periodic task list checked via `cron` jobs (legacy dedicated service removed).
-- **Pairing** (`nanobot/pairing/`): DM sender approval store with persistent pairing codes per channel.
-- **Skills** (`nanobot/skills/`): Built-in skill definitions (cron, github, image-generation, etc.) loaded into agent context.
-- **Security** (`nanobot/security/`): PTH file guard and other security measures activated at CLI entry.
-
-### Entry Points
-
-- **CLI**: `nanobot/cli/commands.py`
-- **Python SDK**: `nanobot/nanobot.py`
-
-## Project-Specific Notes
-
-- Architecture constraints: [`.agent/design.md`](.agent/design.md)
-- Security boundaries: [`.agent/security.md`](.agent/security.md)
-- Common gotchas: [`.agent/gotchas.md`](.agent/gotchas.md)
-
-## Contribution Flow
-
-See [`CONTRIBUTING.md`](./CONTRIBUTING.md) for contribution flow and PR guidelines.
-
-## Code Style
-
-- Python 3.11+, asyncio throughout.
-- Line length: 100.
-- Linting: `ruff` with rules E, F, I, N, W (E501 ignored).
-- pytest with `asyncio_mode = "auto"`.
-
-## Common File Locations
-
-- Config schema: `nanobot/config/schema.py`
-- Provider base / new provider template: `nanobot/providers/base.py`
-- Channel base / new channel template: `nanobot/channels/base.py`
-- Tool registry: `nanobot/agent/tools/registry.py`
-- WebUI dev proxy config: `webui/vite.config.ts`
-- Tests mirror the `nanobot/` package structure.
+- Analyze the required behavior, state ownership, and root cause before extending existing code. Refactor when the structure causes the problem; a smaller diff does not justify another fallback.
+- Do not add defensive tests for hypothetical internal states or unsupported combinations. Each new test needs a reachable path and a meaningful contract to protect.
+- Do not run `ruff format`; mechanical formatting obscures git blame and creates unrelated diffs. This constraint takes precedence over the optional touched-file formatting in `CONTRIBUTING.md`.