Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs(employees): builder v3, talk sending and upgrades
trohitgfef8607faaaa
diff --git a/CLAUDE.md b/CLAUDE.mdindex d272f0c5..9e17ea91 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -13,7 +13,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire| **[UI Migration Plan](./docs-internal/ARCHIVE/ui_migration_plan.md)** | *Archived.* antd → shadcn/ui migration completion log (Waves 1–10). Current state lives in [Frontend Architecture](./docs-internal/frontend_architecture.md) and [Theme System](./docs-internal/theme_system.md). || **[Temporal Cleanup & Resilience Plan (Waves 15-18)](./docs-internal/ARCHIVE/TEMPORAL_CLEANUP_AND_RESILIENCE_PLAN.md)** | *Archived — all four waves shipped.* 15 dead-code retirement (Redis-Streams branch in event_waiter, APScheduler cron stack, orphaned `connection_status` factory), 16 per-queue task routing (`TemporalWorkerPool`), 17 resilience hardening (cron `catchup_window`, SDK interceptors, `DEPLOYMENT_MODE`, periodic heartbeats; the one-shot LLM-step retry it records was later reverted to unlimited-with-backoff), 18 worker performance tuning. Current settings live in [Temporal Architecture](./docs-internal/TEMPORAL_ARCHITECTURE.md). || **[Node Allowlist](./docs-internal/node_allowlist.md)** | Single-config UI visibility — `server/config/node_allowlist.json` controls which nodes / credential categories / skill folders show in the UI. Five lists with two enforcement tiers (mode-gated allowlist + absolute blocklist). `useNodeAllowlist` hook exposes `isVisible` / `isBlocked` / `isAllowed` / `isCredentialCategoryDisabled` / `isSkillFolderDisabled`. Adding a new disabled domain = single JSON edit, no code change. |-| **[Normal Mode (Home)](./docs-internal/normal_mode.md)** | The owner-facing landing screen: describe a job, an LLM on the server drafts a setup screen (a JSON UI spec the client parses, normalises and renders from a fixed catalogue), and Hire builds, validates, saves and starts one workflow per AI employee (`services/employees/`, graph built from `config/employee_apps.json` plus the owner's skill library and a Canvas node, and checked against `node_allowlist.json` `enabled_nodes`). Covers the screen switch (`app/`, `shellMode` / `ui_shell_mode`, `VITE_NORMAL_MODE=false` kill switch, Home shows light/dark only), the employee page (the shared chat, `features/chat`, with the employee named in the header beside New conversation; Pause is in the Workspace header), deleting an employee from the team sidebar (stopped through Reset first, then gone from every tab), the live Ask first rule (`workflow_rules`, the chat's Ask first chip via `set_ask_first`; app replies wait in `approvalGate`, sending tool calls are held as drafts by `services/approvals/tool_calls.py` and sent after a 5 s Undo window by `ApprovedToolCallWorkflow`; cards in the chat), run records for "done today" (`machina-run-record-v1`), automatic-pause reasons, Settings (Profile fields; usage-only Billing via `get_employee_usage`; Skills, the user-skills library each new hire copies onto a Skills node; consumer Connectors with `publisher` / `verified`; Plugins, the starter bundles in `hire/starters.json`; the last three share one catalog page), the Workspace dock (an employee's Canvas board, named by the summary's `canvas_node_id`; a live Browser tab with user takeover, shared with Dev mode; Mobile shows the managed local Android phone), the "Web browser" app (read-only under "ask me first" through the tool's `ask_first_params`; while the agent waits in `request_user` the summary's `browser_request` shows Needs you and Help in browser), Talk on the employee page (a thread over the chat session whose id is the workflow id, rows written through `services/chat_thread.py` and, for the owner's messages and their chat runs, `services/chat/ledger.py` (see [Chat Protocol](./docs-internal/chat_protocol.md)); the talk line is chatTrigger → an agent → `chatReply` "Reply in Chat": a chat hire's own agent is its talk agent, other hires get a "Talk with {name}" agent sharing the worker's tools and skills with its own Context, and schedule hires also "Post to Talk"), Turn on Talk (for talk `off`) and Apply (while the summary's `pending_changes` is true) via `enable_employee_talk` / `apply_employee_changes`, each a restart on the saved graph through `services/deployment/restart.py`, self-extension from Talk through the Agent Builder within the Hire rule (`services/employees/policy.py`), the 3D orb, the wire contract and the `employee_lifecycle` / `approval_lifecycle` / `chat.updated` broadcasts. |+| **[Normal Mode (Home)](./docs-internal/normal_mode.md)** | The owner-facing landing screen: describe a job, an LLM on the server drafts a setup screen (a JSON UI spec the client parses, normalises and renders from a fixed catalogue), and Hire builds, validates, saves and starts one workflow per AI employee (`services/employees/`, graph built from `config/employee_apps.json` plus the owner's skill library and a Canvas node, and checked against `node_allowlist.json` `enabled_nodes`). Covers the screen switch (`app/`, `shellMode` / `ui_shell_mode`, `VITE_NORMAL_MODE=false` kill switch, Home shows light/dark only), the employee page (the shared chat, `features/chat`, with the employee named in the header beside New conversation; Pause is in the Workspace header), deleting an employee from the team sidebar (stopped through Reset first, then gone from every tab), the live Ask first rule (`workflow_rules`, the chat's Ask first chip via `set_ask_first`; app replies wait in `approvalGate`, sending tool calls are held as drafts by `services/approvals/tool_calls.py` and sent after a 5 s Undo window by `ApprovedToolCallWorkflow`; cards in the chat; builder v3 gates every app reply, attaches every app tool and gives the talk agent `chatUi` and the registry's `talk_send` tools, with instructions identical whatever the rule; older employees are upgraded on Apply / Turn on Talk / Start by `services/employees/upgrade.py`), run records for "done today" (`machina-run-record-v1`), automatic-pause reasons, Settings (Profile fields; usage-only Billing via `get_employee_usage`; Skills, the user-skills library each new hire copies onto a Skills node; consumer Connectors with `publisher` / `verified`; Plugins, the starter bundles in `hire/starters.json`; the last three share one catalog page), the Workspace dock (an employee's Canvas board, named by the summary's `canvas_node_id`; a live Browser tab with user takeover, shared with Dev mode; Mobile shows the managed local Android phone), the "Web browser" app (read-only under "ask me first" through the tool's `ask_first_params`; while the agent waits in `request_user` the summary's `browser_request` shows Needs you and Help in browser), Talk on the employee page (a thread over the chat session whose id is the workflow id, rows written through `services/chat_thread.py` and, for the owner's messages and their chat runs, `services/chat/ledger.py` (see [Chat Protocol](./docs-internal/chat_protocol.md)); the talk line is chatTrigger → an agent → `chatReply` "Reply in Chat": a chat hire's own agent is its talk agent, other hires get a "Talk with {name}" agent sharing the worker's tools and skills with its own Context, and schedule hires also "Post to Talk"), Turn on Talk (for talk `off`) and Apply (while the summary's `pending_changes` is true) via `enable_employee_talk` / `apply_employee_changes`, each a restart on the saved graph through `services/deployment/restart.py`, self-extension from Talk through the Agent Builder within the Hire rule (`services/employees/policy.py`), the 3D orb, the wire contract and the `employee_lifecycle` / `approval_lifecycle` / `chat.updated` broadcasts. || **[Chat Protocol](./docs-internal/chat_protocol.md)** | The wire contract between the chat runtime (`server/services/chat/`) and the shared chat UI: AG-UI's event model on the existing WebSocket, snake_case, in the CloudEvents envelope. A message a deployed chat trigger will answer starts a **chat run** admitted with it in one reserved write transaction (`Database.reserved_session`, SQLite `BEGIN IMMEDIATE`): one live run per session (the lane, also a partial unique index; a second send answers `run_in_progress`), idempotent `client_message_id`, claimed and finished by the MachinaWorkflow that answers it behind `machina-chat-run-v1` (the run id is trusted only from an event whose source is `opencompany://services/chat`; `run_scope` rides every node context; only the claimant finishes), the reply saved as `a_<run id>` by `chatReply`, every message appended after the session's active leaf (`chat_threads`) so concurrent writes never fork the thread. `chat_run_event` frames reach only sockets that passed `chat_subscribe`'s owner check (`services/chat/hub.py`: per-run `seq`, `hub_epoch`, bounded per-socket queues with a resync), never the status broadcast. A watchdog (`services/chat/watchdog.py`, started by `main.py`) ends runs nothing will finish (`not_delivered`, `timed_out`, `interrupted`). The agent wired into Reply in Chat (`answers_chat_run`) streams its answer as `text.*` events (`services/chat/stream.py`, activity input only, no workflow patch), its tool calls show as steps (`services/chat/steps.py`; label from the plugin's `chat_step`), and `stop_chat_run` stops a run cooperatively: its LLM step polls the run and returns what was shown, unstarted tools are skipped, the reply is saved `stopped`, and the watchdog cancels a run still stopping after `runs.stop_grace_s`. Generated UI: the `show_ui` tool (`chatUi`) is checked against `config/chat_genui_catalog.json` (`services/genui/spec.py`; refused with every reason so the model retries), saved as a run part, streamed as `activity.*` RFC 6902 patches and sealed into the reply (`services/chat/parts.py`); `chat_ui_state` keeps what the owner set, and a button press comes back as `send_chat_message{ui_event}` -> a `[ui-event]` turn. The answering agent's system prompt ends with a fixed guide (`services/chat/guide.py`), what the owner set reaches its next turn as a `[ui-state]` note (`chat_notes`, `services/chat/notes.py`: claimed by the payload, told when the run answered), a trailing `<followups>` block becomes `parts.followups`, and a `chat_sources` tool's results are numbered per conversation for `[n]` citations (`services/chat/sources.py`). Approvals (`services/approvals/`, its own section): a plugin's `approval` spec marks a tool call that sends; while the workflow asks first it is held as a `tool_call` draft, shown as a card on the reply that made it, sent once after the Undo window by `ApprovedToolCallWorkflow`, with the outcome told to the employee as an `[update]` note. Lists every event, handler, part and error code, including the ones later phases build. || **[Theme System](./docs-internal/theme_system.md)** | 12-way visual theme system — 2 base (light, dark) + 5 utopian (renaissance, greek, edo, steampunk, atomic) + 5 dystopian (cyber, wasteland, rot, plague, surveillance) — driven by `<html data-theme>` + per-theme CSS files in `client/src/themes/`. Token taxonomy (surface / fg / border / accent / typography / motion) in **hex + `color-mix()`**; `@theme inline` bridge maps `--color-X: var(--X)` (no `hsl()` wrapper); per-theme files own the colour hex, base.css owns the shared `--tint-*` alpha scale; action role tokens (incl. `-ink` readable text), **per-theme `--code-*` syntax tokens** (tier 6 — the code editor, console/output JSON viewers, and chat code blocks paint in each theme's own palette: keyword→trigger, string→success, number→agent, function→model, on an adaptive dark/light code surface; replaced the global dracula `--prism-*` block + dead `getPrismTokenCSS`; the `OutputPanel` JSON viewer reads the same vars), per-theme `--pulse-keyframe` animation system + `.opencompany-*` helpers in `animations.css`, decorative-layer wrappers (`.app-frame` / `.canvas-host` / `.modal-frame`), per-component decorative ornaments (panel textures + canvas decorations + node pseudo-element overlays + theme-specific keyframes), **canvas-node visual contract via `--node-color` CSS custom property** (no inline `background` / `border` on node components — base.css + per-theme CSS owns visuals; `NodeStyle` helper type at [types/NodeTypes.ts](./client/src/types/NodeTypes.ts) makes the inline custom-prop typecheck-clean), **`--node-pulse-color` separate from `--node-color`** so executing-node glow uses each theme's highest-contrast accent regardless of plugin accent (Cyber neon cyan, Surveillance REC red, Renaissance ultramarine, etc.), **`data-page-hidden` animation pause** (set by `app/usePageActivitySync.ts`, mounted in `AppShell` on both screens, while the tab is hidden or the window is blurred; base.css declares `html[data-page-hidden] *, *::before, *::after { animation-play-state: paused !important }` to prevent compositor stall on tab return), **per-theme icon glyph system** (290 inline SVG glyph strings across 29 keys × 10 themes via [themedGlyphs.ts](./client/src/assets/icons/themedGlyphs.ts) + theme-aware [NodeIcon.tsx](./client/src/assets/icons/NodeIcon.tsx)), **per-theme canvas-grid + custom cursors** via `--canvas-grid` / `--cursor-default` slots, **decorative HTML primitives** (`<SvgFilterDefs>` mounting `#ink-blot` / `#noise` / `#crt` filter IDs at app root, `<DropCap>` wrapper for Renaissance ornament rule), **parameter panel migrated to Tailwind tokens** (no `useAppTheme()` reads; section headers carry the display-typography triplet; raw `<Button>` swapped to `<ActionButton intent>`), **per-theme scrollbar webkit rules** in all 12 themes, 9-event WebAudio sound system (10 packs via `--sound-pack` token + `useSound()` hook + global hover delegate + sonner toast monkey-patch + `withSound()` HOC + `Sounds.unlock()` gesture-unlock for AudioContext autoplay-policy compliance), `@media (prefers-reduced-motion: reduce)` accessibility, 30 ms throttle on `type` / `hover`, migration recipe, anti-patterns. Read this before adding a new theme, migrating a component to the new contract, or adding a canvas-node component. || **[Design System Bundle](./docs-internal/design-system/IMPLEMENTATION.md)** | Canonical, vendored design-system reference (extracted from this repo; see its `IMPLEMENTATION.md` for the contract). Tokens are **hex + `color-mix()`** (`tokens/{colors,typography,spacing,motion,fonts,base}.css`) — the format the live codebase standardizes on. Includes 34 reference components (`components/`, inline-style reference only — recreate in Tailwind/shadcn idioms), full-app UI kit (`ui_kits/opencompany/index.html`), 12-theme docs (`guidelines/THEMES.md`), **the merged handoff brief (`HANDOFF.md` — glow-ladder traps, step-edge contract, toolbar layout traps, canvas-layer asymmetries, plus the recorded product amendments)**, and **the panel×theme fidelity target (`reference-mockup/Panel Theme Matrix.dc.html` — 17 panels × 12 themes, open in a browser)**. `reference/themes/` no longer vendors CSS snapshots — it points at the authoritative [client/src/themes/](./client/src/themes/). Copy token values verbatim; do not re-derive by eye. Pairs with [Theme System](./docs-internal/theme_system.md). |diff --git a/docs-internal/normal_mode.md b/docs-internal/normal_mode.mdindex e35e72f1..1413860c 100644--- a/docs-internal/normal_mode.md+++ b/docs-internal/normal_mode.md@@ -322,12 +322,19 @@ the client's `HIRE_PAYLOAD_KEYS` must match, locked byon one Skills node (`masterSkill`) with each skill's text copied in, so alater change to the library never alters an employee already hired. Askill named `skill`, or one ending in `-personality`, is left out: it would- take over the Skill tool, or replace the whole system message. With "Ask me before- sending anything" on, a reply goes through `approvalGate`, and tools that- send or spend money are left off, except one that declares- `ask_first_params`: the "Web browser" app's browser stays on with- `interaction: read_only`, so it can read pages and hands any change to the- owner through `request_user`. The Web browser app has nothing to connect+ take over the Skill tool, or replace the whole system message. The rule is+ live (builder version 3, `LIVE_RULE_BUILDER_VERSION`): every app reply goes+ through `approvalGate`, which reads it each time; the apps' tools are all+ attached, and while the owner asks first a call that sends is held for+ them, Stripe is refused and the browser reads only, per call. The talk+ agent alone also gets its talk tools: generated UI in the chat (`chatUi`,+ "Show in chat") and a way to send through each of the hire's apps (the+ registry's `talk_send`: WhatsApp, WhatsApp Business, Telegram, Discord,+ Gmail, Outlook, email), so the owner can ask it to send something; never on+ a worker strangers write to. The instructions read the same with the rule+ on or off (a test holds them byte for byte). An older tool that declares+ `ask_first_params` and no approval spec would still be attached in that+ form. The Web browser app has nothing to connect(its catalogue entry's `connected_check` is `builtin`); its Connectorspanel manages optional login profiles. Every node type must pass`node_allowlist.is_hire_allowed` ([Node Allowlist](./node_allowlist.md));@@ -336,9 +343,8 @@ the client's `HIRE_PAYLOAD_KEYS` must match, locked by[persist_new_workflow](../server/services/workflow_storage/persist.py)(shared with workflow import);5. records on the row the apps the graph actually uses (`built.app_ids`), so- an app the hire named but left out (a tool that sends while they ask- first) never shows as one to connect; summaries read apps off the graph- the same way;+ an app the hire named but left out never shows as one to connect;+ summaries read apps off the graph the same way;6. answers `{employee, started, missing_apps, needs_ai, unsupported_apps, warnings, idempotent}`and, when every app is connected and an AI model exists, starts theemployee in the background. Errors: `invalid_request`, `too_large`,@@ -606,11 +612,12 @@ asks, within the rule Hire applies([policy.py](../server/services/employees/policy.py)):- **Tools**: the ones every hire gets (web search, checklist, clock, Memory,- Canvas) and the app registry's. While "Ask me before sending anything" is- on, nothing that sends or spends, except a tool with `ask_first_params` in- that form (the browser, read-only). Every type must pass `is_hire_allowed`,- and an app's tool needs the app connected. A refusal is one plain sentence- the agent passes on.+ Canvas) and the app registry's. A tool that sends or spends comes whole:+ while "Ask me before sending anything" is on, each call waits for the owner+ (or is refused, or reads only), per call. Only one that cannot wait (no+ approval spec) stays off, or comes in its `ask_first_params` form. Every+ type must pass `is_hire_allowed`, and an app's tool needs the app+ connected. A refusal is one plain sentence the agent passes on.- **Skills**: from the owner's library (Settings > Skills, on or off for newhires) and the Discover folder (`server/skills/employee/`), with their textcopied in; never `skill` or a `*-personality` skill.@@ -650,14 +657,16 @@ unless it was approved, and the recipient always comes from the run'strigger. Full contract: [Chat Protocol, Approvals](./chat_protocol.md#approvals)and [approvalGate](./node-logic-flows/workflow_triggers/approvalGate.md).-The rule also limits what an employee may add to itself from Talk (nothing-that sends or spends; see [Adding tools and skills from Talk](#adding-tools-and-skills-from-talk)),-and a talk agent that asks first is told it cannot send or spend anything for-the owner, only write it for them to send. The summary's `asks_first` follows-the live rule (for a workflow built in Dev mode with no rule, whether it has an-approval gate), shown in a line under the employee's page. A graph built-before the live rule (`builder_version` below 3) needs Apply for a changed rule-to cover everything: `set_ask_first` says so (`needs_apply`).+The summary's `asks_first` follows the live rule (for a workflow built in Dev+mode with no rule, whether it has an approval gate), shown in a line under the+employee's page. A graph an older builder made (`builder_version` below 3) is+upgraded on Apply, Turn on Talk and Start+([upgrade.py](../server/services/employees/upgrade.py)): an ungated app reply+gets a gate (the edge from the agent now goes to the gate, and the reply reads+the gate's text and recipient; one saved mutation with a `delete_edge` op, so+an editor with it open follows), the tools asking first left out come in, a+browser saved read-only is saved whole, and the talk agent gets its talk+tools. Until then `set_ask_first` answers `needs_apply`.## "Done today"diff --git a/docs-internal/workflow_ops_protocol.md b/docs-internal/workflow_ops_protocol.mdindex 5f016f67..476eaebb 100644--- a/docs-internal/workflow_ops_protocol.md+++ b/docs-internal/workflow_ops_protocol.md@@ -102,6 +102,9 @@ An editor adopts them as they are:library skills used to be wiped);- ids the canvas already has are skipped, because the same batch canarrive twice (a retried write announces it again);+- a `delete_edge` takes that edge out (an employee's upgrade puts a gate+ between an agent and its reply, in place of the edge between them);+ one already gone is skipped;- `set_node_parameters` goes to the parameter cache, not the graph.Unlike `save_workflow` this never replaces the graph, so it cannot drop what@@ -161,6 +164,7 @@ failures (toast, log, retry, etc.).| Auto-add Skill on tool connect | WS request `evaluate_auto_skill` (frontend on edge connect/disconnect). The handler reads the wired Master Skill's saved row, so its `set_node_parameters` op is that whole row with only `skills_config` changed | `server/services/auto_skill.py` || Agent Builder runtime tools | `apply_graph_additions`, then the persisted `workflow_ops_apply` push (mid-execution, from the agent's tool call) | `server/nodes/tool/agent_builder/__init__.py` || Turn on Talk (any employee on Home, hired or built in Dev mode) | `apply_graph_additions`, then the persisted push | `server/services/employees/handlers.py` |+| Upgrading an employee an older builder made (on Apply, Turn on Talk, Start) | `apply_graph_additions` with `removed_edges`, then the persisted push | `server/services/employees/upgrade.py` || Vertex managed agent cloud-tool nodes | whole-graph `database.save_workflow`, then a push that is NOT persisted (the editor saves the parameter rows) | `server/nodes/agent/vertex_managed_agent/_ops.py` |## Two delivery modes