Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs: align with credential brand-name rename + logging-infra additions
trohitg977ba2aa73bf
diff --git a/CLAUDE.md b/CLAUDE.mdindex 888342f7..47c68f25 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -385,8 +385,11 @@ from services.some_service import global_instance#### Logging Configuration```bash# In server/.env-LOG_LEVEL=INFO # Default: INFO, DEBUG for verbose-LOG_FORMAT=text # text or json+LOG_LEVEL=INFO # Default: INFO, DEBUG for verbose+LOG_FORMAT=json # 'json' (default) or 'text' for console+LOG_FILE= # Optional rotating-file destination+LOG_FILE_MAX_BYTES=10485760 # 10 MiB ceiling per file+LOG_FILE_BACKUP_COUNT=5 # Keep 5 backups (50 MiB total cap)```**What logs at each level:**@@ -394,6 +397,68 @@ LOG_FORMAT=text # text or json- `INFO`: Workflow completion, deployment start/stop, significant state changes- `ERROR`: Failures, exceptions, validation errors+#### Logging Infrastructure (canonical patterns)++**Console mode is timestamp-less by design.** The supervisor+(`machina/colors.py`) prepends `[HH:MM:SS.fff]` to every aggregated+line, so `configure_logging` does NOT add an inner `TimeStamper` in+console mode. JSON mode keeps ISO timestamps for machine consumers.+Helpers that print pre-logger init (`_startup_log` in `main.py`,+`_clog` in `core/container.py`) emit raw `print()` so the CLI prefix+is the single timing source.++**Context propagation via `structlog.contextvars`.** Bind once at the+entry point; every log record inside that async context picks the+fields up automatically. Stdlib `contextvars` rides `asyncio.gather`+child tasks.++```python+from core.logging import log_context++async with log_context(workflow_id=wf_id, node_id=node_id):+ await do_work() # all logs inside carry workflow_id + node_id+```++`BaseNode.execute()` already wraps its body in+`log_context(node_id, node_type, workflow_id?)` so plugin operation+logs are auto-tagged — don't pass these as kwargs at each call site.++**Per-plugin OpenTelemetry span.** `BaseNode.execute()` opens a+`node.<type>.execute` span with attributes `node.id` / `node.type` /+`workflow.id` around the operation body. Single edit instruments every+plugin — no per-plugin span code needed.++**Source-tag resolver for the Terminal UI panel.** `record.name`+collapses to a ≤12-char tag via `_resolve_source_tag` in+`core/logging.py`:++1. `nodes.<plugin>.*` → `<plugin>` (auto-rule; no per-plugin entry)+2. `routers.<name>.*` → `<name>` (auto-rule)+3. Explicit registry `_LOG_SOURCE_TAGS` — only for cross-cutting+ services with long module names (`workflow_validator` → `validator`,+ `status_broadcaster` → `broadcaster`, `user_auth` → `auth`, etc.)+4. Second-segment fallback (`services.ai` → `ai`)++Plugins that genuinely want a different label from their folder name+call `register_log_source_tag(prefix, tag)` from their package+`__init__.py` — same self-registration pattern as the five plugin+registries (`ws_handler`, `filter_builder`, `trigger_precheck`,+`service_refresh`, `output_schema`).++**RotatingFileHandler** swaps in when `LOG_FILE` is set — no+unbounded log growth.++**NodeUserError vs Exception contract** (`services/plugin/base.py`):+- `NodeUserError` → single WARN line, no traceback, structured response+- `PermissionError` annotated with `.provider` / `.reason` / `.auth` →+ `error_type="PermissionDeniedError"` + `credential` envelope block ++ CloudEvents `credential.{auth}.runtime_failed` broadcast+- Bare `Exception` → `logger.exception` with full traceback++Reach for `NodeUserError` for any user-correctable failure+(missing required field, unknown enum value, bad regex). Reserve+`RuntimeError` / `Exception` for genuinely unexpected server bugs.+### 6. Cleanup & Lifecycle- **Use existing teardown methods** - e.g., `_teardown_all_cron_triggers()` for cron cleanup- **Cleanup in finally blocks** - Ensure resources are released even on error@@ -1115,7 +1180,7 @@ Telegram bot integration using `python-telegram-bot` SDK with long-polling for i| `telegram_receive.py` | `TelegramReceiveNode(TriggerNode)` + `TelegramReceiveOutput` — the trigger plugin (declares `event_type = "telegram_message_received"`). || `client/src/components/CredentialsModal.tsx` | Telegram bot token panel (frontend; identifies commands by WebSocket message-type strings, not Python paths). |-**Authentication:** Bot token from @BotFather on Telegram. Stored via `auth_service.store_api_key('telegram_bot_token', token, models=[])`. **Owner detection has three independent layers** (commit f746ddf): (1) explicit `Your Chat ID (optional)` text input in the Telegram credentials modal — declared as a secondary `FieldDef` in `credential_providers.json` and rendered by `ApiKeyPanel`'s `SecondaryFieldRow`; saves directly to `telegram_owner_chat_id` via `auth_service.store_api_key`; (2) pre-poll peek in `connect()` via `_capture_owner_from_pending_updates()` — `bot.get_updates(timeout=0)` BEFORE `start_polling(drop_pending_updates=True)` discards the queue; (3) atomic write-through in `_on_message_received` — persists FIRST, sets in-memory only on success, ERROR-level logging on failure. Lazy fallback in `telegram_send.py` reads DB and calls `service.set_owner` if in-memory is empty. Credentials panel: Save (store only), Connect (store + connect), Reconnect. All credential access uses `from core.container import container; container.auth_service()` (NOT `from services.auth import get_auth_service`).+**Authentication:** Bot token from @BotFather on Telegram. Stored via `auth_service.store_api_key('telegram', token, models=[])` (renamed from `telegram_bot_token` — class id, catalogue key, and co-located SVG basename now all align with the brand name). **Owner detection has three independent layers** (commit f746ddf): (1) explicit `Your Chat ID (optional)` text input in the Telegram credentials modal — declared as a secondary `FieldDef` in `credential_providers.json` and rendered by `ApiKeyPanel`'s `SecondaryFieldRow`; saves directly to `telegram_owner_chat_id` via `auth_service.store_api_key`; (2) pre-poll peek in `connect()` via `_capture_owner_from_pending_updates()` — `bot.get_updates(timeout=0)` BEFORE `start_polling(drop_pending_updates=True)` discards the queue; (3) atomic write-through in `_on_message_received` — persists FIRST, sets in-memory only on success, ERROR-level logging on failure. Lazy fallback in `telegram_send.py` reads DB and calls `service.set_owner` if in-memory is empty. Credentials panel: Save (store only), Connect (store + connect), Reconnect. All credential access uses `from core.container import container; container.auth_service()` (NOT `from services.auth import get_auth_service`).**Environment Variables:**```bash@@ -3428,7 +3493,11 @@ When `simpleMemory` is connected to `claude_code_agent`, memory continuity is ha**Parallel-batch guard.** When memory is wired AND `len(tasks) > 1`, `claude_code_agent` raises `NodeUserError` at handler entry — concurrent `--continue` spawns against one project_key would race claude's session-resolution.-**Live UI refresh.** `_persist_memory` broadcasts `node_parameters_updated` after `save_node_parameters` so the simpleMemory parameter panel auto-refetches the moment a turn completes — without this the DB has the latest conversation but the UI shows the stale snapshot loaded at workflow open. Mirrors the pattern in `routers/websocket.py:handle_save_node_parameters`.+**Live UI refresh — CloudEvents v1.0 envelope.** `_persist_memory` calls `broadcaster.broadcast_node_parameters_updated(...)` after `save_node_parameters`. The broadcast carries a `WorkflowEvent` envelope (`type: "com.machinaos.node.parameters.updated"`, `subject: <memory_node_id>`, `data: {node_id, parameters, version, source: "cli"}`) wrapped in the wire-key `{type: "node_parameters_updated", data: <envelope>}`. The FE handler in `client/src/contexts/WebSocketContext.tsx` casts to `WorkflowEvent<{node_id, parameters, version, source}>`, reads `inner.node_id || envelope.subject`, and routes the inner payload to `setNodeParameters` + `queryClient.setQueryData` so the simpleMemory parameter panel auto-refreshes the moment the turn completes. Pre-cutover the FE read the old flat top-level keys (`message.parameters`, `message.node_id`) and silently dropped every broadcast — symptom was "memory only updates after a page reload."++**Workspace dir routed via `--add-dir`.** The per-workflow workspace (`data/workspaces/<workflow_id>/`, injected into `ctx.raw` by `workflow.py:_get_workspace_dir`) is spliced into each task's `add_dir` list in `AICliService.run_batch` so the spawned claude can read files dropped by upstream nodes (`fileDownloader`, `documentParser`, code executors). Without this the workspace is invisible: memory-bound runs spawn with `cwd=repo_root` (stable for `--continue`'s `project_key`) and non-memory runs with `cwd=worktree`, neither of which sees the workspace files. Mirrors the ai_agent pattern (`services/ai.py:1899` — `config['workspace_dir'] = context.get('workspace_dir', '')`) but uses claude's native `--add-dir` instead of MCP-tool-config injection because claude has its own filesystem tools.++**Stale-model coercion.** `ClaudeCodeAgentParams.model` is a `Literal` of supported model IDs + aliases — strict validation would reject legacy saved values like `"claude-sonnet-4.6"` (dot-spelled), `"claude-3-5-sonnet-20241022"` (date-suffixed), `""`. A `field_validator("model", "fallback_model", mode="before")` coerces unknown values to the default (`"claude-sonnet-4-6"` for `model`, `None` for `fallback_model`) so old workflows keep loading; the UI dropdown still constrains new edits.**Logs to grep on a live run:**- `[Claude Code memory] memory_node=<id> -> --continue (claude auto-finds latest session under cwd)`diff --git a/docs-internal/credentials_panel.md b/docs-internal/credentials_panel.mdindex 76418231..6eba5b57 100644--- a/docs-internal/credentials_panel.md+++ b/docs-internal/credentials_panel.md@@ -90,7 +90,7 @@ After validation succeeds, the frontend still calls `saveApiKey` (or backend sto### 2.3 Pattern C — Save-only API key (no validation)-**Providers**: android_remote, serper, twitter_client_id, twitter_client_secret, google_client_id, google_client_secret, telegram_bot_token, email_*.+**Providers**: android_remote, serper, twitter_client_id, twitter_client_secret, google_client_id, google_client_secret, telegram, email_*.`saveApiKey(provider, key)` calls WebSocket `save_api_key` directly. Backend `handle_save_api_key` (websocket.py:1023) skips model fetch and just calls `auth_service.store_api_key(provider, key, models=[])`.@@ -227,25 +227,25 @@ No logout WebSocket handler — user can manually delete the credentials file or### 2.9 Pattern I — Bot token + polling (Telegram)-**Credentials stored**: API key table — `telegram_bot_token`. Owner chat ID auto-captured on first private message and stored as `telegram_owner_chat_id`.+**Credentials stored**: API key table — `telegram`. Owner chat ID auto-captured on first private message and stored as `telegram_owner_chat_id`.**Happy path**:```1. User pastes BotFather token, clicks Save- -> saveApiKey('telegram_bot_token', token) // Pattern C+ -> saveApiKey('telegram', token) // Pattern C2. User clicks Connect-> sendRequest('telegram_connect', {token})handle_telegram_connect (websocket.py:1844):-> TelegramService.connect(token)-> validate via getMe-> start long-polling (get_updates_read_timeout=30s)- -> auth_service.store_api_key('telegram_bot_token', token, models=[])+ -> auth_service.store_api_key('telegram', token, models=[])3. broadcast 'telegram_status' arrives with connected=true, bot_username, bot_name, bot_id4. First private message captures owner_chat_id -> stored as telegram_owner_chat_id API key5. broadcast 'telegram_status' updated with owner_chat_id```-**Disconnect**: `telegram_disconnect` stops polling AND removes both `telegram_bot_token` and `telegram_owner_chat_id` API keys.+**Disconnect**: `telegram_disconnect` stops polling AND removes both `telegram` and `telegram_owner_chat_id` API keys.**Reconnect**: Re-establishes polling using stored token (no token re-entry).@@ -329,7 +329,7 @@ Provider × table examples:- Google access/refresh tokens → OAuth token table under provider=`google`- `twitter_client_id`, `twitter_client_secret` → API key table- Twitter access/refresh → OAuth token table under provider=`twitter`-- `telegram_bot_token` → API key table (Pattern I uses no OAuth table)+- `telegram` → API key table (Pattern I uses no OAuth table)### 4.4 Encryption layer[server/core/encryption.py](../server/core/encryption.py): `EncryptionService` uses Fernet (AES-128-CBC + HMAC-SHA256) with PBKDF2-SHA256 key derivation (600,000 iterations, OWASP 2024). Salt is 32 random bytes, generated on first DB init and stored in `credentials_metadata` table. The encryption password is the server-scoped `API_KEY_ENCRYPTION_KEY` env var, loaded once at startup ([server/main.py](../server/main.py) lifespan).diff --git a/docs-internal/credentials_scaling/architecture.md b/docs-internal/credentials_scaling/architecture.mdindex 32ddf6cf..4628351f 100644--- a/docs-internal/credentials_scaling/architecture.md+++ b/docs-internal/credentials_scaling/architecture.md@@ -117,7 +117,7 @@ User presses Ctrl+K or clicks sidebar → search input focus↓User types "whats" → fuzzysort.go() → 3ms → filtered list↓-User selects WhatsApp → setSelectedId('whatsapp_personal')+User selects WhatsApp → setSelectedId('whatsapp')↓PanelRenderer looks up config by id → determines kind='qrPairing'↓diff --git a/docs-internal/node-logic-flows/telegram_social/telegramReceive.md b/docs-internal/node-logic-flows/telegram_social/telegramReceive.mdindex 6bd0056c..1eb1668a 100644--- a/docs-internal/node-logic-flows/telegram_social/telegramReceive.md+++ b/docs-internal/node-logic-flows/telegram_social/telegramReceive.md@@ -124,7 +124,7 @@ flowchart TD## External Dependencies-- **Credentials**: bot token via `auth_service.get_api_key("telegram_bot_token")`+- **Credentials**: bot token via `auth_service.get_api_key("telegram")`(read by the service on connect, not by the handler).- **Services**: `TelegramService`, `event_waiter`, `StatusBroadcaster`.- **Python packages**: `python-telegram-bot` v22.x.diff --git a/docs-internal/node-logic-flows/telegram_social/telegramSend.md b/docs-internal/node-logic-flows/telegram_social/telegramSend.mdindex e6ec9f5a..4fc434cc 100644--- a/docs-internal/node-logic-flows/telegram_social/telegramSend.md+++ b/docs-internal/node-logic-flows/telegram_social/telegramSend.md@@ -137,7 +137,7 @@ flowchart TD## External Dependencies- **Credentials**: `auth_service.get_api_key("telegram_owner_chat_id")` when- restoring the self-owner. Bot token comes from `auth_service.store_api_key("telegram_bot_token", ...)`+ restoring the self-owner. Bot token comes from `auth_service.store_api_key("telegram", ...)`and is consumed by `TelegramService.connect()` elsewhere, not by this handler.- **Services**: `TelegramService` singleton (`services.telegram_service.get_telegram_service`).- **Python packages**: `python-telegram-bot` (v22.x), `markdown-it-py` (via