Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs: capture cache contract, WS replay queue, execution correlation IDs
trohitg671b10ff4f36
diff --git a/CLAUDE.md b/CLAUDE.mdindex bc3a519c..be04c92a 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -5123,4 +5123,10 @@ This function:- **Temporal Activity Heartbeats**: Activities send `activity.heartbeat()` on every non-matching WebSocket broadcast inside the read loop in `services/temporal/activities.py`. This keeps long-running DeepAgent and browser operations alive past the 2-minute `heartbeat_timeout`. Start/end heartbeats alone were causing `TIMEOUT_TYPE_HEARTBEAT` failures on ops taking 5-10 minutes. Connection config: `heartbeat=30`, `receive_timeout=540` (fits within 10-min `start_to_close_timeout`).- **WebSocket `_safe_send` Guard**: `server/routers/websocket.py` checks `websocket.client_state.name != "CONNECTED"` before sending and logs at `debug` level (not `error`) on failure. Prevents "ASGI message after websocket.close" errors when broadcasts race with disconnects.- **Claude Code CLI Flag**: `server/services/claude_code_service.py` uses `--max-budget-usd <amount>` (previously incorrectly `--max-cost`, which caused "unknown option" errors on every run).+- **Persisted-prefix cache contract**: `PersistQueryClientProvider` hydrates entries with the QueryClient's *default* options, so any prefix in `PERSISTED_KEY_PREFIXES` (`client/src/lib/queryPersist.ts`) MUST also have a matching `queryClient.setQueryDefaults(['<prefix>'], { staleTime: FOREVER, gcTime: FOREVER })` declaration in `client/src/lib/queryClient.ts`. Per-call options don't apply on hydration. Canonical set: `nodeSpec`, `nodeGroups`, `credentialValues`, `skillContent`. `credentialCatalogue` has its own `idb-keyval` warm-start path so it is intentionally not in either list.+- **WebSocket `sendRequest` replay queue**: when the socket is not open, requests enqueue with `AbortController`-backed per-request timeouts (default 30s) and replay on reconnect inside `ws.onopen` before `setIsReady(true)`. Queue capped at 200 with FIFO eviction. Intentional close (`event.code === 1000`) drops the queue; transient closes preserve it. Eliminates indefinite spinners during the 3s reconnect window. Implementation in `client/src/contexts/WebSocketContext.tsx` (`pendingSendQueueRef` + `drainPendingSends`).+- **`currentWorkflowId` single source**: lives in `useAppStore.currentWorkflow.id` only. Non-React listeners (WS handlers) read via `useAppStore.getState().currentWorkflow?.id` -- the documented Zustand escape hatch. The push to `nodeStatusStore.setCurrentWorkflowId` is driven from one `useEffect` in `Dashboard.tsx`. Removed the prior `currentWorkflowIdRef` mirror inside WebSocketContext that misrouted broadcasts during workflow switches.+- **Execution correlation IDs**: `handle_execute_node` issues a `uuid4().hex` token at request entry, propagates it through every `node_status` / `node_output` broadcast for the run, and returns it in the response payload as `execution_id`. Frontend `ExecutionResult.executionId` carries it; `OutputSection` dedups runs by it instead of `JSON.stringify(outputs)` (which collapsed distinct executions whose payloads matched). Same trace-id pattern as OpenTelemetry / HTTP request ids.+- **`clear_node_status` idle reset**: `StatusBroadcaster.clear_node_status(node_id)` resets the slot to `{status: "idle", data: {}, cleared: true}` instead of `del`'ing it. Deleting created a race window where the in-flight execution's `success` broadcast re-created the entry and stuck the UI on "completed" for a cancelled node. Idle reset preserves entry identity so subsequent broadcasts update normally; the `cleared: true` flag distinguishes "never ran" from "explicitly cleared."+- **`get_node_output` race**: in-memory `_outputs` cache re-population after a DB-fallback `await` previously overwrote a fresh in-memory write with a stale DB value. Fix uses nested `dict.setdefault` (atomic at the CPython GIL level -- no lock needed). `store_node_output` and `clear_all_outputs` had no real race because asyncio coroutines do not preempt at synchronous statements. Reference for the Python concurrency model: https://docs.python.org/3/library/asyncio-task.html#asyncio-await- never use emojis in prints\ No newline at end of filediff --git a/docs-internal/frontend_architecture.md b/docs-internal/frontend_architecture.mdindex dcb579d9..641b7c1f 100644--- a/docs-internal/frontend_architecture.md+++ b/docs-internal/frontend_architecture.md@@ -374,6 +374,8 @@ components/credentials/CredentialsModal.tsx**Slice-subscribed cache entries MUST set `gcTime: GC_TIME.FOREVER`.** Slice subscribers don't register as TanStack observers, so without this override the cache entry is garbage-collected after the default `gcTime` (5 min) and every consumer reads `undefined`. The user-visible regression is "canvas nodes lose their icons / handles after idle." Applies to `fetchNodeSpec`, `fetchNodeGroups`, and the `useNodeGroups` `useQuery`; the persistor in `lib/queryPersist.ts` only handles cross-reload survival.+**Anchor cache contracts at the prefix root via `setQueryDefaults`, not per-call options.** `PersistQueryClientProvider` hydrates entries from localStorage with the QueryClient's *default* options, so per-call `staleTime: FOREVER` does not stop `gcTime: 5min` eviction on hydration. Every persisted prefix must have a matching `queryClient.setQueryDefaults(['<prefix>'], { staleTime: FOREVER, gcTime: FOREVER })` declaration in [client/src/lib/queryClient.ts](../client/src/lib/queryClient.ts). The current canonical set is `['nodeSpec']`, `['nodeGroups']`, `['credentialValues']`, `['skillContent']`. The persistor whitelist in [client/src/lib/queryPersist.ts](../client/src/lib/queryPersist.ts) must mirror it; a string in the whitelist that doesn't match a real query key (the prior `'pluginCatalogue'` typo) is silently dead. `credentialCatalogue` is intentionally NOT in either list -- it has its own `idb-keyval` warm-start.+**Component rules:**- `PanelRenderer` lazy-loads each panel type so the initial JS payload doesn't grow linearly with provider count.- Panels are config-driven: `StatusCard`, `ActionBar`, `FieldRenderer`, `OAuthConnect` consume `ProviderConfig` fields rather than hand-coding per-provider JSX.@@ -395,6 +397,8 @@ See [ui_migration_plan.md](./ui_migration_plan.md) Phase 6.- `useEffect` fetch-on-mount is banned for anything the backend can push. Subscribe to the context slice instead.- All modifying operations go through WebSocket — REST is reserved for auth + webhooks.- No polling. If a component wants fresh data, call `sendRequest` once (or use TanStack Query with `staleTime: Infinity` + manual `invalidate`).+- **`sendRequest` queues during disconnect with backpressure.** When the socket is not open, the request enqueues with an `AbortController`-backed per-request timeout (default 30s) and replays on reconnect inside `ws.onopen` before `setIsReady(true)`. Queue caps at 200 with FIFO eviction (rejects oldest with `backpressure: too many queued requests`). Intentional close (`event.code === 1000`) drops the queue; transient closes preserve it. Eliminates indefinite spinners during the 3-second reconnect window. Implementation: `pendingSendQueueRef` + `drainPendingSends` in `WebSocketContext.tsx`.+- **`currentWorkflowId` lives in `useAppStore` only.** Non-React listeners (WS handlers) read it via `useAppStore.getState().currentWorkflow?.id` -- the documented Zustand escape hatch (https://github.com/pmndrs/zustand#read-state-without-subscription). The previous `currentWorkflowIdRef` mirror inside WebSocketContext was a one-render-late copy that misrouted broadcasts during workflow switches. The push to `nodeStatusStore.setCurrentWorkflowId` is driven from a single `useEffect` in `Dashboard.tsx`.## Ownership boundary: TanStack Query vs Zustand vs WebSocketContextdiff --git a/docs-internal/status_broadcaster.md b/docs-internal/status_broadcaster.mdindex dda952fe..82da0fb2 100644--- a/docs-internal/status_broadcaster.md+++ b/docs-internal/status_broadcaster.md@@ -142,6 +142,20 @@ Broadcasts are sent to all connected clients without a request-response correlat| `compaction_starting` | Memory compaction begins | `{session_id, node_id}` || `compaction_completed` | Memory compaction ends | `{session_id, success, tokens_before, tokens_after}` |+## Execution correlation IDs++Each `execute_node` request generates a `uuid4().hex` token at handler entry (`server/routers/websocket.py`, `handle_execute_node`). The token is:++- propagated through every `node_status` broadcast for the run (`executing` -> `success` / `error` / `idle`),+- propagated through the matching `node_output` broadcast,+- returned in the `execute_node` response payload as `execution_id`.++The frontend's `ExecutionResult` type carries the token as `executionId`. `OutputSection` dedups by `executionId` directly; the previous JSON-stringified output equality check collapsed two distinct executions whose payloads happened to match. The token also lets future replay / retry / observability work attach metadata to a single run -- same role as a request id in HTTP middleware or a trace id in OpenTelemetry. When neither side carries an `executionId` (legacy results, catch-block stubs), `OutputSection` falls back to structural equality on `outputs`.++## `clear_node_status` is an idle reset, not a delete++`StatusBroadcaster.clear_node_status(node_id)` resets the slot to `{status: "idle", data: {}, cleared: true}` instead of `del`'ing the dict entry. The previous implementation deleted the slot, which created a race window: an in-flight execution's subsequent `success` broadcast re-created the entry and the UI was stuck showing "completed" on a node the user just cancelled. Resetting to idle preserves entry identity (so subsequent broadcasts update it normally) and the `cleared: true` flag lets callers tell apart "never ran" from "explicitly cleared."+## Android Two-State Connection ModelAndroid support uses a two-state model because a relay WebSocket can be connected without a device being paired. The UI needs both signals.