Skip to content

Self-improving AI Employees for everyone

docs: prepend Wave 6 backend-first recipe to node_creation + new_service_integration

trohitg4a438d323a1d
diff --git a/docs-internal/new_service_integration.md b/docs-internal/new_service_integration.md
index 8d040215..0caa7c83 100644
--- a/docs-internal/new_service_integration.md
+++ b/docs-internal/new_service_integration.md
@@ -2,6 +2,8 @@
This guide provides a comprehensive walkthrough for integrating new external services (like Google Workspace, Slack, Notion, etc.) into MachinaOs. It covers OAuth authentication, database models, API handlers, frontend nodes, and AI Agent tool integration.
+> **Wave 6 update (April 2026):** the **Frontend Node Definitions** step (#4 below) has shrunk dramatically. Parameter schemas, validation, conditional visibility, and dynamic-option dispatch now live on the backend via Pydantic + the NodeSpec contract. The legacy frontend `nodeDefinitions/*.ts` `properties` arrays are still active until Phase 3e flips `VITE_NODESPEC_BACKEND` ON, but new integrations should follow the [Wave 6 recommended recipe](./node_creation.md#wave-6-recommended-recipe-backend-first) in the node creation guide so they ship through the new path automatically.
+
## Overview
A complete service integration includes:
@@ -9,10 +11,14 @@ A complete service integration includes:
1. **OAuth Service** - Handle authentication with the external provider
2. **Database Models** - Store OAuth tokens and connection state
3. **API Handlers** - Execute service operations (CRUD)
-4. **Frontend Node Definitions** - Visual workflow nodes
-5. **AI Tool Schemas** - Enable LLM tool calling
-6. **Credentials Modal** - UI for managing connections
-7. **Pricing Configuration** - Cost tracking for API usage
+4. **Pydantic input model + NODE_METADATA entry** - The Wave 6 recipe replaces the bulk of the legacy frontend node definition work. See [node_creation.md Wave 6 section](./node_creation.md#wave-6-recommended-recipe-backend-first).
+5. **Dynamic-option loaders** - If your service has dropdown fields needing live data (label list, channel list, calendar list), add a loader to [server/services/node_option_loaders/](../server/services/node_option_loaders/) and register in `LOAD_OPTIONS_REGISTRY`. One-line registration; see Google Workspace loaders as the reference (`gmailLabels`, `googleCalendarList`, `googleDriveFolders`, `googleTasklists`).
+6. **Frontend visual-component routing** - Add to the appropriate `*_NODE_TYPES` list or routing branch in [client/src/Dashboard.tsx](../client/src/Dashboard.tsx) so React Flow knows which component to render.
+7. **AI Tool Schemas** - Enable LLM tool calling
+8. **Credentials Modal** - UI for managing connections
+9. **Pricing Configuration** - Cost tracking for API usage
+
+The sections below detail each step using Google Workspace as the reference integration. Sections 1-3 + 7-9 are unchanged in Wave 6; section 4-6 are where the backend-first recipe materially cuts the work.
## Architecture Pattern
diff --git a/docs-internal/node_creation.md b/docs-internal/node_creation.md
index 947e7518..acf48e2e 100644
--- a/docs-internal/node_creation.md
+++ b/docs-internal/node_creation.md
@@ -4,12 +4,86 @@ This guide explains how to create new nodes for MachinaOs, covering frontend def
> **Related Documentation:**
> - [Workflow Schema](./workflow-schema.md) - JSON schema, edge handle conventions, config node architecture
+> - [Schema Source of Truth RFC](./schema_source_of_truth_rfc.md) - Wave 3 + Wave 6 backend-as-SSOT design
> - [CLAUDE.md](../CLAUDE.md) - Project overview, key files, architecture patterns
+---
+
+## Wave 6 recommended recipe (backend-first)
+
+> **Status:** Wave 6 backend infrastructure shipped on `feature/credentials-scaling-v2`. The frontend feature flag `VITE_NODESPEC_BACKEND` defaults OFF, so the legacy frontend-`nodeDefinitions/*.ts` recipe (sections below) still works. Once Phase 3e flips the flag, **the only authoring location for parameter schemas is the backend**.
+
+For new nodes added today, follow the backend-first recipe — the legacy frontend definition becomes a thin manifest (or vanishes entirely) once the flag flips. You author once on the backend; the frontend renders via the NodeSpec adapter.
+
+### 4-step recipe
+
+1. **Pydantic input model** — declare in [server/models/nodes.py](../server/models/nodes.py):
+ ```python
+ class MyNodeParams(BaseNodeParams):
+ """Parameters for my new node."""
+ type: Literal["myNode"]
+ operation: Literal["create", "update", "delete"] = "create"
+ name: str = Field(default="", json_schema_extra={
+ "displayOptions": {"show": {"operation": ["create", "update"]}},
+ })
+ resource_id: str = Field(
+ default="",
+ alias="resourceId",
+ json_schema_extra={"displayOptions": {"show": {"operation": ["update", "delete"]}}},
+ )
+ ```
+ Use `Field(ge=, le=, alias=, default=)` for constraints and camelCase aliasing. Use `json_schema_extra` for `displayOptions`, `loadOptionsMethod`, `placeholder`, `validation`, `typeOptions` (`password`, `rows`, `editor`, `editorLanguage`, `accept`, `multipleValues`, …).
+
+2. **Register in the discriminated union** in the same file's `KnownNodeParams = Annotated[Union[...]]`.
+
+3. **Pydantic input registry** — add to `_DIRECT_MODELS` in [server/services/node_input_schemas.py](../server/services/node_input_schemas.py):
+ ```python
+ "myNode": MyNodeParams,
+ ```
+
+4. **Display metadata** — add to `NODE_METADATA` in [server/models/node_metadata.py](../server/models/node_metadata.py):
+ ```python
+ "myNode": {
+ "displayName": "My Node",
+ "icon": "🔧",
+ "group": ["category", "tool"],
+ "description": "What this node does",
+ "version": 1,
+ },
+ ```
+
+That's it. The NodeSpec auto-emits at `GET /api/schemas/nodes/myNode/spec.json` with the full parameter contract, validation rules, conditional visibility, and dynamic option routing. The output schema (Wave 3) is a separate Pydantic model in [server/services/node_output_schemas.py](../server/services/node_output_schemas.py) registered in `NODE_OUTPUT_SCHEMAS`.
+
+### Backend handler
+
+The handler in `server/services/handlers/` and registry entry in [server/services/node_executor.py](../server/services/node_executor.py) `_build_handler_registry()` are unchanged from the legacy recipe — still required, see [Backend: Workflow Handler](#backend-workflow-handler) below.
+
+### Dynamic option loaders
+
+If your node has a dropdown field that needs to fetch options at edit-time (e.g. Gmail labels, WhatsApp groups, Calendar list):
+
+1. Write an async loader in [server/services/node_option_loaders/](../server/services/node_option_loaders/) following the established `whatsapp_loaders.py` / `google_loaders.py` pattern.
+2. Register it in `LOAD_OPTIONS_REGISTRY` (one line) in `node_option_loaders/__init__.py`.
+3. Add `json_schema_extra={"loadOptionsMethod": "myMethodName"}` to the relevant Pydantic Field.
+
+The dropdown auto-populates via the unified `POST /api/schemas/nodes/options/{method}` dispatcher.
+
+### Frontend (Wave 6 path)
+
+Today: still need a thin entry in `client/src/nodeDefinitions/*.ts` so the editor knows the node type exists (palette + Dashboard routing). Once Phase 3e flips the flag, the entry can drop to just a `*_NODE_TYPES` array membership (or vanish entirely if backend `group` membership is enough).
+
+Visual component routing in [client/src/Dashboard.tsx](../client/src/Dashboard.tsx) is type-id based and still required. Pick a component (`SquareNode`, `AIAgentNode`, `TriggerNode`, `ToolkitNode`, `ModelNode`) by adding to the relevant `NODE_TYPES` list or routing branch.
+
+### What the legacy sections below describe
+
+The remaining content is the pre-Wave-6 frontend-first recipe: authoring `properties: [...]`, `displayOptions`, `defaults` arrays inside `client/src/nodeDefinitions/*.ts`. **All still works** until Phase 3e — but new authoring should use the Wave 6 recipe above so your node ships through the new path automatically when the flag flips.
+
+---
+
## Table of Contents
1. [Node Types Overview](#node-types-overview)
-2. [Frontend: Node Definition](#frontend-node-definition)
+2. [Frontend: Node Definition](#frontend-node-definition) (legacy)
3. [Frontend: Visual Component](#frontend-visual-component)
4. [Backend: Workflow Handler](#backend-workflow-handler)
5. [Template Resolution](#template-resolution)