Self-improving AI Employees for everyone
Schema source of truth — RFC
ARCHIVED 2026-09-04 — Superseded. Backend-as-SSOT RFC. Everything it deferred to Wave 7 shipped:
VITE_NODESPEC_BACKENDdefaults ON,client/src/nodeDefinitions/was deleted, and option loaders exist for google, android, whatsapp, whatsapp_business, discord, speech and translate.For current state see:
- plugin_system.md — NodeSpec, registries, credentials
- server/nodes/README.md — authoring recipe
Status: ✅ implemented + extended to full plugin registry · Owner: frontend + backend platform · Landing: 2026-04-14 (output schemas) → Wave 6 (NodeSpec contract) → Wave 10 (plugin pattern + visual contract + icons, 2026-04-15)
Landed outcome (Wave 3 — output schemas)
- Backend registry seeded with Pydantic models in server/services/node_output_schemas.py — live size via
len(NODE_OUTPUT_SCHEMAS)— covering every node type the deletedsampleSchemasmap covered, plus all specialized agents and chat models. GET /api/schemas/nodes/{node_type}.jsonserves static JSON Schema (Cache-Controlpublic, max-age=86400). 404 for unknown types.get_node_output_schemaWebSocket handler for authenticated editor fetches.- InputSection.tsx consumes schemas lazy via
queryClient.fetchQuery(['nodeOutputSchema', nodeType]), in-memory cache only. 1,673 → 972 LOC (−701). sampleSchemas map, 20isXxxNodedetection constants, and 60-line pattern-match chain all gone. outputSchemafield removed fromINodeTypeDescription— the frontend type no longer has any shape-describing surface for runtime output.- Adding a new node type's output shape: one Pydantic model in
node_output_schemas.py. Zero frontend change.
Wave 6 extension (input schemas + full NodeSpec)
Wave 3 proved the pattern for output schemas. Wave 6 generalises it to the full parameter contract — the business-logic surface the frontend still owned via nodeDefinitions/*.ts properties arrays, displayOptions, validation rules, dynamic-option loaders, and credential mappings.
Shipped contract:
- Input schemas — server/services/node_input_schemas.py holds the
_DIRECT_MODELSregistry (aliasedNODE_INPUT_MODELS) of Pydantic input models across every node type — live count vialen(_DIRECT_MODELS). Post-Wave-11 each plugin'sParamsclass self-registers here at import (the earlier Wave 6 Literal-union introspection ofSpecializedAgentParams/AndroidServiceParamsis gone — specialized agents and Android service nodes are now individual plugin folders underserver/nodes/agent/andserver/nodes/android/). - NodeSpec envelope — server/services/node_spec.py assembles
{type, displayName, icon, group, description, version, inputs (JSON Schema), outputs (JSON Schema, reuses Wave 3), credentials, uiHints}fusing three sources of truth. - Display metadata — server/models/node_metadata.py defines the
NodeMetadata/NodeHandleTypedDicts and theNODE_METADATAdict — live size vialen(NODE_METADATA). Post-Wave-11 it starts empty at import and is plugin-populated viaservices.node_registry.register_node(called fromBaseNode.__init_subclass__), rather than being hand-maintained. - Endpoints — alongside the Wave 3
/nodes/{type}.jsonoutput endpoint:GET /api/schemas/nodes/{type}/spec.json,/nodes/{type}/input.json,/nodes/specs(list), all with 24hCache-Control. WS mirrors:get_node_spec,list_node_specs. Icon endpoints (RFC §6.5 + F7):GET /api/schemas/nodes/{type}/iconserves the plugin's co-locatedicon.svg(or per-node-typeicon_<nodeType>.svgfor multi-node folders like whatsapp);GET /api/schemas/credentials/{provider}/iconserves the credential brand icon fromserver/credentials/icons/<provider>.svg. Both with long cache headers. - loadOptionsMethod dispatch — per-plugin
_option_loaders.pyfiles (server/nodes/android/_option_loaders.py, server/nodes/google/_option_loaders.py, server/nodes/whatsapp/_option_loaders.py) self-register into a shared loader registry, generalising the WhatsApp-only pattern. Adding a new dynamic-option loader = one registration inside the owning plugin folder.POST /api/schemas/nodes/options/{method}+ WSload_options. - Node-groups index —
GET /api/schemas/nodes/groupsreturns{group: [node_type, ...]}derived from every NodeSpec'sgrouparray — live count vialen(NODE_GROUPS)— replacing the hand-rolled*_NODE_TYPESarrays scattered across 6 frontend files. - Pydantic Field hints — authors encode
displayOptions.show/.hide,loadOptionsMethod,placeholder,validation,typeOptionsviaField(json_schema_extra={...}). Currently ~36 rules encoded across TwitterSend/User, TelegramSend, HttpRequest, WhatsAppSend, SocialSend, Gmail, Calendar.
Frontend consumer (feature-flagged):
VITE_NODESPEC_BACKENDflag at client/src/lib/featureFlags.ts. Defaults OFF — legacynodeDefinitions/*.tswins; ON routes rendering through backend NodeSpec via the adapter.- client/src/lib/nodeSpec.ts — shared
fetchNodeSpec/prefetchAllNodeSpecs/resolveNodeDescriptionhelpers following the Wave 3 colocation pattern (no new hook file). - client/src/adapters/nodeSpecToDescription.ts — NodeSpec wire shape → legacy
INodeTypeDescriptionbridge. Handlesrequired, JSON Schemaformat(dateTime/file/binary),displayOptions,validation, typeOptions lift (loadOptionsMethod,password,rows,editor,editorLanguage,accept, …), and reads hints from both top-level and nesteduiHintswrapper. - Dashboard idle-time prefetch (
prefetchAllNodeSpecsin client/src/lib/nodeSpec.ts) warms every NodeSpec after WebSocket connect. (TheVITE_NODESPEC_BACKENDflag is now the default-on path — backend NodeSpec is the SSOT; the legacynodeDefinitions/*.tsfiles were deleted in the Wave 7/10 reductions.)
Adding a new node type's parameter surface (Wave 6 recipe; superseded post-Wave-11 — see the plugin recipe in 10.C and Wave 11):
At Wave 6 the surface was three separate data files:
- Define
XyzParams(BaseNodeParams)with atype: Literal["xyz"]discriminator and Field constraints, register it in theKnownNodeParamsdiscriminated union, and register it in the_DIRECT_MODELSdict innode_input_schemas.py. - Add
NODE_METADATA["xyz"] = {displayName, icon, group, description, version}innode_metadata.py.
Post-Wave-11 all three collapse onto the plugin class: Params is declared on the BaseNode subclass in server/nodes/<group>/<plugin>/__init__.py, and register_node (via __init_subclass__) writes the class's Params into _DIRECT_MODELS (aliased as NODE_INPUT_MODELS) and its metadata into NODE_METADATA at import time. There is no server/models/nodes.py and no KnownNodeParams union any more — plugin Params extend BaseModel directly (no type: discriminator).
Zero frontend change for parameter rendering. Icons that are SVG assets still need frontend import until SVG payload migration is its own follow-up.
What's deferred (Wave 7):
- Phase 3e flag flip + frontend
nodeDefinitions/*.tsreductions (~3000 LOC deletion). Adapter is ready; snapshot Playwright tests are the remaining prerequisite. - Phase 5.b per-component migration from
*_NODE_TYPESarrays to backend groups. - Gmail/Calendar/Telegram/Twitter loadOptions loaders (Phase 4 registry is ready for one-line registrations).
- ParameterRenderer → DIY widget registry (the capstone). Now unblocked on backend since NodeSpec's
uiHintsalready carry widget routing.
Problem
The editor's Input panel (client/src/components/parameterPanel/InputSection.tsx) needs to show the runtime output shape of every connected source node so users can drag variables into downstream parameters. Today there are two places this shape is declared:
- A 350-line
sampleSchemasmap inline inInputSection.tsx(59 entries). - As-of Wave 3 Phase 1 Batch 1 (commit
f1b2813), a newoutputSchemafield on eachINodeTypeDescriptionthat the frontend consults before the legacy map.
Both are frontend duplications of what the backend already owns — the handler's return type (Pydantic models colocated with each node handler). Every node addition requires a frontend edit to keep schemas in sync.
Historical note. This RFC was written pre-Wave-11 when handler bodies lived under
server/services/handlers/*. Post-Wave-11 those bodies inlined into per-plugin folders underserver/nodes/<group>/<plugin>/__init__.py; onlytodo.py/tools.py/triggers.pyremain underserver/services/handlers/. Path references throughout the earlier sections have been updated to their current locations; the design decisions still hold.
How this scales to 1000+ nodes elsewhere
Raw-GitHub-API research against three schema-driven platforms (n8n ~500 nodes, Activepieces ~400 pieces, Nango 100+ integrations):
n8n — the canonical comparison point, React Flow / Vue inspector:
- Node type descriptions come from the backend via
GET /node-typeson boot (packages/frontend/editor-ui/src/app/stores/nodeTypes.store.ts). - The Input panel (
VirtualSchema.vue) pulls the output shape from two sources in order of preference:- Actual execution data —
getSchemaForExecutionData(props.data). - A JSON Schema preview fetched lazy per-node via
GET /schemas/{nodeType}/{version}/{resource?}/{operation?}.json(schemaPreview.api.ts). These are static JSON files served alongside the backend,withCredentials: false, keyed${nodeType}_${version}_${resource}_${operation}in aMap<string, Result<JSONSchema7, Error>>in-memory cache. - Empty state when neither is available.
- Actual execution data —
Activepieces (packages/pieces/framework/src/lib/*):
- Piece-authors declare Actions in TypeScript inside the piece's backend package. Actions have no
outputSchemafield at all. Triggers carrysampleData: unknown— a free-form example value the UI renders in the variable picker before first run. - The backend emits two payloads to the editor:
PieceMetadataSummary(icon + counts + suggested) for catalog browsing, and the fullPieceMetadata(actions + triggers records) on select.
Nango (packages/types/lib/nangoYaml/index.ts):
- YAML config declares
input: ModelName/output: ModelName[]per sync or action. Model definitions in the same YAML are codegen'd into TypeScript types shipped to both backend and frontend.
Finding: zero of these platforms keep node-shape declarations on the frontend. n8n explicitly serves per-node JSON Schema files from the backend; Activepieces skips declared schemas entirely in favor of real run data.
Decision
Adopt n8n's layered pattern.
Schema source of truth: backend only. Pydantic models colocated with node handlers.
Frontend rendering order (three-tier, mirrors VirtualSchema.vue):
- Real execution data from the most recent run (already wired at
InputSection.tsx:164-174). GET /api/schemas/nodes/{nodeType}.json— JSON Schema served lazy from the backend, cached in-memory per node type.- Empty state /
{ data: 'any' }fallback.
What the frontend keeps:
INodeTypeDescriptionstays a UI-only description:displayName,icon,group,inputs,outputs,properties,defaults,uiHints,credentials. No data schemas.uiHints(Wave 2 Phase 1) is genuinely UI-owned (panel visibility, editor variants, selector dispatch) and stays.
What leaves the frontend:
outputSchemafield onINodeTypeDescription(added by commitf1b2813) — deleted.sampleSchemasmap inInputSection.tsx— deleted once the backend endpoint has coverage for the node types currently listed there.
Backend contract
GET /api/schemas/nodes/{node_type}.json
200 → application/json: JSON Schema 7 describing the node's output shape
404 → node has no declared schema (frontend falls back to run data / empty)
headers: Cache-Control: public, max-age=86400 (or similar)
WebSocket parallel:
request: { type: "get_node_output_schema", node_type: "whatsappReceive" }
response: { schema: JSONSchema7 | null }
Schema generation: each handler's existing Pydantic response model (or a new minimal NodeOutputSchema Pydantic model colocated with the handler) emits JSON Schema via .model_json_schema(). Registry at server/services/node_output_schemas.py: NODE_OUTPUT_SCHEMAS: dict[str, type[BaseModel]] mapping node type → model class. Missing entries return 404.
Frontend changes
New
useNodeOutputSchemaQuery(nodeType)inline at top ofInputSection.tsx(one consumer — inline per the Wave 2/3 colocation rule):function useNodeOutputSchemaQuery(nodeType: string | null) { return useQuery<Record<string, any> | null>({ queryKey: ['nodeOutputSchema', nodeType], queryFn: () => nodeType ? sendRequest('get_node_output_schema', { node_type: nodeType }) .then(r => r?.schema ?? null) : Promise.resolve(null), staleTime: Infinity, enabled: !!nodeType, }); }InputSectiondispatch becomes three-tier:if (executionData?.length) outputSchema = executionData[0][0].json; else if (backendSchema) outputSchema = backendSchema; else outputSchema = { data: 'any' };Delete the
sampleSchemasmap + pattern-match else-if chain + allisXxxconstant flags.Revert commit
f1b2813(frontendoutputSchemaannotations). Remove theoutputSchemafield fromINodeTypeDescription.
Migration order
- Write this RFC. (this file)
- Frontend: add the
useNodeOutputSchemaQueryhook + three-tier dispatch. Keep the legacysampleSchemasmap as a temporary final fallback below the backend call so nothing breaks before the backend endpoint exists. ThenodeDef.outputSchemabranch goes away in this commit (revert + remove the type field). - Backend: implement
/api/schemas/nodes/{node_type}.json+get_node_output_schemaWS handler. Seed the registry from existing Pydantic response models (start with the high-traffic nodes:chatTrigger,webhookTrigger,whatsappReceive,aiAgent/chatAgentfamily,httpRequest, code executors, Google Workspace). - Frontend: once the backend has coverage for every type in the legacy
sampleSchemasmap, delete the map. Verify via manual smoke of the 20 node categories.
What this unblocks
Phase 6 (ParameterRenderer → DIY widget registry) was blocked on the backend emitting NodeSpec { jsonSchema, uiSchema, _uiHints? }. The endpoint defined here is the jsonSchema slice of that same NodeSpec — Phase 6 extends it with uiSchema + _uiHints. Phase 1-REVISED is therefore the on-ramp to Phase 6, not a detour.
Non-goals
- Build-time codegen of TypeScript types from the backend schemas (Nango's pattern) — deferred. The frontend reads schemas as plain JSON; typed inference on top would be nice-to-have but isn't load-bearing.
- Versioning of schemas per node-type version — deferred. All current node types are v1; add
{version}to the URL path the first time we bump.
Wave 10 — plugin pattern + visual contract
Wave 6 made the backend authoritative for node schemas. Wave 10 closed the remaining tribal-code paths so adding a new node = one Python file, zero frontend edits.
10.A — Visual contract extended
NodeMetadata TypedDict in server/models/node_metadata.py
gains the fields the frontend previously hardcoded:
color(hex / dracula token)componentKind(square/circle/trigger/start/agent/chat/tool/model/generic)handles: NodeHandle[]— full React Flow topology (replaces the 400-lineAGENT_CONFIGSmap inAIAgentNode.tsx)credentials,hideOutputHandle,visibility
Each field flows through get_node_spec() into the /api/schemas/nodes/{type}/spec.json
envelope, so every consumer (React Flow dispatch, parameter panel, palette)
reads from one source.
10.C — register_node registry writer
server/services/node_registry.register_node(...)
writes to four registries in one call: NODE_METADATA, _DIRECT_MODELS,
NODE_OUTPUT_SCHEMAS, _HANDLER_REGISTRY (and invalidates the per-type
NodeSpec + input-schema caches). server/nodes/__init__.py recursively
imports every submodule at import time via pkgutil.walk_packages, so
plugin registration is a side effect at startup. Post-Wave-11 the caller is
BaseNode.__init_subclass__ (via register_node_class), so every plugin
folder self-registers — live migrated count via
glob server/nodes/**/__init__.py.
New node checklist (post-Wave-10):
# server/nodes/<group>/<plugin>/__init__.py
from typing import Literal
from pydantic import Field
from services.plugin import ActionNode, NodeContext, Operation
class MyNodeParams(ActionNode.Params):
query: str = Field(default="", json_schema_extra={"placeholder": "Search..."})
class MyNodeOutput(ActionNode.Output):
result: dict = {}
class MyNode(ActionNode):
type = "myNode"
display_name = "My Node"
group = ("tool",)
component_kind = "square"
Params = MyNodeParams
Output = MyNodeOutput
@Operation("run")
async def run(self, ctx: NodeContext, params: MyNodeParams) -> MyNodeOutput:
return MyNodeOutput(success=True, result={...})
Drop icon.svg and meta.json ({"color": "#xxx"}) into the plugin folder. Zero edits elsewhere.
10.B — Icon + color: per-plugin folder + visuals.json fallback
Icons live as icon.svg (or icon_<nodeType>.svg for multi-node folders like whatsapp) in the plugin folder; colors live in meta.json. visuals.json is the fallback registry for emoji / lobehub:<brand> icons + the skill reverse-map. Full resolution chain + wire format + frontend rendering all documented in server/nodes/README.md → Icon + color. Credential brand icons via Credential.get_icon_path() resolve to server/credentials/icons/<provider>.svg or co-located credential_<id>.svg; served by GET /api/schemas/credentials/{provider}/icon.
10.G — Parameter panel fully spec-driven
ParameterRenderer.tsxgainscase 'code'+case 'dateTime'+ genericloadOptionsMethoddispatch via the backendload_optionsWS handler (unlocking the 4 Google Workspace dynamic-option loaders that were previously unreachable).displayOptions.shownow propagates into nestedfixedCollectionrenders. Password masking wins over multi-row textarea.MiddleSection/InputSection/SquareNode/ParameterPanelretire the last 14 hardcoded type-array fallbacks (TRIGGER_NODE_TYPES,AGENT_WITH_SKILLS_TYPES,SKILL_NODE_TYPES, etc.). Every widget decision reads a uiHint or handle-topology fact declared by the node's own plugin module.nodeDefinitions/*.tsfiles strip everyicon:field (−272 LOC) — backend is sole declaration site.INodeTypeDescription.iconnarrowed toicon?: string.
Contract invariants (tests/test_node_spec.py — live count via pytest --collect-only)
TestWave10GContractInvariants enforces:
- every agent-kind node has
uiHints.hasSkills - every tool-kind node has
uiHints.isToolPanel - every Google Workspace node has a field gated by
displayOptions.show.operation - every code executor emits
editor: "code"on itscodefield - every
api_key/apiKeyfield emitspassword: True - every
asset:<key>icon resolves to a real SVG underclient/src/assets/icons/ - every palette group carries non-empty label + icon
Wave 11 — plugin-first class hierarchy
Wave 10 shipped the @register_node decorator and per-file plugin
layout but kept handlers as standalone async functions. Wave 11
promotes nodes to a class-based plugin model, extracts runtime
infrastructure into services/plugin/, and unifies the node-creation
API around a single BaseNode hierarchy. See
plugin_system.md for the canonical reference.
What changed
BaseNodeclass hierarchy. Three concrete kinds —ActionNode/TriggerNode/ToolNode. Each subclass declares everything on its class object:type, metadata,Params+OutputPydantic models,credentials,handles,task_queue,retry_policy, operations.__init_subclass__writes to the four legacy registries on import, so existing consumers are unchanged.@Operationdecorator for multi-op dispatch.- Declarative
RoutingDSL +Connectionfacade (Nango pattern). Plugins never see tokens; routing owns templating + HTTP. - Credential classes (
ApiKeyCredential,OAuth2Credential) with declarativeinject()methods. Auto-registry via__init_subclass__. - Per-plugin Temporal activities (Wave 11.F). Every subclass
exposes
cls.as_activity()with namenode.{type}.v{version}.TemporalWorkerPoolspawns one worker per declaredtask_queuewith tuned concurrency (ai-heavy=4, rest-api=50, triggers-poll=100). - Folder layout mirrors palette groups.
server/nodes/<group>/<name>.py. - Auto-populate trigger registries (Wave 11.D.11). Plugins
declaring
event_type+build_filterauto-register intoevent_waiter.TRIGGER_REGISTRY+FILTER_BUILDERS. - Shared helpers under
services/plugin/:edge_walker.py(agent connection discovery),base.py(BaseNode lifecycle),context.py(typed NodeContext),scaling.py(TaskQueue + RetryPolicy),operation.py(@Operation decorator + collector),routing.py(declarative REST DSL),connection.py(authed httpx wrapper),credential.py,interceptor.py.
Contract invariants
Adds a further set of invariants on top of the Wave 10 suite (live total via pytest --collect-only on tests/test_node_spec.py):
- Every
BaseNodesubclass hastype,display_name,group. Params+Outputare PydanticBaseModelsubclasses.- Every declared credential resolves to
CREDENTIAL_REGISTRY. @Operationnames unique per class;routing=…requires credentials.task_queue∈TaskQueue.ALL;retry_policyisRetryPolicy.- Every
ToolNodeJSON schema has no$defs/$ref. - Every event-mode
TriggerNodedeclaresevent_type. - Fast-path covers every AI-tool-usable plugin.
- Trigger registry auto-populates for every event-mode plugin.
Status
Plugins (live count via glob server/nodes/**/__init__.py) live across 9 Temporal task queues (rest-api,
ai-heavy, code-exec, triggers-poll, triggers-event, android,
browser, messaging, machina-default). Handler bodies are fully
inlined into plugin files — services/handlers/ shrank from
12.8K LOC / 16 files → a few files (live set: todo.py / tools.py /
triggers.py + __init__.py). Only cross-cutting orchestration remains
there: tools.py (AI-tool dispatch + agent delegation through
BaseNode.execute() via the node registry — the live delegation handler is
_execute_delegated_agent), triggers.py (generic event-trigger handler),
and todo.py. The shared Google credential helper moved out of handlers
into its owning plugin folder at
server/nodes/google/_auth_helper.py.
Agent connection collection lives at
server/services/plugin/edge_walker.py::collect_agent_connections
(a 5-tuple: memory, skill, tool, input, task), and per-plugin pre-dispatch
prep is server/nodes/agent/_inline.py::prepare_agent_call.
Sunset of the empty nodes/{agents,services,triggers,tools,utilities}.py
bulk files + dead dispatcher fallbacks is complete (Wave 11.D.13).
Sub-waves shipped
- 11.A / B / B.1 —
BaseNodehierarchy,ActionNode/TriggerNode/ToolNode,__init_subclass__registry writes. - 11.C —
@Operationdecorator + multi-op dispatch. - 11.D.0–13 — handler inlining, bulk-file sunset, trigger registry auto-population, dispatcher cleanup.
- 11.E — declarative
Credentialsubclasses (ApiKeyCredential,OAuth2Credential) withinject()methods and auto-registry viaCredential.__init_subclass__. - 11.E.1 — credentials modularised into per-domain
server/nodes/<group>/_credentials.pyfiles (or inline for single-use). Centralserver/credentials/directory deleted. 18Credentialsubclasses at the time (25 concrete subclasses today). - 11.E.2 / E.3 / E.4 — credential polish + doc sync.
- 11.F — per-plugin Temporal activities
(
cls.as_activity()namednode.{type}.v{version}),TemporalWorkerPoolper task queue with tuned concurrency (ai-heavy=4, rest-api=50, triggers-poll=100, etc.). - 11.G — nodes cookbook (
server/nodes/README.md) + CLAUDE.md sync.
References
- n8n
schemaPreview.api.ts - n8n
VirtualSchema.vue - n8n
nodeTypes.store.ts - n8n node-icon conventions —
file:+fa:prefix-dispatch which Wave 10.B generalises - Activepieces
piece-metadata.ts - Activepieces
action.ts - Nango
nangoYaml/index.ts - @lobehub/icons — the default
<lib>:<brand>target for AI provider brand logos