Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
fix(Wave 13): canary event framework — EventType SA + canary-only emit + status lifecycle + OOM + templates + cancel sweep
trohitgbae0954cdd21
diff --git a/CLAUDE.md b/CLAUDE.mdindex 32c63fce..fcd5e629 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -5233,4 +5233,5 @@ This function:- **Telegram owner detection — three layers** (f746ddf): the bot owner is captured by THREE independent paths so the realistic setup flows all converge to a working state. (1) **Explicit field**: secondary `FieldDef` in the credentials modal lets the user paste their Telegram user ID directly (recommended; works without DM'ing the bot). (2) **Pre-poll peek** in `connect()` via `_capture_owner_from_pending_updates()` — calls `bot.get_updates(timeout=0)` BEFORE `start_polling(drop_pending_updates=True)` discards the queue, scans for any historical private DM, captures + persists atomically. (3) **Atomic write-through in `_on_message_received`** — persists FIRST, sets in-memory ONLY on success. Invariant: "in-memory has owner ⇒ DB has owner" so a process restart can re-capture cleanly. Failure logged at `ERROR` with `exc_info=True` (was `WARNING` previously, masking persist failures). The lazy fallback in `telegram_send.py` (read DB → `service.set_owner`) sits on top of all three.- **Supervisor Job Object loud-failure (machina/tree.py + supervisor.py)** (8a64eb9): the Windows process-tree mechanism depends entirely on a Job Object with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` (no fallback — `start_new_session` is POSIX-only). `pywin32>=308` is a hard supervisor dependency on Windows; the declaration in `machina/pyproject.toml` carries an inline comment so future maintainers don't drop it. `_JobObject.__init__` splits failures into explicit `ImportError` (with reinstall command) and `Exception` (with `repr`) branches — both write to `stderr` instead of swallowing. `_JobObject.add()` verifies enrollment via `win32job.IsProcessInJob(handle, self._handle)` after assignment; on mismatch it queries `IsProcessInJob(handle, None)` so the warning explicitly names the wrapper-job hypothesis (npm / pnpm / conhost wrappers occasionally place us in their own non-nesting Job). `supervisor._spawn_once` checks `add_to_job` return value and emits a yellow `WARN: pid=N not enrolled in Job Object` per child if False. Without these guards, a stale pywin32 install or a wrapping Job Object silently leaks orphan Python processes on every Ctrl-C and they accumulate to the point of holding SQLite locks that block subsequent backend startup.- **Credentials envelope-shape invariant** (dc94cde): the `useCredentialPanel` query stores `{values: CredentialFormValues, hadStored: boolean}` (an envelope, NOT a flat dict). The earlier `writeValues` called `setQueryData<CredentialFormValues>` with a spread `(prev) => ({...prev, [key]: value})`, which at runtime merged the typed character at the envelope level next to `values` and `hadStored` instead of inside `.values` — the input selector `panel.values[field.key]` re-rendered with the original (server) value on every keystroke and the input felt frozen. Fix: `writeValues` now preserves the envelope shape and updates only the inner `values` dict; `hadStored` is preserved verbatim (it reflects real server state, not local edits). When introducing other queries shaped as `{data, meta}` envelopes, follow the same pattern — never spread `prev` as if it were the inner payload.+- **Wave 13 canary fixes** — six load-bearing corrections to the Wave 12 event framework. See [docs-internal/event_framework.md → Wave 13 fixes](./docs-internal/event_framework.md#wave-13-fixes) for the full breakdown. Key invariants to remember when working on canary triggers: (1) `register_canary_trigger_type(node_type, cloudevent_type)` requires the CloudEvents reverse-DNS string as second arg — must match the producer's `WorkflowEvent.type` exactly or the deployment manager's `EventType` SA won't match `dispatch.emit`'s Visibility query (silent firing failure). Diverging re-registration raises `ValueError` so plugin upgrades surface loudly. (2) Plugin `_events.py` for canary-registered triggers is canary-only — no `event_waiter.dispatch` / `send_custom_event` calls (those have zero consumers in canary-on mode). `dispatch.emit(envelope, wire_routing_key=...)` handles BOTH Temporal Signal fan-out AND in-process WS broadcast. (3) `TriggerListenerWorkflow` + `PollingTriggerWorkflow` call `broadcast_trigger_status_activity` before/after each child spawn for firing-pulse UX (matches legacy `triggers.py` collector/processor). (4) `PollingTriggerNode.as_poll_activity` returns `seen_ids: list(current)` — NOT `list(prior_seen | current)` (the latter grows unboundedly; Gmail at ~100/day hit ~36K entries in a year). The legacy `_build_poll_coroutine` does `seen = set(current)` at end of cycle for the same reason. Visibility-filtered providers (Gmail-unread) re-emit on re-surface, which is correct. (5) Canary trigger output persistence: `MachinaWorkflow.run`'s pre-executed loop schedules `store_node_output_activity` for every firing trigger so `ParameterResolver` can resolve `{{triggerNode.field}}` in downstream nodes (the legacy `_execute_from_trigger` did this via `_store_output(trigger_node_id, "output_0", ...)`; canary skipped it pre-fix). Skips non-firing siblings (`_trigger_output={"not_triggered": True}`). (6) `DeploymentManager.cancel` sweeps stuck node statuses via `_clear_stuck_node_statuses(workflow_id, include_waiting=True)` + emits terminal `update_workflow_status(executing=False, workflow_id=...)`. Without these the FE leaves downstream nodes glowing forever after deployment cancel and the toolbar Start/Stop indicator stays at `executing=True`. The delegation guard inside `_clear_stuck_node_statuses` still protects in-flight fire-and-forget child agents.- never use emojis in prints\ No newline at end of filediff --git a/docs-internal/event_framework.md b/docs-internal/event_framework.mdindex 75b2a473..9b790566 100644--- a/docs-internal/event_framework.md+++ b/docs-internal/event_framework.md@@ -30,6 +30,7 @@ phase plan lives in `~/.claude/plans/properly-fix-the-tech-dreamy-tarjan.md`.| B11 — FE `plugin_connection_status` envelope handler | ✅ commit `899771c` || D4 — Drop legacy `*_status` raw frames (status-only round) | ✅ commit `5ea4e90` (whatsapp/android/telegram status retired; FE consumes typed channel) || Canary flag default flip + invariant lock | ✅ this commit (default `True`; `EVENT_FRAMEWORK_ENABLED=false` is the rollback) |+| Wave 13 — EventType SA mismatch fix + canary-only emit path + trigger status lifecycle + polling OOM + template resolution + cancel sweep | ✅ shipped (see "Wave 13 fixes" below) |### Dropped / Deferred / Pending@@ -201,6 +202,68 @@ This is why Wave 12 explicitly does NOT add a custom `event_dlq` SQLModel table.Wave 12 D3 (pending) adds thin WS handlers that wrap these Visibility queries for the FE admin surface, so operators can inspect failed runs without leaving the MachinaOs UI.+## Wave 13 fixes++Six load-bearing corrections to the Wave 12 canary path. All locked by regression tests.++### 1. `EventType` Search Attribute must equal the producer's `event.type`++Pre-fix the deployment manager registered the **legacy snake_case** `event_type` from `TriggerConfig` (e.g. `"chat_message_received"`) as the `EventType` SA on every canary listener. But `dispatch.emit` Visibility-queries by `event.type` from the CloudEvents envelope (reverse-DNS, e.g. `"com.machinaos.chat.message.received"`). The strings never matched — the listener started OK, never reacted to incoming events.++Fix:+- `register_canary_trigger_type(node_type, cloudevent_type)` now requires the CloudEvents type as a second arg ([canary_registry.py](../server/services/deployment/canary_registry.py)). Re-registering with a diverging `cloudevent_type` raises `ValueError` so plugin upgrades that change the envelope shape surface loudly.+- `cloudevent_type_for(node_type)` lookup, used by `DeploymentManager._start_canary_listener` for the `EventType` SA value (was `config.event_type`).+- `WorkflowEvent.task_completed` factory unified to single type `com.machinaos.agent.task.completed` (was `.succeeded` / `.failed` split — broke single-SA listener matching).++Locked by `TestCloudEventTypeMatchesSearchAttribute` in [`test_canary_registry.py`](../server/tests/test_canary_registry.py).++### 2. Plugin `_events.py` is canary-only — no more dual-emit++`event_waiter.dispatch` / `broadcaster.send_custom_event` calls inside canary-registered plugin `_events.py` files (chat / webhook / task / telegram / email) had zero consumers in canary-on mode — the deployment manager skips `setup_event_trigger` when `is_canary_trigger_type(...)` is True, so no legacy waiter ever registered. Removed.++- `dispatch.emit(envelope, wire_routing_key=...)` is now the single delivery path. It Signals running `TriggerListenerWorkflow` consumers via Temporal Visibility AND broadcasts to FE on the same wire key.+- `google/_events.py:dispatch_gmail_received` deleted (gmail polling uses `PollingTriggerWorkflow`, not `event_waiter`).+- `whatsapp/_events.py` kept the legacy raw frame on `whatsapp_message_received` for received messages because the FE message-list handler reads `data.sender` (legacy shape) directly — drop blocked on FE migration (D4 follow-up). The duplicate typed-envelope sibling on the same wire key was dropped.+- `twitter/_events.py` keeps both paths — twitter is the only deferred canary plugin (needs PollingTriggerNode subclass refactor first).++### 3. Trigger node status pulse on every firing++Pre-fix the canary listener broadcast `waiting` once at deploy and stayed there forever — when an event fired, FE saw downstream nodes light up but no visual signal on the trigger node itself. The legacy `services/deployment/triggers.py` collector/processor did `waiting → idle (Graph executing...) → waiting` per event; canary skipped this.++Fix: new `broadcast_trigger_status_activity` ([activities.py](../server/services/temporal/activities.py)). `TriggerListenerWorkflow._spawn_child_run` calls it before + after each child spawn:+- Before: `status="idle"` with `message="Graph executing..."` data+- After: `status="waiting"` with the next-event message++`PollingTriggerWorkflow._spawn_child_run` does the same. Matches legacy UX.++### 4. Polling `seen` set OOM leak++Both polling paths grew `seen: Set[str]` unboundedly:+- `as_poll_activity` returned `seen_ids: list(prior_seen | current)` every cycle — Temporal payload paid for the union forever.+- `_build_poll_coroutine` called `seen.add(msg_id)` per emit with no eviction.++At Gmail's ~100 msgs/day / 60s poll, this hit ~36K entries / ~1.4MB just for IDs in a year. Fix: rebase `seen = current` at end of every cycle in both paths ([polling.py](../server/services/plugin/polling.py)). Items the provider no longer reports drop out. Bounded by the provider's natural window size.++Semantic note: if a filtered item disappears then reappears (e.g. user marks an email unread again under `is:unread`), it re-emits. That's the correct semantic for visibility-filtered providers; the old unbounded set suppressed legitimate re-emits.++### 5. Template resolution against trigger output (canary path)++Pre-fix the canary path's pre-executed trigger output never reached the workflow output store. `MachinaWorkflow.run` set `outputs[node_id]` in workflow memory only — never persisted. `ParameterResolver._gather_connected_outputs` reads from the store, so `{{triggerNode.field}}` templates in downstream nodes resolved to empty.++The legacy `DeploymentManager._execute_from_trigger` called `_store_output(trigger_node_id, "output_0", trigger_output)` explicitly. Canary skipped it.++Fix: new `store_node_output_activity` ([activities.py](../server/services/temporal/activities.py)). `MachinaWorkflow.run`'s pre-executed loop schedules it for every firing trigger (skips non-firing siblings with `_trigger_output={"not_triggered": True}`). Writes to `output_main` + `output_top` + `output_0` so any downstream edge handle resolves.++### 6. `DeploymentManager.cancel` full status sweep++Pre-fix cancel reset only cron + listener trigger nodes to `idle`. Downstream agents/tools/actions that were mid-execute when cancel hit stayed in `executing` forever on FE. Toolbar Start/Stop indicator also stayed at `executing=True` because no terminal `executing=False` workflow_status broadcast was emitted.++Fix ([manager.py](../server/services/deployment/manager.py) `DeploymentManager.cancel`):+- `await broadcaster._clear_stuck_node_statuses(workflow_id, include_waiting=True)` — sweeps every node still broadcast as `executing`/`waiting`. The delegation guard inside `_clear_stuck_node_statuses` still protects in-flight fire-and-forget child agents.+- `await broadcaster.update_workflow_status(executing=False, workflow_id=...)` — terminal toolbar state. Avoids relying on `workflow_run_ended`'s counter (can race against in-flight `workflow_run_started` callers).++Locked by `TestCancelSweepsStuckNodeStatuses` in [`test_deployment_canary_listener.py`](../server/tests/test_deployment_canary_listener.py).+## References- Plan: `~/.claude/plans/properly-fix-the-tech-dreamy-tarjan.md`diff --git a/server/nodes/agent/_events.py b/server/nodes/agent/_events.pyindex 87e2ad40..a7166436 100644--- a/server/nodes/agent/_events.py+++ b/server/nodes/agent/_events.py@@ -1,34 +1,33 @@-"""Wave 12 B8: CloudEvents broadcaster wrappers for agent task delegation.+"""CloudEvents broadcaster wrappers for agent task delegation.Plugin-folder-owned wrappers around the cross-cutting-``WorkflowEvent.task_completed`` factory (cross-cutting factory itself-stays in ``services/events/envelope.py`` per RFC §6.4 — it's-parametrised by ``task_id`` + ``agent`` + ``status`` and uniformly-applies to every delegated agent task, not single-plugin shape).--What lives here vs the central factory:- - Central ``WorkflowEvent.task_completed(task_id, status, agent, data)``- factory STAYS in services/events/envelope.py (cross-cutting).- - The broadcaster *wrappers* below live with the agent plugin folder- so the 3 callsites in ``services/handlers/tools.py`` (the- fire-and-forget delegation dispatcher) import from- ``nodes.agent._events`` rather than building the wire frame- inline.--Wire-routing key ``"task_completed"`` is preserved — the-``taskTrigger`` plugin filter still subscribes on this string. The-inner envelope carries the spec-compliant ``com.machinaos.agent.task.{-succeeded, failed}`` type.+``WorkflowEvent.task_completed`` factory. The factory itself stays in+``services/events/envelope.py`` per RFC §6.4 — it's parametrised by+``task_id`` + ``agent`` + ``status`` and uniformly applies to every+delegated agent task, not a single-plugin shape. The broadcaster+wrappers below live with the agent plugin folder so the 3 callsites in+``services/handlers/tools.py`` (the fire-and-forget delegation+dispatcher) import from ``nodes.agent._events`` rather than building+the wire frame inline.++Delivery: a single ``dispatch.emit`` call. ``dispatch.emit`` Signals+running :class:`TriggerListenerWorkflow` consumers via Temporal+Visibility AND broadcasts the same envelope to in-process WS clients on+the ``task_completed`` wire-routing key — one canonical path. Pre-Wave-13+this also called ``broadcaster.send_custom_event`` for the legacy+collector path; that's dead since taskTrigger is canary-registered and+the deployment manager skips ``setup_event_trigger`` for canary types."""from __future__ import annotations-from typing import Any, Mapping, Optional+from typing import Any, Optional-# Legacy wire-routing key. The ``taskTrigger`` plugin filters by this-# string today (see ``services/event_waiter.py:TRIGGER_REGISTRY``).-_LEGACY_EVENT_TYPE = "task_completed"+# Outer wire-routing key. FE consumers (if any) and the legacy+# back-compat receipt path still switch on this string; the inner+# envelope carries ``com.machinaos.agent.task.completed``.+_WIRE_ROUTING_KEY = "task_completed"async def broadcast_agent_task_completed(@@ -40,12 +39,7 @@ async def broadcast_agent_task_completed(workflow_id: Optional[str] = None,result: Optional[str] = None,) -> None:- """Broadcast a delegated agent task completing successfully.-- Replaces inline ``broadcaster.send_custom_event('task_completed',- {...})`` calls in ``services/handlers/tools.py``. Builds the- payload exactly as the consumer (taskTrigger filter) expects.- """+ """Broadcast a delegated agent task completing successfully."""await _broadcast_task_event(task_id=task_id,status="completed",@@ -66,12 +60,7 @@ async def broadcast_agent_task_failed(workflow_id: Optional[str] = None,error: str,) -> None:- """Broadcast a delegated agent task failing.-- Replaces inline ``broadcaster.send_custom_event('task_completed',- {status: 'error', ...})`` calls in- ``services/handlers/tools.py``.- """+ """Broadcast a delegated agent task failing."""await _broadcast_task_event(task_id=task_id,status="error",@@ -94,24 +83,18 @@ async def _broadcast_task_event(result: Optional[str] = None,error: Optional[str] = None,) -> None:- """Internal helper. Two delivery paths (in priority order):-- 1. **Legacy WS broadcast + asyncio.Future event_waiter** via- ``broadcaster.send_custom_event`` — the in-process collector/- processor path; default while the canary flag is off.- 2. **Temporal-durable listeners** via- ``services.events.dispatch.emit`` (Wave 12 C1 rollout #2). The- envelope reuses the cross-cutting :meth:`WorkflowEvent.task_completed`- factory (parametrised by task_id + agent + status — RFC §6.4- cross-cutting tier). ``emit`` is a no-op when the feature flag- is off, so the legacy path keeps working unchanged.-- Payload shape preserved exactly so the taskTrigger filter and the- legacy FE consumers don't see any wire change.+ """Single delivery path: build the envelope, route through+ :func:`services.events.dispatch.emit`.++ ``emit`` does two things in one call: (a) Visibility query for+ running :class:`TriggerListenerWorkflow` consumers tagged with+ ``EventType='com.machinaos.agent.task.completed'`` and signals each;+ (b) direct in-process WS broadcast on the ``task_completed`` wire+ key so any FE consumers receive the envelope. Payload shape in+ ``data`` is exactly what the taskTrigger filter reads."""from services.events.dispatch import emitfrom services.events.envelope import WorkflowEvent- from services.status_broadcaster import get_status_broadcasterpayload: dict[str, Any] = {"task_id": task_id,@@ -126,19 +109,13 @@ async def _broadcast_task_event(elif status == "error" and error is not None:payload["error"] = error- broadcaster = get_status_broadcaster()- await broadcaster.send_custom_event(_LEGACY_EVENT_TYPE, payload)-- # Temporal-durable fan-out (Wave 12 C1 rollout #2). Cross-cutting- # factory because task_completed is parametrised by (task_id,- # agent, status) and uniformly applies across every agent type.envelope = WorkflowEvent.task_completed(task_id=task_id,status="completed" if status == "completed" else "error",agent=agent_name,data=payload,)- await emit(envelope, wire_routing_key=_LEGACY_EVENT_TYPE)+ await emit(envelope, wire_routing_key=_WIRE_ROUTING_KEY)__all__ = [diff --git a/server/nodes/email/__init__.py b/server/nodes/email/__init__.pyindex 6808e157..0fa40dc2 100644--- a/server/nodes/email/__init__.py+++ b/server/nodes/email/__init__.py@@ -15,4 +15,4 @@ from ._events import dispatch_email_received # noqa: F401 — re-exportfrom ._filters import build_filter as build_email_filterregister_filter_builder("emailReceive", build_email_filter)-register_canary_trigger_type("emailReceive")+register_canary_trigger_type("emailReceive", "com.machinaos.email.message.received")diff --git a/server/nodes/email/_events.py b/server/nodes/email/_events.pyindex a2864a47..3c0cf612 100644--- a/server/nodes/email/_events.py+++ b/server/nodes/email/_events.py@@ -1,22 +1,14 @@-"""Wave 12 B4 + canary opt-in: CloudEvents factories + dispatcher for email.--Plugin-specific event emission — replaces:- - ``event_waiter.dispatch("email_received", email_data)`` in- ``email_receive/__init__.py``+"""CloudEvents factory + dispatcher for email.Per RFC plugin_authoring_rfc.md §6.4: plugin-specific factories live inthe plugin folder.-Legacy ``event_type`` (``"email_received"``) is preserved on the-dispatch path so the ``emailReceive`` trigger node's ``event_type``-ClassVar still matches without a coordinated registry-side rename.--Dual-dispatch (canary opt-in, post-2026-05-15):- - Legacy ``event_waiter.dispatch`` for the in-process collector/processor- (still production-live for non-canary triggers).- - Typed ``services.events.dispatch.emit`` for Temporal-durable- ``TriggerListenerWorkflow`` consumers (when canary registry +- ``event_framework_enabled`` are both on — both default-true).+Delivery: single ``dispatch.emit`` call routes events to running+:class:`TriggerListenerWorkflow` consumers via Temporal Visibility AND+broadcasts the envelope to FE on the ``email_received`` wire key.+emailReceive is canary-registered (see ``nodes/email/__init__.py``) so+the deployment manager skips ``setup_event_trigger`` and the legacy+``event_waiter.dispatch`` path has zero consumers — removed."""from __future__ import annotations@@ -26,12 +18,10 @@ from typing import Any, Mappingfrom services.events.envelope import WorkflowEvent-# Legacy event_type the event_waiter dispatches by; trigger nodes-# subscribe on this string (matches ``EmailReceiveNode.event_type``).-_LEGACY_EVENT_TYPE = "email_received"---# ---- Typed factory ---------------------------------------------------------+# Outer wire-routing key. Matches ``EmailReceiveNode.event_type`` and+# the FE WS channel; the inner envelope carries+# ``com.machinaos.email.message.received``.+_WIRE_ROUTING_KEY = "email_received"def email_message_received(email_data: Mapping[str, Any]) -> WorkflowEvent:@@ -48,42 +38,15 @@ def email_message_received(email_data: Mapping[str, Any]) -> WorkflowEvent:)-# ---- Dispatcher wrapper -------------------------------------------------------async def dispatch_email_received(email_data: Mapping[str, Any]) -> int:- """Dispatch an incoming email to waiting ``emailReceive`` trigger- nodes.-- Two delivery paths (mirrors :func:`telegram._events.dispatch_telegram_message_received`):-- 1. **Legacy event_waiter waiters** via :func:`event_waiter.dispatch`- (in-process collector/processor; still default for trigger nodes- outside the canary registry).- 2. **Temporal-durable listeners** via- :func:`services.events.dispatch.emit`. ``emit`` is a pass-through- no-op when ``event_framework_enabled`` is off; otherwise it- Visibility-queries running ``TriggerListenerWorkflow`` instances- and signals each.-- Returns the count of legacy waiters resolved.- """- from services import event_waiter+async def dispatch_email_received(email_data: Mapping[str, Any]) -> None:+ """Dispatch an incoming email via the canary CloudEvents path."""from services.events.dispatch import emit- payload = dict(email_data)- resolved = event_waiter.dispatch(_LEGACY_EVENT_TYPE, payload)-- # Temporal-durable fan-out (canary opt-in via- # ``register_canary_trigger_type("emailReceive")`` in- # ``nodes/email/__init__.py``).await emit(- email_message_received(payload),- wire_routing_key=_LEGACY_EVENT_TYPE,+ email_message_received(dict(email_data)),+ wire_routing_key=_WIRE_ROUTING_KEY,)- return resolved-__all__ = ["dispatch_email_received",diff --git a/server/nodes/google/__init__.py b/server/nodes/google/__init__.pyindex 32a95ddd..11155443 100644--- a/server/nodes/google/__init__.py+++ b/server/nodes/google/__init__.py@@ -35,7 +35,6 @@ from services.ws_handler_registry import ()from . import _router-from ._events import dispatch_gmail_received # noqa: F401 — re-exportfrom ._filters import build_gmail_filterfrom ._handlers import WS_HANDLERSfrom ._option_loaders import (@@ -65,4 +64,4 @@ register_option_loader("googleTasklists", load_tasklists)# Temporal activity is emitted by GmailReceiveNode.as_poll_activity()# and registered via collect_polling_activities() in the worker. See# services/deployment/canary_registry.py.-register_canary_trigger_type("googleGmailReceive")+register_canary_trigger_type("googleGmailReceive", "com.machinaos.gmail.message.received")diff --git a/server/nodes/google/_events.py b/server/nodes/google/_events.pyindex b04882db..73a575c1 100644--- a/server/nodes/google/_events.py+++ b/server/nodes/google/_events.py@@ -1,20 +1,24 @@-"""Wave 12 B5: CloudEvents factories + broadcaster wrappers for google.+"""CloudEvents factory for gmail.-Plugin-specific event emission — replaces:- - ``event_waiter.dispatch("gmail_email_received", email_data)`` in- ``gmail_receive/__init__.py``.+Per RFC plugin_authoring_rfc.md §6.4: plugin-specific factories live in+the plugin folder.++No dispatcher wrapper: the canary delivery path for ``googleGmailReceive``+is :class:`services.temporal.polling_trigger_workflow.PollingTriggerWorkflow`+(not Signal-based). The polling activity (built by+:meth:`services.plugin.polling.PollingTriggerNode.as_poll_activity`)+returns the raw email dict back to the workflow which spawns the child+MachinaWorkflow directly — no envelope-on-the-wire step is needed.++The factory below is kept for parity with the other plugin ``_events.py``+modules (and for any future ad-hoc emit usage); the legacy+``dispatch_gmail_received`` shim was deleted in Wave 13 because it+dispatched to ``event_waiter`` waiters that were never registered in+canary-on mode (the polling workflow owns delivery).OAuth completion broadcasts route through the cross-cutting``StatusBroadcaster.broadcast_credential_event("credential.oauth.connected", ...)``path (RFC §6.4 cross-cutting tier) — NOT a plugin-named broadcast.-No migration needed here for that flow.--Per RFC plugin_authoring_rfc.md §6.4: plugin-specific factories live-in the plugin folder.--Legacy ``event_type`` (``"gmail_email_received"``) is preserved so-the ``gmailReceive`` trigger node's ``event_type`` ClassVar still-matches without a coordinated registry-side rename."""from __future__ import annotations@@ -24,14 +28,6 @@ from typing import Any, Mappingfrom services.events.envelope import WorkflowEvent-# Legacy event_type the event_waiter dispatches by; trigger nodes-# subscribe on this string (matches ``GmailReceiveNode.event_type``).-_LEGACY_EVENT_TYPE = "gmail_email_received"---# ---- Typed factory -----------------------------------------------------------def gmail_message_received(email_data: Mapping[str, Any]) -> WorkflowEvent:"""Incoming Gmail message envelope. ``subject`` is the Gmailmessage_id (correlation key for the consumer-side ``markAsRead``@@ -46,18 +42,6 @@ def gmail_message_received(email_data: Mapping[str, Any]) -> WorkflowEvent:)-# ---- Dispatcher wrapper -------------------------------------------------------def dispatch_gmail_received(email_data: Mapping[str, Any]) -> int:- """Dispatch an incoming Gmail message to waiting ``gmailReceive``- trigger nodes. Returns the count of resolved waiters."""- from services import event_waiter-- return event_waiter.dispatch(_LEGACY_EVENT_TYPE, dict(email_data))--__all__ = [- "dispatch_gmail_received","gmail_message_received",]diff --git a/server/nodes/google/gmail_receive/__init__.py b/server/nodes/google/gmail_receive/__init__.pyindex 7a697b33..1356df31 100644--- a/server/nodes/google/gmail_receive/__init__.py+++ b/server/nodes/google/gmail_receive/__init__.py@@ -169,10 +169,11 @@ class GmailReceiveNode(PollingTriggerNode):"gmail", node_id, "receive", 1,{"workflow_id": context.workflow_id, "session_id": context.session_id},)- # Wave 12 B5: route through plugin _events.py wrapper.- from nodes.google import dispatch_gmail_received-- dispatch_gmail_received(email_data)+ # Inline Run path: the node returns ``email_data`` as+ # its output and the activity wrapper broadcasts+ # ``success`` status to FE. No separate dispatch+ # needed — deployment mode uses PollingTriggerWorkflow+ # which owns event fan-out in-workflow.logger.info(f"[GmailReceive] New email: {email_data.get('subject', 'no subject')}",)diff --git a/server/nodes/scheduler/cron_scheduler/__init__.py b/server/nodes/scheduler/cron_scheduler/__init__.pyindex fb430d54..42e380fa 100644--- a/server/nodes/scheduler/cron_scheduler/__init__.py+++ b/server/nodes/scheduler/cron_scheduler/__init__.py@@ -42,7 +42,7 @@ logger = get_logger(__name__)register_temporal_plugin(SimplePlugin(name="cron-scheduler", workflows=[CronTriggerWorkflow]))-register_canary_trigger_type("cronScheduler")+register_canary_trigger_type("cronScheduler", "com.machinaos.cron.tick")def _calculate_wait_seconds(p: Dict[str, Any]) -> int:diff --git a/server/nodes/telegram/__init__.py b/server/nodes/telegram/__init__.pyindex 39405e00..50929ce1 100644--- a/server/nodes/telegram/__init__.py+++ b/server/nodes/telegram/__init__.py@@ -65,7 +65,7 @@ register_output_schema("telegramSend", TelegramSendOutput)# unconditionally; emit() is gated by Settings.event_framework_enabled# so the legacy path stays default. See# services/deployment/canary_registry.py.-register_canary_trigger_type(TelegramReceiveNode.type)+register_canary_trigger_type(TelegramReceiveNode.type, "com.machinaos.telegram.message.received")__all__ = ["TelegramCredential",diff --git a/server/nodes/telegram/_events.py b/server/nodes/telegram/_events.pyindex 35b2b81f..e539437c 100644--- a/server/nodes/telegram/_events.py+++ b/server/nodes/telegram/_events.py@@ -144,39 +144,22 @@ async def broadcast_telegram_status(})-async def dispatch_telegram_message_received(event_data: Mapping[str, Any]) -> int:- """Dispatch an incoming Telegram message to waiting- ``telegramReceive`` trigger nodes.-- Two delivery paths (in priority order):-- 1. **Legacy event_waiter waiters** via :func:`event_waiter.dispatch`- (in-process collector/processor; default while the canary flag- is off).- 2. **Temporal-durable listeners** via- :func:`services.events.dispatch.emit` (Wave 12 C1 rollout #3).- ``emit`` is a no-op pass-through when the feature flag is off,- so the legacy path keeps working unchanged.-- Returns the count of legacy waiters resolved.+async def dispatch_telegram_message_received(event_data: Mapping[str, Any]) -> None:+ """Dispatch an incoming Telegram message via the canary CloudEvents path.++ Single delivery: :func:`services.events.dispatch.emit` Signals running+ :class:`TriggerListenerWorkflow` consumers via Temporal Visibility AND+ broadcasts the envelope to FE on the ``telegram_message_received``+ wire key. telegramReceive is canary-registered so no legacy+ ``event_waiter`` waiter is ever registered for it."""- from services import event_waiterfrom services.events.dispatch import emit- payload = dict(event_data)- # Trigger nodes subscribe by event_type string; keep the legacy- # string so the `telegramReceive.event_type` ClassVar still matches.- resolved = event_waiter.dispatch(_MESSAGE_LEGACY_EVENT_TYPE, payload)-- # Temporal-durable fan-out (Wave 12 C1 rollout #3). No-op when- # event_framework_enabled is off.await emit(- telegram_message_received(payload),+ telegram_message_received(dict(event_data)),wire_routing_key=_MESSAGE_LEGACY_EVENT_TYPE,)- return resolved-__all__ = ["broadcast_telegram_status",diff --git a/server/nodes/telegram/_service.py b/server/nodes/telegram/_service.pyindex 0a9d12fd..1e0b8ff6 100644--- a/server/nodes/telegram/_service.py+++ b/server/nodes/telegram/_service.py@@ -493,12 +493,11 @@ class TelegramService(ServiceSingleton):f"chat_type={event_data.get('chat_type')}")- # Wave 12 B3: route via plugin _events.py wrapper (single- # source of truth for the wire format).+ # Route via plugin _events.py wrapper — canary CloudEvents+ # path (Visibility-query Signal + WS broadcast).from . import dispatch_telegram_message_received- resolved = await dispatch_telegram_message_received(event_data)- logger.info(f"[Telegram] Dispatched to {resolved} waiter(s)")+ await dispatch_telegram_message_received(event_data)except Exception as e:logger.error(f"[Telegram] Message handler error: {e}")diff --git a/server/nodes/tool/agent_builder/_events.py b/server/nodes/tool/agent_builder/_events.pyindex 5e6400cc..94cd3049 100644--- a/server/nodes/tool/agent_builder/_events.py+++ b/server/nodes/tool/agent_builder/_events.py@@ -1,18 +1,19 @@-"""Wave 12 B10: CloudEvents factory + broadcaster wrapper for agent_builder.--Plugin-specific event emission — replaces the inline-``broadcaster.send_custom_event("workflow_ops_apply", {...})`` call in-``agent_builder/__init__.py:_broadcast``.--The agent_builder canvas-mutation broadcast carries a flat shape-(``{workflow_id, caller_node_id, operations}``) consumed by the-frontend's ``useWorkflowOpsListener`` hook directly (NOT via the-typed-envelope path). Per the existing X4-allowlist rationale, the-WIRE format stays flat for FE back-compat; the typed envelope is-constructed for audit/future use only.--Per RFC plugin_authoring_rfc.md §6.4: plugin-specific factories live-in the plugin folder.+"""CloudEvents factory + broadcaster wrapper for agent_builder.++Per RFC plugin_authoring_rfc.md §6.4: plugin-specific factories live in+the plugin folder.++This is NOT a trigger event — it's a canvas-mutation broadcast consumed+by the frontend's ``useWorkflowOpsListener`` hook. The wire format is+flat (``{workflow_id, caller_node_id, operations}``) for FE back-compat;+the typed envelope is constructed for parity with the other plugin+``_events.py`` modules but not consumed today.++Delivery: direct WS broadcast (no ``dispatch.emit`` — no+:class:`TriggerListenerWorkflow` consumer exists for canvas mutations,+and no legacy ``event_waiter`` waiter listens on ``workflow_ops_apply``+either). Pre-Wave-13 this routed through ``send_custom_event`` which+also called ``event_waiter.dispatch_async`` — that dispatch was dead."""from __future__ import annotations@@ -22,11 +23,8 @@ from typing import Any, Dict, List, Optionalfrom services.events.envelope import WorkflowEvent-# Legacy wire-routing key consumed by ``client/src/hooks/useWorkflowOpsListener``.-_LEGACY_EVENT_TYPE = "workflow_ops_apply"---# ---- Typed factory ---------------------------------------------------------+# Flat wire-routing key consumed by ``client/src/hooks/useWorkflowOpsListener``.+_WIRE_ROUTING_KEY = "workflow_ops_apply"def workflow_ops_applied(@@ -50,18 +48,17 @@ def workflow_ops_applied()-# ---- Broadcaster wrapper -----------------------------------------------------async def broadcast_workflow_ops(*,workflow_id: Optional[str],caller_node_id: str,operations: List[Dict[str, Any]],) -> None:- """Emit a workflow_ops_apply event. Routes through- ``broadcaster.send_custom_event`` (raw flat payload for FE- back-compat; envelope constructed for future audit use).+ """Broadcast a canvas-mutation event for the FE to apply.++ Flat payload (FE back-compat with ``useWorkflowOpsListener``); the+ typed envelope is constructed for audit/future migration but not+ on the wire today."""if not operations:return@@ -73,14 +70,17 @@ async def broadcast_workflow_ops("caller_node_id": caller_node_id,"operations": list(operations),}- # Typed envelope constructed for future audit; not on the wire today- # because FE listener consumes the flat payload via send_custom_event.+ # Envelope constructed for parity / future migration; not on the+ # wire today because FE listener reads the flat shape._ = workflow_ops_applied(workflow_id=workflow_id,caller_node_id=caller_node_id,operations=operations,)- await broadcaster.send_custom_event(_LEGACY_EVENT_TYPE, payload)+ await broadcaster.broadcast({+ "type": _WIRE_ROUTING_KEY,+ "data": payload,+ })__all__ = [diff --git a/server/nodes/trigger/chat_trigger/__init__.py b/server/nodes/trigger/chat_trigger/__init__.pyindex 0d7ed808..9058e295 100644--- a/server/nodes/trigger/chat_trigger/__init__.py+++ b/server/nodes/trigger/chat_trigger/__init__.py@@ -77,4 +77,4 @@ class ChatTriggerNode(TriggerNode):# ./_events.py calls dispatch.emit unconditionally; the canary registry# tells DeploymentManager to start a listener for this type. See# services/deployment/canary_registry.py.-register_canary_trigger_type(ChatTriggerNode.type)+register_canary_trigger_type(ChatTriggerNode.type, "com.machinaos.chat.message.received")diff --git a/server/nodes/trigger/chat_trigger/_events.py b/server/nodes/trigger/chat_trigger/_events.pyindex ed5c17da..066dce3a 100644--- a/server/nodes/trigger/chat_trigger/_events.py+++ b/server/nodes/trigger/chat_trigger/_events.py@@ -1,19 +1,14 @@-"""Wave 12 B7: CloudEvents factory + dispatch wrapper for chat_trigger.+"""CloudEvents factory + dispatch wrapper for chat_trigger.-Plugin-specific event emission — replaces:- - ``event_waiter.dispatch("chat_message_received", event_data)`` in- ``routers/websocket.py:handle_send_chat_message``.+Per RFC plugin_authoring_rfc.md §6.4: plugin-specific factories live in+the plugin folder.-The Chat console message-send handler in the WS router previously-dispatched the event itself, hardcoding the plugin-specific event_type-string in framework code. Phase B7 inverts this: the chat_trigger-plugin owns its own dispatch primitive, the WS handler imports it.--Per RFC plugin_authoring_rfc.md §6.4: plugin-specific factories live-in the plugin folder.--Legacy ``event_type`` (``"chat_message_received"``) is preserved so-the ``ChatTriggerNode.event_type`` ClassVar still matches.+Delivery: single ``dispatch.emit`` call routes events to running+:class:`TriggerListenerWorkflow` consumers via Temporal Visibility AND+broadcasts the envelope to FE on the ``chat_message_received`` wire key.+chatTrigger is canary-registered (see ``nodes/trigger/chat_trigger/__init__.py``)+so the deployment manager skips ``setup_event_trigger`` and the legacy+``event_waiter.dispatch`` path has zero consumers — removed."""from __future__ import annotations@@ -23,12 +18,10 @@ from typing import Any, Mappingfrom services.events.envelope import WorkflowEvent-# Legacy event_type the event_waiter dispatches by; trigger nodes-# subscribe on this string (matches ``ChatTriggerNode.event_type``).-_LEGACY_EVENT_TYPE = "chat_message_received"---# ---- Typed factory ---------------------------------------------------------+# Outer wire-routing key. Matches ``ChatTriggerNode.event_type`` and the+# FE WS channel; the inner envelope carries+# ``com.machinaos.chat.message.received``.+_WIRE_ROUTING_KEY = "chat_message_received"def chat_message_received(event_data: Mapping[str, Any]) -> WorkflowEvent:@@ -44,37 +37,14 @@ def chat_message_received(event_data: Mapping[str, Any]) -> WorkflowEvent:)-# ---- Dispatcher wrapper -------------------------------------------------------async def dispatch_chat_message_received(event_data: Mapping[str, Any]) -> int:- """Dispatch an incoming chat message to waiting ``chatTrigger`` nodes.-- Two delivery paths (in priority order):-- 1. **Legacy event_waiter waiters** via :func:`event_waiter.dispatch`- (in-process collector/processor; default while- ``Settings.event_framework_enabled`` is off).- 2. **Temporal-durable listeners** via :func:`services.events.dispatch.emit`- (Wave 12 C1 canary). ``emit`` is a no-op pass-through when the- feature flag is off, so the legacy path keeps working unchanged.-- Returns the count of legacy waiters resolved (Temporal listeners- are signalled asynchronously through Visibility queries and don't- show up in this count — that's by design, the chat panel UI only- cares about the legacy resolved-count for its "delivered" badge).- """- from services import event_waiter+async def dispatch_chat_message_received(event_data: Mapping[str, Any]) -> None:+ """Dispatch an incoming chat message via the canary CloudEvents path."""from services.events.dispatch import emit- payload = dict(event_data)- resolved = event_waiter.dispatch(_LEGACY_EVENT_TYPE, payload)-- # Temporal-durable fan-out (Wave 12 C1 canary). Always-call;- # dispatch.emit is a no-op when the feature flag is off.- await emit(chat_message_received(payload), wire_routing_key=_LEGACY_EVENT_TYPE)-- return resolved+ await emit(+ chat_message_received(dict(event_data)),+ wire_routing_key=_WIRE_ROUTING_KEY,+ )__all__ = [diff --git a/server/nodes/trigger/task_trigger/__init__.py b/server/nodes/trigger/task_trigger/__init__.pyindex 579cd39e..4a40f5ca 100644--- a/server/nodes/trigger/task_trigger/__init__.py+++ b/server/nodes/trigger/task_trigger/__init__.py@@ -85,4 +85,4 @@ class TaskTriggerNode(TriggerNode):# consumer path. The producer side lives in nodes/agent/_events.py# (broadcast_agent_task_completed / _failed dual-emit via dispatch.emit).# See services/deployment/canary_registry.py for the contract.-register_canary_trigger_type(TaskTriggerNode.type)+register_canary_trigger_type(TaskTriggerNode.type, "com.machinaos.agent.task.completed")diff --git a/server/nodes/trigger/webhook_trigger/__init__.py b/server/nodes/trigger/webhook_trigger/__init__.pyindex 7c512622..7365a609 100644--- a/server/nodes/trigger/webhook_trigger/__init__.py+++ b/server/nodes/trigger/webhook_trigger/__init__.py@@ -115,4 +115,4 @@ class WebhookTriggerNode(TriggerNode):# Gated runtime by Settings.event_framework_enabled in DeploymentManager;# the producer side (broadcast_webhook_received calling dispatch.emit) is# unconditional. See services/deployment/canary_registry.py.-register_canary_trigger_type(WebhookTriggerNode.type)+register_canary_trigger_type(WebhookTriggerNode.type, "com.machinaos.webhook.received")diff --git a/server/nodes/trigger/webhook_trigger/_events.py b/server/nodes/trigger/webhook_trigger/_events.pyindex 7b16a531..e60192dd 100644--- a/server/nodes/trigger/webhook_trigger/_events.py+++ b/server/nodes/trigger/webhook_trigger/_events.py@@ -1,20 +1,15 @@-"""Wave 12 B9: CloudEvents factory + dispatch wrapper for webhook_trigger.--Plugin-specific event emission — replaces:- - ``broadcaster.send_custom_event("webhook_received", webhook_data)``- in ``routers/webhook.py`` (unmapped-path fallback).--The webhook router previously emitted the event itself, hardcoding the-plugin-specific event_type string in framework code. Phase B9 inverts-this: the webhook_trigger plugin owns its dispatch primitive.--Per RFC plugin_authoring_rfc.md §6.4: plugin-specific factories live-in the plugin folder.--Wire-key behaviour preserved: still goes through-``broadcaster.send_custom_event(...)`` which (a) does the legacy raw-WS broadcast AND (b) dispatches the event into ``event_waiter`` so-``webhookTrigger`` nodes match it.+"""CloudEvents factory + broadcaster wrapper for webhook_trigger.++Per RFC plugin_authoring_rfc.md §6.4: plugin-specific factories live in+the plugin folder.++Delivery: single ``dispatch.emit`` call routes events to running+:class:`TriggerListenerWorkflow` consumers via Temporal Visibility AND+broadcasts the envelope to FE on the ``webhook_received`` wire key.+webhookTrigger is canary-registered (see ``nodes/trigger/webhook_trigger/__init__.py``)+so the deployment manager skips ``setup_event_trigger`` and the legacy+``broadcaster.send_custom_event`` path (which dispatched to+``event_waiter``) has zero consumers — removed."""from __future__ import annotations@@ -24,13 +19,10 @@ from typing import Any, Mappingfrom services.events.envelope import WorkflowEvent-# Legacy wire-routing key. The frontend has no listener for it today-# (webhook receivers don't have a status panel); the consumer is the-# ``webhookTrigger`` node via the event_waiter dispatch.-_LEGACY_EVENT_TYPE = "webhook_received"---# ---- Typed factory ---------------------------------------------------------+# Outer wire-routing key. Matches ``WebhookTriggerNode.event_type`` and+# the FE WS channel; the inner envelope carries+# ``com.machinaos.webhook.received``.+_WIRE_ROUTING_KEY = "webhook_received"def webhook_received(webhook_data: Mapping[str, Any]) -> WorkflowEvent:@@ -46,42 +38,14 @@ def webhook_received(webhook_data: Mapping[str, Any]) -> WorkflowEvent:)-# ---- Broadcaster wrapper -----------------------------------------------------async def broadcast_webhook_received(webhook_data: Mapping[str, Any]) -> None:- """Broadcast an incoming webhook to ``webhookTrigger`` nodes.-- Three delivery paths (kept in this order for back-compat):-- 1. **Legacy WS broadcast + asyncio.Future waiters** via- ``broadcaster.send_custom_event`` (the in-process collector /- processor path; default while the canary flag is off).- 2. **Temporal Signal fan-out** via ``services.events.dispatch.emit``- (Wave 12 C1 canary). Gated by ``Settings.event_framework_enabled``;- inside :func:`emit` the flag-off branch is a no-op pass-through- so it's cheap to always invoke. When on, the dispatch helper- runs a Visibility query and signals every running- :class:`TriggerListenerWorkflow` whose ``EventType`` matches- the envelope.-- Both paths are non-overlapping consumers — legacy reaches the- in-process collector/processor task pair; the Temporal path- reaches the durable listener workflow. Producer doesn't need to- pick one.- """- from services.status_broadcaster import get_status_broadcaster-- broadcaster = get_status_broadcaster()- envelope = webhook_received(webhook_data)- await broadcaster.send_custom_event(_LEGACY_EVENT_TYPE, dict(webhook_data))-- # Temporal-durable fan-out (Wave 12 C1 canary). Always-call;- # dispatch.emit is a no-op when the feature flag is off, so the- # legacy path keeps working unchanged.+ """Broadcast an incoming webhook via the canary CloudEvents path."""from services.events.dispatch import emit- await emit(envelope, wire_routing_key=_LEGACY_EVENT_TYPE)+ await emit(+ webhook_received(dict(webhook_data)),+ wire_routing_key=_WIRE_ROUTING_KEY,+ )__all__ = [diff --git a/server/nodes/whatsapp/__init__.py b/server/nodes/whatsapp/__init__.pyindex 84a37324..c27ca2ae 100644--- a/server/nodes/whatsapp/__init__.py+++ b/server/nodes/whatsapp/__init__.py@@ -74,7 +74,7 @@ register_option_loader("whatsappGroupMembers", load_group_members)# fans out via services.events.dispatch.emit; emit() is gated by# Settings.event_framework_enabled so the legacy path stays default.# See services/deployment/canary_registry.py.-register_canary_trigger_type("whatsappReceive")+register_canary_trigger_type("whatsappReceive", "com.machinaos.whatsapp.message.received")# Wave 12 C4 sub-piece A: opt this plugin into the social-provider# registry so the social node dispatches by platform identifierdiff --git a/server/nodes/whatsapp/_events.py b/server/nodes/whatsapp/_events.pyindex 5adc0347..1dd78d52 100644--- a/server/nodes/whatsapp/_events.py+++ b/server/nodes/whatsapp/_events.py@@ -188,87 +188,80 @@ async def broadcast_whatsapp_message(direction: Literal["sent", "received"],params: Mapping[str, Any],) -> None:- """Emit a message event (sent or received). Wire-routing key- preserves the legacy ``whatsapp_message_<direction>`` channel.-- For inbound (``received``) messages, also fans out via- :func:`services.events.dispatch.emit` so Wave 12 C1 canary- :class:`TriggerListenerWorkflow` consumers can pick up the- envelope (no-op when ``Settings.event_framework_enabled`` is off,- so the legacy WS path stays default).+ """Emit a message event (sent or received).++ Inbound (``received``) routes via :func:`services.events.dispatch.emit`+ — the canary CloudEvents path Signals running+ :class:`TriggerListenerWorkflow` consumers AND broadcasts the+ envelope on the ``whatsapp_message_received`` wire key.++ Outbound (``sent``) is observation-only: no trigger consumer+ exists, so it bypasses ``dispatch.emit`` and goes out as a raw+ legacy frame on ``whatsapp_message_sent`` for the FE message-list+ UI."""from services.status_broadcaster import get_status_broadcasterbroadcaster = get_status_broadcaster()- wire_key = (- _MESSAGE_SENT_WIRE_KEY if direction == "sent"- else _MESSAGE_RECEIVED_WIRE_KEY- )payload = dict(params)- # Legacy raw frame.- await broadcaster.broadcast({- "type": wire_key,- "data": payload,- })-- # Typed CloudEvents sibling — same outer wire key (the inner- # envelope is what carries the typed contract).- event = whatsapp_message_event(direction, payload)- await broadcaster.broadcast({- "type": wire_key,- "data": event.model_dump(mode="json"),- })-- # Wave 12 C1 rollout #4: Temporal-durable canary fan-out (received- # only; outbound message events are pure observation, no trigger- # node consumes them). emit() no-ops when the feature flag is off.if direction == "received":+ # Canary path — single ``emit`` call delivers to Temporal+ # listeners + FE WS. The FE message-list handler at+ # WebSocketContext.tsx still reads ``data.*`` for back-compat,+ # so we also broadcast the legacy raw frame until that handler+ # migrates to envelope-shape (Wave 12 D4 follow-up).from services.events.dispatch import emit- await emit(event, wire_routing_key=wire_key)+ await broadcaster.broadcast({+ "type": _MESSAGE_RECEIVED_WIRE_KEY,+ "data": payload,+ })+ await emit(+ whatsapp_message_event("received", payload),+ wire_routing_key=_MESSAGE_RECEIVED_WIRE_KEY,+ )+ else:+ # Outbound observation only — direct legacy raw broadcast.+ await broadcaster.broadcast({+ "type": _MESSAGE_SENT_WIRE_KEY,+ "data": payload,+ })async def broadcast_whatsapp_newsletter(verb: Literal["joined", "left", "muted", "live_updated"],params: Mapping[str, Any],) -> None:- """Emit a newsletter-lifecycle event. ``verb`` selects the- legacy wire key from :data:`_NEWSLETTER_WIRE_KEYS`."""+ """Emit a newsletter-lifecycle event for FE observation.++ Not a trigger event — no canary consumer. Direct raw broadcast on+ the legacy wire key until the FE newsletter UI migrates to read+ the envelope shape, at which point this becomes a ``dispatch.emit``+ call like :func:`broadcast_whatsapp_message`.+ """from services.status_broadcaster import get_status_broadcasterbroadcaster = get_status_broadcaster()- wire_key = _NEWSLETTER_WIRE_KEYS[verb]-await broadcaster.broadcast({- "type": wire_key,+ "type": _NEWSLETTER_WIRE_KEYS[verb],"data": dict(params),})- event = whatsapp_newsletter_event(verb, params)- await broadcaster.broadcast({- "type": wire_key,- "data": event.model_dump(mode="json"),- })-async def broadcast_whatsapp_history_synced(params: Mapping[str, Any]) -> None:- """Emit a history-sync-complete event."""+ """Emit a history-sync-complete event for FE observation.++ Not a trigger event — no canary consumer. Direct raw broadcast.+ """from services.status_broadcaster import get_status_broadcasterbroadcaster = get_status_broadcaster()-await broadcaster.broadcast({"type": _HISTORY_SYNCED_WIRE_KEY,"data": dict(params),})- event = whatsapp_history_synced(params)- await broadcaster.broadcast({- "type": _HISTORY_SYNCED_WIRE_KEY,- "data": event.model_dump(mode="json"),- })-__all__ = ["broadcast_whatsapp_history_synced",diff --git a/server/routers/websocket.py b/server/routers/websocket.pyindex 462a6ec0..22a58310 100644--- a/server/routers/websocket.py+++ b/server/routers/websocket.py@@ -1001,16 +1001,16 @@ async def handle_send_chat_message(data: Dict[str, Any], websocket: WebSocket) -"session_id": session_id}- # Dispatch to chatTrigger waiters (legacy event_waiter + Temporal-durable- # canary via dispatch.emit; the wrapper is async since Wave 12 C1).- resolved = await dispatch_chat_message_received(event_data)+ # Dispatch via canary CloudEvents path — Visibility-query Signal+ # fan-out to running TriggerListenerWorkflow consumers + in-process+ # WS broadcast on ``chat_message_received``.+ await dispatch_chat_message_received(event_data)- logger.info(f"[ChatMessage] Dispatched message to {resolved} chatTrigger waiter(s)")+ logger.info(f"[ChatMessage] Dispatched canary event for session={session_id}")return {"success": True,"message": "Chat message sent",- "resolved_count": resolved,"timestamp": timestamp}diff --git a/server/services/deployment/canary_registry.py b/server/services/deployment/canary_registry.pyindex 3d023f27..e769c6c7 100644--- a/server/services/deployment/canary_registry.py+++ b/server/services/deployment/canary_registry.py@@ -20,46 +20,86 @@ ensures events reach that listener.Membership semantics (set-shaped):- register_canary_trigger_type("webhookTrigger")- register_canary_trigger_type("chatTrigger")+ register_canary_trigger_type("webhookTrigger", "com.machinaos.webhook.received")+ register_canary_trigger_type("chatTrigger", "com.machinaos.chat.message.received")is_canary_trigger_type("webhookTrigger") # Trueis_canary_trigger_type("whatsappReceive") # False (not yet opted in)--Idempotent on re-import (multiple registrations of the same type are a-no-op). No collision semantics because there's no "value" to conflict —-membership is the contract.+ cloudevent_type_for("webhookTrigger") # "com.machinaos.webhook.received"++Idempotent on re-import (multiple registrations of the same type with+the same cloudevent_type are a no-op; a conflicting cloudevent_type+raises so plugin upgrades surface drift loudly).++**Why cloudevent_type is required (Wave 12 bug fix, 2026-05-15).** The+``TriggerListenerWorkflow`` is tagged at start with an ``EventType``+Search Attribute. :func:`services.events.dispatch.emit` runs a+Visibility query ``EventType='<value>' AND ExecutionStatus='Running'``+to find listeners and signal them. The query uses ``event.type`` from+the CloudEvents envelope (reverse-DNS, e.g.+``com.machinaos.chat.message.received``). The Search Attribute MUST+carry the same value or the query never matches and no signal is+delivered — symptom was "TriggerListener started ok but never reacts+to incoming events". Pre-fix the deployment manager registered the+legacy snake_case ``event_type`` from ``TriggerConfig`` (e.g.+``chat_message_received``), which silently broke every C1 canary."""from __future__ import annotations-from typing import FrozenSet, Iterator, Set+from typing import Dict, FrozenSet, Iterator, Optional-_REGISTERED: Set[str] = set()+_REGISTERED: Dict[str, str] = {}-def register_canary_trigger_type(node_type: str) -> None:+def register_canary_trigger_type(node_type: str, cloudevent_type: str) -> None:"""Opt this trigger node type into the Wave 12 C1 canary path.- Called from a plugin's ``__init__.py``. Idempotent: re-importing the- plugin module (e.g. during ``importlib.reload`` in the test suite)- is a no-op.+ Called from a plugin's ``__init__.py``. Idempotent on re-import when+ the same ``cloudevent_type`` is supplied; a conflicting+ ``cloudevent_type`` raises :class:`ValueError` so an accidentally+ diverging plugin upgrade can't silently produce events that the+ listener can't see.Args:node_type: The trigger node's ``BaseNode.type`` string. Thestring MUST match what the workflow JSON carries for nodesof this type — that's the key DeploymentManager looks upwhen deciding which trigger goes to the canary listener.+ cloudevent_type: The CloudEvents ``type`` field the plugin's+ ``_events.py`` factory will populate on outgoing envelopes+ (reverse-DNS, e.g. ``"com.machinaos.chat.message.received"``).+ Used as the ``EventType`` Search Attribute value when the+ listener workflow is started so+ :func:`services.events.dispatch.emit`'s Visibility query+ matches it.Raises:- TypeError: if ``node_type`` is not a non-empty string.+ TypeError: if ``node_type`` or ``cloudevent_type`` is not a+ non-empty string.+ ValueError: if ``node_type`` was already registered with a+ different ``cloudevent_type``."""if not isinstance(node_type, str) or not node_type:raise TypeError(- f"register_canary_trigger_type expected a non-empty str, "- f"got {type(node_type).__name__}: {node_type!r}"+ f"register_canary_trigger_type expected a non-empty str for "+ f"node_type, got {type(node_type).__name__}: {node_type!r}"+ )+ if not isinstance(cloudevent_type, str) or not cloudevent_type:+ raise TypeError(+ f"register_canary_trigger_type expected a non-empty str for "+ f"cloudevent_type, got {type(cloudevent_type).__name__}: "+ f"{cloudevent_type!r}"+ )+ existing = _REGISTERED.get(node_type)+ if existing is not None and existing != cloudevent_type:+ raise ValueError(+ f"register_canary_trigger_type: node_type={node_type!r} already "+ f"registered with cloudevent_type={existing!r}; refusing to "+ f"overwrite with {cloudevent_type!r}. A diverging registration "+ f"would silently break the Visibility-query → Signal fan-out.")- _REGISTERED.add(node_type)+ _REGISTERED[node_type] = cloudevent_typedef is_canary_trigger_type(node_type: str) -> bool:@@ -67,6 +107,15 @@ def is_canary_trigger_type(node_type: str) -> bool:return node_type in _REGISTERED+def cloudevent_type_for(node_type: str) -> Optional[str]:+ """Return the CloudEvents ``type`` registered for ``node_type``.++ ``None`` when the trigger isn't canary-registered — caller decides+ whether to fall back to a legacy event_type or error.+ """+ return _REGISTERED.get(node_type)++def canary_trigger_types() -> FrozenSet[str]:"""Return an immutable snapshot of registered canary trigger types.@@ -84,5 +133,6 @@ def __iter__() -> Iterator[str]: # pragma: no cover — module dunders only__all__ = ["register_canary_trigger_type","is_canary_trigger_type",+ "cloudevent_type_for","canary_trigger_types",]diff --git a/server/services/deployment/manager.py b/server/services/deployment/manager.pyindex a81b9b52..d3f271d9 100644--- a/server/services/deployment/manager.py+++ b/server/services/deployment/manager.py@@ -283,6 +283,31 @@ class DeploymentManager:# (client.list_schedules vs client.list_workflows).cron_schedule_count = await self._cancel_canary_cron_schedules(workflow_id)+ # Sweep any downstream nodes still glowing on the canvas. The+ # cron + listener resets above only cover the trigger nodes the+ # manager owns directly; if a child run was mid-execution when+ # cancellation hit, downstream agents / tools / actions may have+ # been broadcast ``executing`` and would otherwise stay glowing+ # forever. ``include_waiting=True`` because explicit user-cancel+ # is the "every indicator goes quiet" signal — matches the+ # behaviour of the ``handle_cancel_execution`` WS path. The+ # delegation guard inside ``_clear_stuck_node_statuses`` still+ # protects in-flight fire-and-forget child agents.+ await self._broadcaster._clear_stuck_node_statuses(+ workflow_id, include_waiting=True,+ )++ # And broadcast a final ``executing=False`` for the deployment+ # so the toolbar Start/Stop indicator reflects the cancel. The+ # legacy path's run-counter eviction (``workflow_run_ended``)+ # doesn't fire here because deployment-level cancel can race+ # ahead of in-flight ``workflow_run_started`` callers — emit+ # the terminal state directly to avoid a stuck ``executing=True``+ # on the FE toolbar.+ await self._broadcaster.update_workflow_status(+ executing=False, workflow_id=workflow_id,+ )+# Clear cron iteration counters for this workflow's cron nodesfor node_id in cron_node_ids:self._cron_iterations.pop(node_id, None)@@ -675,9 +700,37 @@ class DeploymentManager:if state is None:raise RuntimeError(f"No deployment state for workflow {workflow_id}")+ from services.deployment.canary_registry import cloudevent_type_for+config = event_waiter.get_trigger_config(node_type)+ # Legacy snake_case event_type — only used in the listener_args+ # payload + node-status display strings. NOT the Search Attribute+ # value; that has to be the CloudEvents reverse-DNS string the+ # producer puts on outgoing envelopes (see EventType SA pair+ # below).event_type = config.event_type if config else f"unknown_{node_type}"+ # CloudEvents type the producer's _events.py factory emits on its+ # outgoing envelope. Used as the EventType Search Attribute value+ # so services.events.dispatch.emit's Visibility query+ # ``ListWorkflows(query="EventType='<event.type>'")`` actually+ # matches this listener.+ #+ # Pre-fix (2026-05-15) the EventType SA was set to ``event_type``+ # (the legacy snake_case), but dispatch.emit queries with the+ # producer's CloudEvents reverse-DNS string. The mismatch silently+ # zeroed the signal fan-out — listener started OK, never reacted.+ cloudevent_type = cloudevent_type_for(node_type)+ if cloudevent_type is None:+ logger.warning(+ "Canary listener: no cloudevent_type registered for node_type; "+ "falling back to legacy event_type for EventType SA. This will "+ "silently fail dispatch.emit fan-out — register cloudevent_type "+ "in the plugin's __init__.py.",+ node_id=node_id, workflow_id=workflow_id, node_type=node_type,+ )+ cloudevent_type = event_type+# Pick workflow type by plugin class. EventTriggerKind picks up# the "polling" classification for ops dashboards independent# of the per-plugin kind (gmail / twitter / ...).@@ -730,7 +783,10 @@ class DeploymentManager:task_queue="machina-tasks",id_conflict_policy=WorkflowIDConflictPolicy.USE_EXISTING,search_attributes=TypedSearchAttributes([- SearchAttributePair(event_type_key, event_type),+ # MUST be the CloudEvents type the producer emits on+ # outgoing envelopes — dispatch.emit's Visibility query+ # substitutes ``event.type`` into the EventType filter.+ SearchAttributePair(event_type_key, cloudevent_type),SearchAttributePair(trigger_node_id_key, node_id),SearchAttributePair(event_workflow_id_key, workflow_id),SearchAttributePair(event_trigger_kind_key, trigger_kind),diff --git a/server/services/events/envelope.py b/server/services/events/envelope.pyindex 9b7cc749..661fc60e 100644--- a/server/services/events/envelope.py+++ b/server/services/events/envelope.py@@ -324,11 +324,24 @@ class WorkflowEvent(BaseModel):agent: str,data: Optional[Mapping[str, Any]] = None,) -> "WorkflowEvent":- """Delegated child-agent completion. Type discriminates on- succeeded vs failed; ``taskTrigger`` filters by both."""+ """Delegated child-agent completion.++ Both success and failure share one ``type``+ (``com.machinaos.agent.task.completed``) so the+ ``TriggerListenerWorkflow`` for ``taskTrigger`` can register a+ single ``EventType`` Search Attribute and match every+ completion event via :func:`services.events.dispatch.emit`'s+ Visibility query. Status discrimination lives in ``data.status``+ (``"completed"`` or ``"error"``) — :class:`TaskTriggerNode`'s+ filter already keys off this field.++ Pre-fix (commit ?) the type split into ``.succeeded`` /+ ``.failed`` which left the listener's SA value matching only+ one branch and silently lost the other half of events.+ """return cls(source="machinaos://services/agent",- type=f"{_TYPE_PREFIX}agent.task.{'succeeded' if status == 'completed' else 'failed'}",+ type=f"{_TYPE_PREFIX}agent.task.completed",subject=task_id,data=dict(data) if data else {"task_id": task_id, "agent": agent, "status": status},)diff --git a/server/services/plugin/polling.py b/server/services/plugin/polling.pyindex 2dcb00b2..bbfd70ab 100644--- a/server/services/plugin/polling.py+++ b/server/services/plugin/polling.py@@ -260,7 +260,16 @@ class PollingTriggerNode(TriggerNode, abstract=True):except Exception:pass- return {"events": events, "seen_ids": list(prior_seen | current)}+ # OOM fix: bound ``seen_ids`` to what the provider currently+ # reports as visible — old IDs that have dropped off the+ # provider's window (archived / deleted / aged out) fall+ # out of seen too. Pre-fix this was ``prior_seen | current``+ # which grew forever (every poll appended new IDs; nothing+ # ever got evicted). At Gmail's ~100 msgs/day cadence the+ # set hit ~36K entries in a year — ~1.4MB just for IDs,+ # serialised through Temporal payload on every cycle. Now+ # bounded by the provider's natural window size.+ return {"events": events, "seen_ids": list(current)}return _poll_cycle_activity@@ -313,6 +322,11 @@ class PollingTriggerNode(TriggerNode, abstract=True):current = await self.fetch_ids(service, params)new_ids = current - seenif not new_ids:+ # OOM fix: even when nothing new arrives we+ # still rebase ``seen`` to ``current`` so old+ # IDs that dropped off the provider's window+ # don't linger forever.+ seen = set(current)continuelogger.debug("Polling trigger cycle",@@ -321,7 +335,6 @@ class PollingTriggerNode(TriggerNode, abstract=True):new=len(new_ids),)for msg_id in new_ids:- seen.add(msg_id)payload = await self.fetch_detail(service, msg_id, params)@@ -333,6 +346,18 @@ class PollingTriggerNode(TriggerNode, abstract=True):await self.post_emit(service, msg_id, params)except Exception:pass+ # OOM fix: rebase ``seen`` to the provider's current+ # window. Pre-fix this added every new ID to ``seen``+ # forever (``seen.add(msg_id)`` inside the loop, no+ # eviction). Long-running pollers (Gmail with the+ # default 60s interval) accumulated tens of thousands+ # of entries over weeks/months. Now bounded by the+ # provider's natural window size — items that have+ # been emitted are still in ``current`` until they+ # age/archive out, at which point dropping them is+ # correct (the provider will never re-surface them+ # under the same filter).+ seen = set(current)except asyncio.CancelledError:raiseexcept Exception as exc: # noqa: BLE001 -- per-cycle isolationdiff --git a/server/services/temporal/activities.py b/server/services/temporal/activities.pyindex 73f404c6..9d595db9 100644--- a/server/services/temporal/activities.py+++ b/server/services/temporal/activities.py@@ -372,3 +372,97 @@ async def emit_event_activity(event_payload: Dict[str, Any]) -> Dict[str, Any]:"event_id": event.id,"event_type": event.type,}+++# ---------------------------------------------------------------------------+# Trigger-node status broadcast activity+#+# Used by :class:`services.temporal.trigger_listener_workflow.TriggerListenerWorkflow`+# and :class:`services.temporal.polling_trigger_workflow.PollingTriggerWorkflow`+# to surface firing/waiting transitions to FE. Workflows can't broadcast+# directly (sandboxed, deterministic) so they offload to this activity.+#+# Mirrors the legacy ``services/deployment/triggers.py`` collector/processor+# transitions: ``"waiting" → "idle (processing)" → "waiting"`` per event.+# Without this, the canary path would keep the trigger node visually+# stuck on "waiting" with no firing pulse — a UX regression vs legacy.++async def store_node_output_activity(payload: Dict[str, Any]) -> None:+ """Persist a node's output to the workflow output store so+ :class:`services.parameter_resolver.ParameterResolver` can resolve+ ``{{nodeName.field}}`` templates in downstream nodes.++ Used by :class:`services.temporal.workflow.MachinaWorkflow` for+ pre-executed trigger nodes — they bypass ``NodeExecutor.execute``+ (which is what normally persists outputs) because the workflow+ received their output via the parent ``TriggerListenerWorkflow`` /+ ``PollingTriggerWorkflow`` signal, not via the standard activity+ dispatch. Without this persist step, ``ParameterResolver`` reads+ ``None`` from the store and downstream templates like+ ``{{chatTrigger.message}}`` silently resolve to empty.++ Mirrors the per-handle write pattern in+ :func:`services.temporal.agent_activities.store_agent_output`.++ Payload shape::++ {+ "node_id": str,+ "session_id": str,+ "result": dict, # the trigger output payload+ }+ """+ from core.container import container++ workflow_service = container.workflow_service()+ node_id = payload["node_id"]+ session_id = payload.get("session_id", "default")+ data = payload.get("result") or {}+ # Mirror NodeExecutor — write to every standard output handle so+ # the resolver finds the data regardless of which handle name the+ # downstream edge targets.+ for output_name in ("output_main", "output_top", "output_0"):+ try:+ await workflow_service.store_node_output(+ session_id, node_id, output_name, data,+ )+ except Exception as exc: # noqa: BLE001 — non-fatal+ activity.logger.warning(+ f"store_node_output_activity failed for "+ f"node={node_id!r} handle={output_name!r}: {exc}"+ )+++async def broadcast_trigger_status_activity(payload: Dict[str, Any]) -> None:+ """Broadcast a trigger node status update for FE animation.++ Payload shape:+ {+ "node_id": str,+ "status": "waiting" | "idle" | ...,+ "data": dict, # optional, defaults to {}+ "workflow_id": str | None,+ }++ Embedded-worker mode (default): direct in-process call to the+ status_broadcaster singleton — same event loop as FastAPI handlers.+ Non-fatal: broadcast failures are logged at WARNING and swallowed+ so they don't fail the calling workflow.+ """+ from services.status_broadcaster import get_status_broadcaster++ try:+ broadcaster = get_status_broadcaster()+ await broadcaster.update_node_status(+ node_id=payload["node_id"],+ status=payload.get("status", "waiting"),+ data=payload.get("data", {}) or {},+ workflow_id=payload.get("workflow_id"),+ )+ except Exception as exc: # noqa: BLE001 — non-fatal+ activity.logger.warning(+ f"broadcast_trigger_status_activity failed for "+ f"node={payload.get('node_id')!r}: {exc}"+ )diff --git a/server/services/temporal/polling_trigger_workflow.py b/server/services/temporal/polling_trigger_workflow.pyindex f4fbe5c8..9ca0b468 100644--- a/server/services/temporal/polling_trigger_workflow.py+++ b/server/services/temporal/polling_trigger_workflow.py@@ -201,7 +201,11 @@ class PollingTriggerWorkflow:nodes) stay single-source. Mirrors:meth:`TriggerListenerWorkflow._spawn_child_run` exactly."""- from services.temporal.trigger_listener_workflow import _build_run_graph+ from services.temporal.trigger_listener_workflow import (+ _broadcast_trigger_idle,+ _broadcast_trigger_waiting,+ _build_run_graph,+ )trigger_node_id = listener_data["trigger_node_id"]nodes = listener_data["nodes"]@@ -231,6 +235,13 @@ class PollingTriggerWorkflow:event_id = event.get("id") or workflow.uuid4().hexchild_id = f"run-{workflow_id}-{event_id}"+ await _broadcast_trigger_idle(+ node_id=trigger_node_id,+ workflow_id=workflow_id,+ event_id=event_id,+ event_type=listener_data.get("event_type", ""),+ )+await workflow.start_child_workflow("MachinaWorkflow",args=[{@@ -252,5 +263,12 @@ class PollingTriggerWorkflow:f"event.id={event.get('id')}")+ await _broadcast_trigger_waiting(+ node_id=trigger_node_id,+ workflow_id=workflow_id,+ event_type=listener_data.get("event_type", ""),+ processed_count=self._processed_count + 1,+ )+__all__ = ["PollingTriggerWorkflow"]diff --git a/server/services/temporal/trigger_listener_workflow.py b/server/services/temporal/trigger_listener_workflow.pyindex 8fced073..37ccd403 100644--- a/server/services/temporal/trigger_listener_workflow.py+++ b/server/services/temporal/trigger_listener_workflow.py@@ -149,6 +149,12 @@ class TriggerListenerWorkflow:the same deployment are marked ``_pre_executed=True`` with``{not_triggered: True}`` so MachinaWorkflow doesn't block onthem (mirrors ``DeploymentManager._execute_from_trigger``).++ Status broadcast lifecycle (matches legacy+ ``deployment/triggers.py`` collector/processor):+ - Before spawn → trigger node ``"idle"`` with "Graph executing..."+ - After spawn returns → trigger node ``"waiting"`` (child runs+ independently per ``parent_close_policy=ABANDON``)."""trigger_node_id = listener_data["trigger_node_id"]nodes = listener_data["nodes"]@@ -180,6 +186,13 @@ class TriggerListenerWorkflow:event_id = event.get("id") or workflow.uuid4().hexchild_id = f"run-{workflow_id}-{event_id}"+ await _broadcast_trigger_idle(+ node_id=trigger_node_id,+ workflow_id=workflow_id,+ event_id=event_id,+ event_type=event.get("type", ""),+ )+await workflow.start_child_workflow("MachinaWorkflow",args=[{@@ -201,6 +214,78 @@ class TriggerListenerWorkflow:f"event.id={event.get('id')} event.type={event.get('type')}")+ await _broadcast_trigger_waiting(+ node_id=trigger_node_id,+ workflow_id=workflow_id,+ event_type=listener_data.get("event_type", ""),+ processed_count=self._processed_count + 1,+ )+++# ---------------------------------------------------------------------------+# Status-broadcast helpers — thin wrappers around the activity so the+# spawn loop reads cleanly. Activity name string matches the @activity.defn+# registration in services/temporal/activities.py.+# ---------------------------------------------------------------------------+++_STATUS_ACTIVITY_NAME = "broadcast_trigger_status_activity"+_STATUS_ACTIVITY_TIMEOUT = timedelta(seconds=5)+++async def _broadcast_trigger_idle(+ *,+ node_id: str,+ workflow_id: Optional[str],+ event_id: str,+ event_type: str,+) -> None:+ """Broadcast trigger node ``"idle"`` status with a "Graph executing..."+ message — matches the legacy collector/processor transition so FE+ shows a firing pulse instead of a stuck "waiting" indicator."""+ await workflow.execute_activity(+ _STATUS_ACTIVITY_NAME,+ {+ "node_id": node_id,+ "status": "idle",+ "data": {+ "message": "Graph executing...",+ "is_processing": True,+ "event_id": event_id,+ "event_type": event_type,+ },+ "workflow_id": workflow_id,+ },+ start_to_close_timeout=_STATUS_ACTIVITY_TIMEOUT,+ )+++async def _broadcast_trigger_waiting(+ *,+ node_id: str,+ workflow_id: Optional[str],+ event_type: str,+ processed_count: int,+) -> None:+ """Broadcast trigger node back to ``"waiting"`` after the child run+ has been spawned (child completes independently per+ ``parent_close_policy=ABANDON``)."""+ await workflow.execute_activity(+ _STATUS_ACTIVITY_NAME,+ {+ "node_id": node_id,+ "status": "waiting",+ "data": {+ "message": "Waiting for next event...",+ "is_processing": False,+ "event_type": event_type,+ "processed_count": processed_count,+ },+ "workflow_id": workflow_id,+ },+ start_to_close_timeout=_STATUS_ACTIVITY_TIMEOUT,+ )+# ---------------------------------------------------------------------------# Pure graph builder — kept in this module on purpose: the only consumer isdiff --git a/server/services/temporal/worker.py b/server/services/temporal/worker.pyindex 8e8fd784..77a41a10 100644--- a/server/services/temporal/worker.py+++ b/server/services/temporal/worker.py@@ -140,6 +140,10 @@ class TemporalWorkerManager:)from services.temporal.agent_activities import collect_agent_activitiesfrom services.temporal.agent_workflow import AgentWorkflow+ from services.temporal.activities import (+ broadcast_trigger_status_activity,+ store_node_output_activity,+ )per_type = collect_plugin_activities() # no queue filter; all pluginsagent_activities = collect_agent_activities()@@ -164,6 +168,8 @@ class TemporalWorkerManager:],activities=[self._activities.execute_node_activity,+ broadcast_trigger_status_activity,+ store_node_output_activity,*per_type,*agent_activities,*polling_activities,@@ -183,7 +189,6 @@ class TemporalWorkerManager:span.set_attribute("task_queue", self.task_queue)span.set_attribute("pool_size", self.pool_size)- print(f"[Temporal] Worker started (queue={self.task_queue}, pool={self.pool_size})", flush=True)logger.info("Starting Temporal worker",task_queue=self.task_queue,@@ -385,6 +390,11 @@ async def run_standalone_worker(session = await create_shared_session(pool_size)activities = NodeExecutionActivities(session)+ from services.temporal.activities import (+ broadcast_trigger_status_activity,+ store_node_output_activity,+ )+try:worker = Worker(client,@@ -394,7 +404,11 @@ async def run_standalone_worker(TriggerListenerWorkflow,PollingTriggerWorkflow,],- activities=[activities.execute_node_activity], # Pass bound method+ activities=[+ activities.execute_node_activity,+ broadcast_trigger_status_activity,+ store_node_output_activity,+ ],max_concurrent_activities=pool_size,max_concurrent_workflow_tasks=10,graceful_shutdown_timeout=_graceful_shutdown_timeout(),@@ -428,6 +442,10 @@ async def create_worker(session = await create_shared_session()activities = NodeExecutionActivities(session)+ from services.temporal.activities import (+ broadcast_trigger_status_activity,+ store_node_output_activity,+ )return Worker(client,@@ -437,7 +455,11 @@ async def create_worker(TriggerListenerWorkflow,PollingTriggerWorkflow,],- activities=[activities.execute_node_activity], # Pass bound method+ activities=[+ activities.execute_node_activity,+ broadcast_trigger_status_activity,+ store_node_output_activity,+ ],max_concurrent_activities=100,max_concurrent_workflow_tasks=10,graceful_shutdown_timeout=_GRACEFUL_SHUTDOWN_TIMEOUT,diff --git a/server/services/temporal/workflow.py b/server/services/temporal/workflow.pyindex 4446b282..07d00430 100644--- a/server/services/temporal/workflow.py+++ b/server/services/temporal/workflow.py@@ -231,9 +231,10 @@ class MachinaWorkflow:for node in exec_nodes:if node.get("_pre_executed"):node_id = node["id"]+ trigger_output = node.get("_trigger_output", {})outputs[node_id] = {"success": True,- "result": node.get("_trigger_output", {}),+ "result": trigger_output,"pre_executed": True,}completed.add(node_id)@@ -241,6 +242,28 @@ class MachinaWorkflow:pre_executed_count += 1workflow.logger.info(f"Pre-executed trigger: {node_id}")+ # Persist the trigger output to the workflow output+ # store so ParameterResolver can resolve+ # ``{{triggerNodeName.field}}`` in downstream nodes.+ # Pre-executed nodes bypass NodeExecutor.execute (the+ # normal write site), so without this activity the+ # legacy path's ``_store_output(trigger_node_id, ...)``+ # behaviour is missing on the canary path and templates+ # against the trigger silently resolve to empty.+ # Skip the persist for non-firing siblings — their+ # ``_trigger_output`` is ``{not_triggered: True}``, no+ # downstream template should resolve against them.+ if not trigger_output.get("not_triggered"):+ await workflow.execute_activity(+ "store_node_output_activity",+ {+ "node_id": node_id,+ "session_id": session_id,+ "result": trigger_output,+ },+ start_to_close_timeout=timedelta(seconds=10),+ )+workflow.logger.info(f"Pre-executed: {pre_executed_count}, To execute: {len(node_map) - pre_executed_count}")# 5. Retry policy for node activities. Wave 12 D1: importeddiff --git a/server/tests/nodes/test_email.py b/server/tests/nodes/test_email.pyindex bcd84458..740e49c7 100644--- a/server/tests/nodes/test_email.py+++ b/server/tests/nodes/test_email.py@@ -229,8 +229,9 @@ class TestEmailReceive:patch.object(EmailService, "fetch_detail", fetch_detail_mock), \patch("asyncio.sleep", new=AsyncMock(return_value=None)), \patch("services.status_broadcaster.get_status_broadcaster") as bcast, \- patch("services.event_waiter.dispatch") as dispatch_mock:+ patch("services.events.dispatch.emit") as emit_mock:bcast.return_value.update_node_status = AsyncMock(return_value=None)+ emit_mock.return_value = Noneresult = await harness.execute("emailReceive",@@ -244,9 +245,13 @@ class TestEmailReceive:assert payload["subject"] == "new!"# Baseline + one diffing pollassert poll_ids_mock.await_count == 2- # Event dispatched exactly once- dispatch_mock.assert_called_once()- assert dispatch_mock.call_args.args[0] == "email_received"+ # Event routed through the canary CloudEvents path (legacy+ # event_waiter.dispatch was retired in Wave 13 — emailReceive+ # is canary-registered and the legacy collector has no consumer).+ emit_mock.assert_called_once()+ envelope = emit_mock.call_args.args[0]+ assert envelope.type == "com.machinaos.email.message.received"+ assert emit_mock.call_args.kwargs["wire_routing_key"] == "email_received"async def test_mark_as_read_adds_seen_flag(self, harness):from nodes.email._service import EmailService@@ -262,7 +267,7 @@ class TestEmailReceive:patch.object(HimalayaService, "flag_message", flag_mock), \patch("asyncio.sleep", new=AsyncMock(return_value=None)), \patch("services.status_broadcaster.get_status_broadcaster") as bcast, \- patch("services.event_waiter.dispatch"):+ patch("services.events.dispatch.emit"):bcast.return_value.update_node_status = AsyncMock(return_value=None)result = await harness.execute(@@ -300,7 +305,7 @@ class TestEmailReceive:patch.object(EmailService, "poll_ids", poll_ids_mock), \patch("asyncio.sleep", new=AsyncMock(return_value=None)), \patch("services.status_broadcaster.get_status_broadcaster") as bcast, \- patch("services.event_waiter.dispatch"):+ patch("services.events.dispatch.emit"):bcast.return_value.update_node_status = AsyncMock(return_value=None)result = await harness.execute(diff --git a/server/tests/test_canary_registry.py b/server/tests/test_canary_registry.pyindex 25df0a5a..9b7a266c 100644--- a/server/tests/test_canary_registry.py+++ b/server/tests/test_canary_registry.py@@ -39,55 +39,87 @@ if "machina" not in sys.modules:@pytest.fixturedef fresh_registry(monkeypatch):- """Reset the registry's backing set so each test runs in isolation."""+ """Reset the registry's backing dict so each test runs in isolation."""from services.deployment import canary_registry- monkeypatch.setattr(canary_registry, "_REGISTERED", set())+ monkeypatch.setattr(canary_registry, "_REGISTERED", {})return canary_registry+# Dummy CloudEvents types used in registry tests — the real values come+# from each plugin's ``_events.py`` factory.+_CE_WEBHOOK = "com.machinaos.webhook.received"+_CE_CHAT = "com.machinaos.chat.message.received"+_CE_TASK = "com.machinaos.agent.task.completed"++class TestRegistryContract:- """Surface API: register, query, snapshot."""+ """Surface API: register, query, snapshot, cloudevent_type lookup."""def test_unregistered_type_is_not_canary(self, fresh_registry):assert fresh_registry.is_canary_trigger_type("webhookTrigger") is False+ assert fresh_registry.cloudevent_type_for("webhookTrigger") is Nonedef test_register_then_query_returns_true(self, fresh_registry):- fresh_registry.register_canary_trigger_type("webhookTrigger")+ fresh_registry.register_canary_trigger_type("webhookTrigger", _CE_WEBHOOK)assert fresh_registry.is_canary_trigger_type("webhookTrigger") is True+ assert fresh_registry.cloudevent_type_for("webhookTrigger") == _CE_WEBHOOK- def test_register_is_idempotent(self, fresh_registry):- fresh_registry.register_canary_trigger_type("chatTrigger")- fresh_registry.register_canary_trigger_type("chatTrigger")- fresh_registry.register_canary_trigger_type("chatTrigger")+ def test_register_is_idempotent_with_same_cloudevent_type(self, fresh_registry):+ fresh_registry.register_canary_trigger_type("chatTrigger", _CE_CHAT)+ fresh_registry.register_canary_trigger_type("chatTrigger", _CE_CHAT)+ fresh_registry.register_canary_trigger_type("chatTrigger", _CE_CHAT)assert fresh_registry.canary_trigger_types() == frozenset({"chatTrigger"})+ assert fresh_registry.cloudevent_type_for("chatTrigger") == _CE_CHAT++ def test_register_rejects_conflicting_cloudevent_type(self, fresh_registry):+ """Reload-tolerance is bounded: same type re-registered with a+ diverging CloudEvents type is a loud :class:`ValueError`, not a+ silent overwrite. Catches plugin upgrades that change the+ producer envelope shape without coordinating with the listener."""+ fresh_registry.register_canary_trigger_type("chatTrigger", _CE_CHAT)+ with pytest.raises(ValueError, match="diverging"):+ fresh_registry.register_canary_trigger_type(+ "chatTrigger", "com.machinaos.chat.message.changed"+ )def test_multiple_types_coexist(self, fresh_registry):- for t in ("webhookTrigger", "chatTrigger", "taskTrigger"):- fresh_registry.register_canary_trigger_type(t)+ for t, ce in (+ ("webhookTrigger", _CE_WEBHOOK),+ ("chatTrigger", _CE_CHAT),+ ("taskTrigger", _CE_TASK),+ ):+ fresh_registry.register_canary_trigger_type(t, ce)assert fresh_registry.canary_trigger_types() == frozenset({"webhookTrigger", "chatTrigger", "taskTrigger",})+ assert fresh_registry.cloudevent_type_for("taskTrigger") == _CE_TASKdef test_snapshot_is_immutable(self, fresh_registry):"""``canary_trigger_types()`` returns a frozenset — mutating itmust not leak back into the registry."""- fresh_registry.register_canary_trigger_type("webhookTrigger")+ fresh_registry.register_canary_trigger_type("webhookTrigger", _CE_WEBHOOK)snap = fresh_registry.canary_trigger_types()assert isinstance(snap, frozenset)- # frozenset has no add() — TypeError if attempted.+ # frozenset has no add() — AttributeError if attempted.with pytest.raises(AttributeError):snap.add("chatTrigger") # type: ignore[attr-defined]- def test_rejects_non_string(self, fresh_registry):+ def test_rejects_non_string_node_type(self, fresh_registry):+ with pytest.raises(TypeError):+ fresh_registry.register_canary_trigger_type(None, _CE_WEBHOOK) # type: ignore[arg-type]with pytest.raises(TypeError):- fresh_registry.register_canary_trigger_type(None) # type: ignore[arg-type]+ fresh_registry.register_canary_trigger_type(123, _CE_WEBHOOK) # type: ignore[arg-type]++ def test_rejects_empty_node_type(self, fresh_registry):with pytest.raises(TypeError):- fresh_registry.register_canary_trigger_type(123) # type: ignore[arg-type]+ fresh_registry.register_canary_trigger_type("", _CE_WEBHOOK)- def test_rejects_empty_string(self, fresh_registry):+ def test_rejects_missing_cloudevent_type(self, fresh_registry):+ with pytest.raises(TypeError):+ fresh_registry.register_canary_trigger_type("webhookTrigger", None) # type: ignore[arg-type]with pytest.raises(TypeError):- fresh_registry.register_canary_trigger_type("")+ fresh_registry.register_canary_trigger_type("webhookTrigger", "")class TestNoFrameworkSideAllowlist:@@ -154,12 +186,14 @@ class TestPluginSelfRegistration:rather than masking the import failure."""- @pytest.mark.parametrize("plugin_module,expected_type", [- ("nodes.trigger.webhook_trigger", "webhookTrigger"),- ("nodes.trigger.chat_trigger", "chatTrigger"),- ("nodes.trigger.task_trigger", "taskTrigger"),+ @pytest.mark.parametrize("plugin_module,expected_type,expected_cloudevent_type", [+ ("nodes.trigger.webhook_trigger", "webhookTrigger", "com.machinaos.webhook.received"),+ ("nodes.trigger.chat_trigger", "chatTrigger", "com.machinaos.chat.message.received"),+ ("nodes.trigger.task_trigger", "taskTrigger", "com.machinaos.agent.task.completed"),])- def test_plugin_import_registers_its_type(self, plugin_module, expected_type):+ def test_plugin_import_registers_its_type(+ self, plugin_module, expected_type, expected_cloudevent_type+ ):from services.deployment import canary_registrytry:@@ -169,7 +203,108 @@ class TestPluginSelfRegistration:assert canary_registry.is_canary_trigger_type(expected_type), (f"Importing {plugin_module} should call "- f"register_canary_trigger_type({expected_type!r}). Check the "+ f"register_canary_trigger_type({expected_type!r}, ...). Check the ""module's __init__.py for the register_canary_trigger_type ""call at module scope.")+ # The cloudevent_type MUST match what the producer's _events.py+ # factory puts on outgoing envelopes — otherwise dispatch.emit's+ # Visibility query never finds the listener.+ assert canary_registry.cloudevent_type_for(expected_type) == expected_cloudevent_type, (+ f"{plugin_module} registered {expected_type!r} with the wrong "+ f"cloudevent_type. Expected {expected_cloudevent_type!r} to match "+ f"the producer's WorkflowEvent.type. If the producer factory "+ "changed, update both the factory AND the register_canary_trigger_type "+ "call together — drift here silently breaks the canary signal "+ "fan-out (TriggerListener starts ok, never reacts to events)."+ )+++class TestCloudEventTypeMatchesSearchAttribute:+ """Regression: every canary trigger's registered cloudevent_type MUST+ equal the ``event.type`` field a producer-side factory call would+ emit. The deployment manager uses ``cloudevent_type_for(node_type)``+ as the ``EventType`` Search Attribute on the listener workflow; the+ producer side uses ``event.type`` from ``WorkflowEvent`` factories+ as the Visibility query value in ``services.events.dispatch.emit``.++ These two strings MUST agree or the Signal fan-out silently zeroes+ out — symptom is "TriggerListenerWorkflow started but never reacts+ to incoming events". This invariant locks the contract.+ """++ def test_every_registered_type_has_reverse_dns_cloudevent_type(self):+ """All registered cloudevent_types are reverse-DNS strings+ (``com.machinaos.*``) — that's the CloudEvents §3.1.2 type+ convention this codebase declares (see envelope.py:_TYPE_PREFIX).+ Anything else means a plugin registered the legacy snake_case+ event_waiter event_type by mistake."""+ from services.deployment import canary_registry++ for node_type in canary_registry.canary_trigger_types():+ ce_type = canary_registry.cloudevent_type_for(node_type)+ assert ce_type is not None, (+ f"canary_trigger_types includes {node_type!r} but "+ f"cloudevent_type_for() returned None — registry corrupted."+ )+ assert ce_type.startswith("com.machinaos."), (+ f"{node_type!r} registered with cloudevent_type={ce_type!r}; "+ f"expected a reverse-DNS string starting with 'com.machinaos.'. "+ f"That's the format dispatch.emit's Visibility query "+ f"substitutes from envelope.type — anything else fails to "+ f"match the listener's EventType Search Attribute."+ )++ def test_dispatch_emit_query_uses_event_type_field(self):+ """Locks the contract on dispatch.emit's Visibility query: it+ MUST substitute ``event.type`` (CloudEvents reverse-DNS), not+ any other field. If this regresses to ``event.subject`` /+ legacy event_type / etc. the SA lookup breaks."""+ from services.events import dispatch as dispatch_mod++ src = inspect.getsource(dispatch_mod._signal_running_consumers)+ # The query template MUST format the CloudEvents type into the+ # EventType filter.+ assert "EventType=" in dispatch_mod._RUNNING_CONSUMERS_QUERY, (+ "dispatch.emit's _RUNNING_CONSUMERS_QUERY no longer filters by "+ "EventType — listener-side SA registration relies on this key."+ )+ assert "event.type" in src or "event_type=event.type" in src, (+ "dispatch.emit no longer substitutes event.type into the "+ "Visibility query. The listener registered SA value must match "+ "what's substituted here; using subject / id / data fields "+ "breaks the contract."+ )++ def test_deployment_manager_uses_cloudevent_type_for_event_type_sa(self):+ """Source-introspection regression on the deployment-side fix:+ ``_start_canary_listener`` must construct the EventType+ SearchAttributePair from ``cloudevent_type_for(node_type)``,+ NOT from ``config.event_type`` (the legacy event_waiter string)."""+ from services.deployment.manager import DeploymentManager++ src = inspect.getsource(DeploymentManager._start_canary_listener)+ assert "cloudevent_type_for" in src, (+ "DeploymentManager._start_canary_listener no longer calls "+ "cloudevent_type_for() to resolve the EventType Search Attribute. "+ "Pre-fix it used event_waiter.TriggerConfig.event_type (legacy "+ "snake_case) which never matched dispatch.emit's Visibility "+ "query (CloudEvents reverse-DNS). Restore the registry lookup."+ )+ # The SearchAttributePair for EventType MUST use the cloudevent_type+ # variable, not the legacy event_type one.+ pair_pattern = re.search(+ r"SearchAttributePair\s*\(\s*event_type_key\s*,\s*(\w+)",+ src,+ )+ assert pair_pattern is not None, (+ "Couldn't find the SearchAttributePair(event_type_key, ...) "+ "construction. Check _start_canary_listener for the SA wiring."+ )+ sa_value_name = pair_pattern.group(1)+ assert sa_value_name == "cloudevent_type", (+ f"SearchAttributePair(event_type_key, {sa_value_name}) — expected "+ f"'cloudevent_type'. The legacy 'event_type' variable holds the "+ f"snake_case event_waiter string and will silently break the "+ f"Signal fan-out (TriggerListener starts ok, never reacts)."+ )diff --git a/server/tests/test_chat_trigger_canary_producer.py b/server/tests/test_chat_trigger_canary_producer.pyindex 1a3da1da..322ac109 100644--- a/server/tests/test_chat_trigger_canary_producer.py+++ b/server/tests/test_chat_trigger_canary_producer.py@@ -1,16 +1,16 @@-"""Wave 12 C1 rollout #1: chatTrigger producer dual-emit invariant.+"""chatTrigger producer canary-emit invariant.Locks the contract: ``nodes.trigger.chat_trigger._events.dispatch_chat_message_received``-must route to BOTH the legacy in-process ``event_waiter.dispatch`` AND-the Temporal-durable ``services.events.dispatch.emit`` path. Without-the second call, a canary-enabled chatTrigger deployment would silently-miss all incoming chat messages — the TriggerListenerWorkflow has no-other event source.+routes through the canary CloudEvents path+(:func:`services.events.dispatch.emit`) ONLY. The legacy+``event_waiter.dispatch`` path was removed in Wave 13 — chatTrigger is+canary-registered, the deployment manager skips ``setup_event_trigger``,+and the legacy collector has zero consumers in production.Same regex-introspection invariant style as-``tests/test_credential_broadcasts.py`` — source-level assertions-catch the wire contract drifting without paying the cost of standing-up Temporal in CI.+``tests/test_credential_broadcasts.py`` — source-level assertions catch+the wire contract drifting without paying the cost of standing up+Temporal in CI."""from __future__ import annotations@@ -19,8 +19,8 @@ import inspectimport reimport sysimport types-from typing import Any, Dict, List-from unittest.mock import AsyncMock, MagicMock+from typing import Any, List+from unittest.mock import MagicMockimport pytest@@ -38,64 +38,50 @@ _EVENT_WAITER_DISPATCH_PATTERN = re.compile(r"event_waiter\.dispatch\s*\(")_EVENTS_EMIT_PATTERN = re.compile(r"\bemit\s*\(")-class TestChatTriggerProducerDualEmit:- """Producer wrapper emits BOTH legacy and Temporal envelopes."""+class TestChatTriggerProducerCanaryEmit:+ """Producer wrapper emits via the canary CloudEvents path only."""def test_dispatcher_is_async(self):from nodes.trigger.chat_trigger._events import dispatch_chat_message_received- # async coroutine functions return a coroutine when called.- # Both `asyncio.iscoroutinefunction` and `inspect.iscoroutinefunction`- # detect this.assert inspect.iscoroutinefunction(dispatch_chat_message_received), (- "dispatch_chat_message_received must be async since Wave 12 C1 — "- "it awaits services.events.dispatch.emit. Sync callers must "- "switch to `await dispatch_chat_message_received(...)`."+ "dispatch_chat_message_received must be async — it awaits "+ "services.events.dispatch.emit.")- def test_dispatcher_routes_both_legacy_and_temporal(self):+ def test_dispatcher_uses_canary_path_only(self):from nodes.trigger.chat_trigger import _eventssrc = inspect.getsource(_events.dispatch_chat_message_received)- # Legacy event_waiter.dispatch path stays for back-compat.- assert _EVENT_WAITER_DISPATCH_PATTERN.search(src), (- "dispatch_chat_message_received must still call "- "event_waiter.dispatch(_LEGACY_EVENT_TYPE, ...) for the "- "in-process waiter path. Without it, canary-flag-off "- "deployments lose their existing chat dispatch."- )-- # Temporal-durable path via dispatch.emit.assert _EVENTS_EMIT_PATTERN.search(src), ("dispatch_chat_message_received must call "- "services.events.dispatch.emit(envelope, ...) for the "- "Temporal-durable canary path. Without it, the "- "TriggerListenerWorkflow receives nothing when the "- "event_framework_enabled flag is on."+ "services.events.dispatch.emit(envelope, ...) — the canary "+ "CloudEvents path Signals running TriggerListenerWorkflow "+ "consumers AND broadcasts to FE on the chat_message_received "+ "wire key."+ )+ assert not _EVENT_WAITER_DISPATCH_PATTERN.search(src), (+ "dispatch_chat_message_received must NOT call "+ "event_waiter.dispatch — chatTrigger is canary-registered, "+ "the legacy collector path has zero consumers, and that "+ "call was removed in Wave 13. Reintroducing it would "+ "double-dispatch through dead infrastructure.")@pytest.mark.asyncio- async def test_runtime_dual_emit_calls_both_paths(self, monkeypatch):- """Integration smoke: invoke the dispatcher and assert both- downstream calls fire. event_framework_enabled=False so emit()- no-ops on the Temporal side; the call still happens though."""+ async def test_runtime_emits_canary_envelope(self, monkeypatch):+ """Invoking the dispatcher calls dispatch.emit with the right+ envelope. The legacy event_waiter is not touched."""from nodes.trigger.chat_trigger import _events- from services import event_waiterfrom services.events import dispatch as dispatch_mod- legacy_calls: List[Dict[str, Any]] = []emit_calls: List[Any] = []- def fake_dispatch(event_type, data):- legacy_calls.append({"event_type": event_type, "data": data})- return 0-async def fake_emit(event, **kwargs):emit_calls.append({"event": event, **kwargs})return event- monkeypatch.setattr(event_waiter, "dispatch", fake_dispatch)monkeypatch.setattr(dispatch_mod, "emit", fake_emit)result = await _events.dispatch_chat_message_received({@@ -104,12 +90,8 @@ class TestChatTriggerProducerDualEmit:"timestamp": "2026-05-14T00:00:00",})- # Legacy returned the resolved count.- assert result == 0-- assert len(legacy_calls) == 1- assert legacy_calls[0]["event_type"] == "chat_message_received"- assert legacy_calls[0]["data"]["message"] == "hello"+ # No return value — canary-only emit doesn't carry a waiter count.+ assert result is Noneassert len(emit_calls) == 1event = emit_calls[0]["event"]diff --git a/server/tests/test_deployment_canary_listener.py b/server/tests/test_deployment_canary_listener.pyindex 3014c7ca..9fec426e 100644--- a/server/tests/test_deployment_canary_listener.py+++ b/server/tests/test_deployment_canary_listener.py@@ -55,7 +55,7 @@ def fresh_canary_registry(monkeypatch):"""from services.deployment import canary_registry- monkeypatch.setattr(canary_registry, "_REGISTERED", set())+ monkeypatch.setattr(canary_registry, "_REGISTERED", {})return canary_registry@@ -139,7 +139,9 @@ class TestCanaryScope:async def test_disabled_when_flag_off(self, monkeypatch, fresh_canary_registry):from services.deployment.manager import DeploymentManager- fresh_canary_registry.register_canary_trigger_type("webhookTrigger")+ fresh_canary_registry.register_canary_trigger_type(+ "webhookTrigger", "com.machinaos.webhook.received"+ )class _Off:event_framework_enabled = False@@ -170,8 +172,12 @@ class TestCanaryScope:async def test_enabled_when_registered_and_flag_on(self, monkeypatch, fresh_canary_registry):from services.deployment.manager import DeploymentManager- fresh_canary_registry.register_canary_trigger_type("webhookTrigger")- fresh_canary_registry.register_canary_trigger_type("chatTrigger")+ fresh_canary_registry.register_canary_trigger_type(+ "webhookTrigger", "com.machinaos.webhook.received"+ )+ fresh_canary_registry.register_canary_trigger_type(+ "chatTrigger", "com.machinaos.chat.message.received"+ )class _On:event_framework_enabled = True@@ -307,8 +313,13 @@ class TestStartCanaryListener:# EventTriggerKind derived via _trigger_kind_for (strips "Trigger" /# "Receive" suffix) — NOT hardcoded "webhook".assert attrs_by_name["EventTriggerKind"] == "webhook"- # EventType comes from event_waiter.TRIGGER_REGISTRY['webhookTrigger'].- assert attrs_by_name["EventType"] == "webhook_received"+ # EventType MUST be the CloudEvents reverse-DNS string the+ # producer puts on outgoing envelopes (registered via+ # canary_registry). dispatch.emit's Visibility query substitutes+ # event.type into the EventType filter — if the SA carries the+ # legacy snake_case event_waiter string instead, the query+ # never matches and no signal reaches the listener.+ assert attrs_by_name["EventType"] == "com.machinaos.webhook.received"@pytest.mark.asyncioasync def test_chat_trigger_uses_chat_kind_in_search_attrs(self, monkeypatch):@@ -345,7 +356,9 @@ class TestStartCanaryListener:sa = recorded[0]["search_attributes"]attrs_by_name = {pair.key.name: pair.value for pair in sa}assert attrs_by_name["EventTriggerKind"] == "chat"- assert attrs_by_name["EventType"] == "chat_message_received"+ # CloudEvents reverse-DNS — see test_search_attributes_include_event_workflow_id+ # for the full rationale.+ assert attrs_by_name["EventType"] == "com.machinaos.chat.message.received"@pytest.mark.asyncioasync def test_returns_none_when_temporal_not_connected(self, monkeypatch):@@ -516,6 +529,139 @@ class TestCancelCanaryListeners:# ---------------------------------------------------------------------------+class TestCancelSweepsStuckNodeStatuses:+ """Regression: ``DeploymentManager.cancel`` must broadcast a full+ status cleanup so the FE doesn't leave downstream nodes glowing+ forever after the deployment goes down.++ Pre-fix the cancel only reset trigger nodes (cron + listener) to+ "idle". Downstream agents/tools/actions that were mid-execute when+ cancel hit stayed in "executing" status on every connected client+ because no sweep was issued. Also the toolbar Start/Stop indicator+ stuck at ``executing=True`` because no terminal+ ``workflow_run_ended`` / ``update_workflow_status(executing=False)``+ was emitted.+ """++ def test_cancel_source_calls_stuck_node_sweep_with_include_waiting(self):+ """Source-introspection: ``cancel`` must call+ ``_clear_stuck_node_statuses(..., include_waiting=True)`` so+ every node currently broadcast as ``executing`` OR ``waiting``+ for this deployment goes back to idle on the FE. Without+ ``include_waiting=True`` non-firing trigger siblings (and any+ other ``waiting`` indicators outside the cron/listener buckets+ the manager owns directly) stay glowing."""+ import inspect++ from services.deployment.manager import DeploymentManager++ src = inspect.getsource(DeploymentManager.cancel)+ assert "_clear_stuck_node_statuses" in src, (+ "DeploymentManager.cancel no longer sweeps stuck node "+ "statuses. Downstream nodes mid-execute at cancel-time "+ "will stay glowing on FE forever. Restore the "+ "``_broadcaster._clear_stuck_node_statuses(workflow_id, "+ "include_waiting=True)`` call after the trigger resets."+ )+ assert "include_waiting=True" in src, (+ "DeploymentManager.cancel must pass include_waiting=True to "+ "the stuck-node sweep — explicit user-cancel is the "+ "'every indicator goes quiet' signal. Without it, sibling "+ "trigger nodes (or other waiting indicators) outlive the "+ "deployment."+ )++ def test_cancel_source_broadcasts_terminal_workflow_status(self):+ """Source-introspection: ``cancel`` must emit a final+ ``update_workflow_status(executing=False, workflow_id=...)`` so+ the toolbar Start/Stop indicator reflects the cancel. The+ run-counter eviction in ``workflow_run_ended`` can race against+ in-flight child runs that already incremented the counter."""+ import inspect++ from services.deployment.manager import DeploymentManager++ src = inspect.getsource(DeploymentManager.cancel)+ assert "update_workflow_status" in src, (+ "DeploymentManager.cancel must broadcast "+ "``update_workflow_status(executing=False, workflow_id=...)`` "+ "so the FE toolbar Start/Stop indicator goes quiet after a "+ "deployment is cancelled."+ )+ assert "executing=False" in src, (+ "DeploymentManager.cancel must pass executing=False to the "+ "terminal workflow_status broadcast. Anything else leaves "+ "the toolbar showing the deployment as still active."+ )++ @pytest.mark.asyncio+ async def test_cancel_runtime_calls_sweep_and_terminal_broadcast(self):+ """Runtime smoke: stub the broadcaster + state + trigger_manager+ and assert cancel actually invokes the sweep + terminal status+ broadcast with the right args."""+ from services.deployment.manager import DeploymentManager+ from services.deployment.state import DeploymentState++ sweep_calls: List[Dict[str, Any]] = []+ status_calls: List[Dict[str, Any]] = []++ broadcaster = MagicMock()+ broadcaster.update_node_status = AsyncMock()+ broadcaster.update_workflow_status = AsyncMock(+ side_effect=lambda **kw: status_calls.append(kw),+ )+ broadcaster._clear_stuck_node_statuses = AsyncMock(+ side_effect=lambda workflow_id, include_waiting=False: sweep_calls.append({+ "workflow_id": workflow_id,+ "include_waiting": include_waiting,+ }) or 0,+ )++ database = MagicMock()+ mgr = DeploymentManager(+ database=database,+ execute_workflow_fn=AsyncMock(return_value={"success": True}),+ store_output_fn=AsyncMock(),+ broadcaster=broadcaster,+ )+ # Seed deployment state — the cancel path bails early if+ # the workflow isn't in self._deployments.+ mgr._deployments["wf-1"] = DeploymentState(+ deployment_id="deploy_wf-1",+ workflow_id="wf-1",+ is_running=True,+ nodes=[{"id": "n1", "type": "aiAgent"}],+ edges=[],+ session_id="sess",+ )++ # Patch _cancel_canary_listeners / _cancel_canary_cron_schedules+ # to be no-ops so the test focuses on the sweep + terminal+ # broadcast contract.+ mgr._cancel_canary_listeners = AsyncMock(return_value=0)+ mgr._cancel_canary_cron_schedules = AsyncMock(return_value=0)++ result = await mgr.cancel("wf-1")+ assert result["success"] is True++ # Sweep ran once with include_waiting=True (every indicator+ # goes quiet on explicit user cancel).+ assert len(sweep_calls) == 1, (+ f"Expected one stuck-node sweep on cancel, got "+ f"{len(sweep_calls)}: {sweep_calls!r}"+ )+ assert sweep_calls[0] == {+ "workflow_id": "wf-1",+ "include_waiting": True,+ }++ # Terminal executing=False broadcast for the toolbar.+ assert {"executing": False, "workflow_id": "wf-1"} in status_calls, (+ f"Expected update_workflow_status(executing=False, "+ f"workflow_id='wf-1') on cancel; got {status_calls!r}"+ )++class TestNoInstanceStateForCanaryListeners:"""Lock the architectural decision: DeploymentManager does NOT keepa Python dict of canary listener handles or ids. Temporal's Visibilitydiff --git a/server/tests/test_event_framework_phase_a.py b/server/tests/test_event_framework_phase_a.pyindex 3149b252..eb980bc8 100644--- a/server/tests/test_event_framework_phase_a.py+++ b/server/tests/test_event_framework_phase_a.py@@ -32,6 +32,7 @@ from __future__ import annotationsimport sysimport typesfrom datetime import timedelta+from typing import Any, Dict, Listfrom unittest.mock import MagicMockimport pytest@@ -425,3 +426,105 @@ class TestA8EmitEventActivity:assert result["delivered"] is Falseassert "error" in result+++class TestTriggerOutputPersistenceForTemplateResolution:+ """Wave 13: the canary path's pre-executed trigger nodes must+ persist their output to the workflow output store, otherwise+ :class:`services.parameter_resolver.ParameterResolver` can't see+ them and downstream ``{{triggerNode.field}}`` templates silently+ resolve to empty.++ Pre-fix MachinaWorkflow's pre-executed handler set+ ``outputs[node_id]`` in WORKFLOW MEMORY only — never persisted.+ The legacy ``DeploymentManager._execute_from_trigger`` path called+ ``_store_output(trigger_node_id, "output_0", trigger_output)``+ before executing downstream. The fix adds a+ ``store_node_output_activity`` call inside MachinaWorkflow.run's+ pre-executed loop that mirrors that persist.+ """++ def test_activity_exists_and_is_async(self):+ import inspect++ from services.temporal.activities import store_node_output_activity++ assert inspect.iscoroutinefunction(store_node_output_activity), (+ "store_node_output_activity must be async — workflow.execute_activity "+ "awaits it."+ )++ def test_machina_workflow_calls_persist_activity_for_pre_executed(self):+ """Source-introspection regression: MachinaWorkflow.run's+ pre-executed handler must call store_node_output_activity so+ ParameterResolver can read the trigger output back."""+ import inspect++ from services.temporal.workflow import MachinaWorkflow++ src = inspect.getsource(MachinaWorkflow.run)++ assert "store_node_output_activity" in src, (+ "MachinaWorkflow.run no longer schedules the "+ "store_node_output_activity for pre-executed trigger nodes. "+ "Without this persist, the legacy "+ "DeploymentManager._execute_from_trigger contract "+ "(``_store_output(trigger_node_id, 'output_0', ...)``) is "+ "missing on the canary path and ``{{triggerNode.field}}`` "+ "templates resolve to empty."+ )+ # Sanity: the persist guard skips non-firing siblings (their+ # _trigger_output is ``{not_triggered: True}``).+ assert "not_triggered" in src, (+ "MachinaWorkflow.run must skip the persist for non-firing "+ "trigger siblings (``_trigger_output={'not_triggered': True}``). "+ "Otherwise sibling triggers in the same deployment write empty "+ "outputs that template references could pick up by mistake."+ )++ @pytest.mark.asyncio+ async def test_store_activity_writes_three_handles(self, monkeypatch):+ """The activity must write to every standard output handle —+ downstream edges target different handle names (``output_main``+ on agent nodes, ``output_0`` on trigger nodes, ``output_top`` on+ legacy code paths). Writing only one would miss the others."""+ from services.temporal.activities import store_node_output_activity++ calls: List[Dict[str, Any]] = []++ class _StubWorkflowService:+ async def store_node_output(self, session_id, node_id, output_name, data):+ calls.append({+ "session_id": session_id,+ "node_id": node_id,+ "output_name": output_name,+ "data": data,+ })++ class _StubContainer:+ def workflow_service(self):+ return _StubWorkflowService()++ from core import container as container_mod+ monkeypatch.setattr(container_mod, "container", _StubContainer())++ await store_node_output_activity({+ "node_id": "trig-1",+ "session_id": "sess-1",+ "result": {"message": "hello", "session_id": "sess-1"},+ })++ # All three standard handles get the data — same write pattern+ # NodeExecutor uses for non-pre-executed nodes.+ output_names = {c["output_name"] for c in calls}+ assert output_names == {"output_main", "output_top", "output_0"}, (+ f"Expected writes to output_main + output_top + output_0; "+ f"got {sorted(output_names)!r}. Missing handles cause "+ f"downstream template resolution to fail when the edge "+ f"targets a handle that wasn't written."+ )+ # Same node_id + session_id + data on every write.+ assert {c["node_id"] for c in calls} == {"trig-1"}+ assert {c["session_id"] for c in calls} == {"sess-1"}+ for c in calls:+ assert c["data"] == {"message": "hello", "session_id": "sess-1"}diff --git a/server/tests/test_polling_trigger_workflow.py b/server/tests/test_polling_trigger_workflow.pyindex 58438210..c8a65ce8 100644--- a/server/tests/test_polling_trigger_workflow.py+++ b/server/tests/test_polling_trigger_workflow.py@@ -157,9 +157,53 @@ class TestAsPollActivity:new_ids = {e["id"] for e in result["events"]}assert new_ids == {"c", "d"}- # seen_ids is the union of prior + current.+ # OOM fix: seen_ids is bounded by ``current`` — items the+ # provider no longer reports drop out. Here ``current`` still+ # includes a/b/c/d so all four stay.assert set(result["seen_ids"]) == {"a", "b", "c", "d"}+ @pytest.mark.asyncio+ async def test_seen_ids_bounded_by_current_window_no_unbounded_growth(self):+ """OOM regression guard: when items drop off the provider's+ ``current`` window (archived / aged out), they must drop out of+ ``seen_ids`` too. Pre-fix this returned ``prior_seen | current``+ which grew forever; a Gmail trigger polling every 60s with ~100+ emails/day would accumulate ~36K IDs in a year (~1.4MB just for+ the seen set, serialised through Temporal payload every cycle)."""+ # Provider only reports {e, f} as currently visible — a/b/c/d+ # have aged out (e.g. archived in Gmail, marked read+filter is+ # ``is:unread``, paged off the latest-N window, etc.).+ cls = _make_fake_polling_class("test.bound", current_ids={"e", "f"})+ activity_fn = cls.as_poll_activity()+ inner = self._underlying(activity_fn)++ result = await inner({+ "node_id": "n1",+ "params": {},+ # Carry-forward from an earlier cycle.+ "seen_ids": ["a", "b", "c", "d", "e"],+ "baseline_only": False,+ })++ # Only the genuinely new id (``f``) emits — ``e`` was already+ # in prior_seen.+ new_ids = {ev["id"] for ev in result["events"]}+ assert new_ids == {"f"}, (+ "Only items in ``current`` that weren't in ``prior_seen`` "+ "should emit."+ )+ # And the returned seen_ids must NOT contain a/b/c/d — those+ # are gone from the provider's window. If they linger, every+ # subsequent cycle pays Temporal-payload cost for IDs that will+ # never matter again, and the set grows without bound.+ assert set(result["seen_ids"]) == {"e", "f"}, (+ f"seen_ids must be bounded by the provider's current window "+ f"(got {sorted(result['seen_ids'])!r}). Old IDs that have "+ f"dropped off the provider must NOT be carried forward — "+ f"pre-fix this returned ``list(prior_seen | current)`` and "+ f"leaked unboundedly."+ )+@pytest.mark.asyncioasync def test_detail_payload_id_fallback(self):"""When fetch_detail returns a payload without 'id', the@@ -216,6 +260,71 @@ class TestAsPollActivity:# ---------------------------------------------------------------------------+class TestPollCoroutineSeenSetBounded:+ """OOM regression guard for the legacy poll coroutine path+ (``_build_poll_coroutine``).++ Pre-fix the coroutine maintained ``seen: Set[str]`` that grew on+ every emit via ``seen.add(msg_id)`` with no eviction — long-running+ pollers (Gmail with the default 60s interval) accumulated tens of+ thousands of entries over weeks/months. The fix rebases ``seen``+ to ``current`` at the end of every cycle so items that drop off+ the provider's window drop out of ``seen`` too.+ """++ def test_coroutine_source_rebases_seen_to_current(self):+ """Source-introspection lock: ``_build_poll_coroutine`` must+ contain ``seen = set(current)`` (or equivalent rebase) and must+ NOT contain ``seen.add(`` inside the new-id emit loop. Driving+ the live coroutine through cycles is fragile because the+ per-class ``poll_interval_clamp`` enforces a 10s minimum+ sleep — and the bug is in single-line state mutation that+ source introspection catches reliably."""+ import inspect++ from services.plugin import PollingTriggerNode++ # Build a coroutine on a real subclass — we only need its+ # source, not its execution.+ class _T(PollingTriggerNode, abstract=True):+ type = "introspect.bound"+ display_name = "Introspect"++ async def setup_service(self, params): return MagicMock()+ async def fetch_ids(self, service, params): return set()+ async def fetch_detail(self, service, msg_id, params): return {"id": msg_id}+ async def post_emit(self, service, msg_id, params): pass++ # _build_poll_coroutine returns a closure; introspect the+ # MRO-resolved method body instead. Strip comments and+ # docstrings so the regex only sees executable code.+ raw = inspect.getsource(PollingTriggerNode._build_poll_coroutine)+ code_only = "\n".join(+ line.split("#", 1)[0] # drop trailing/inline comments+ for line in raw.splitlines()+ )++ # The fix: rebase seen to the provider's current snapshot at+ # the end of every cycle.+ assert "seen = set(current)" in code_only, (+ "OOM fix regressed: _build_poll_coroutine no longer rebases "+ "``seen`` to ``current`` at end of cycle. Pre-fix it called "+ "``seen.add(msg_id)`` per emit with no eviction; old IDs "+ "accumulated forever (~36K entries in a year for a 60s "+ "poller). Restore the ``seen = set(current)`` rebase after "+ "the new-id emit loop."+ )++ # And the anti-pattern (per-emit unbounded add) is gone from+ # executable code. Comments / docstrings that REFERENCE the+ # historical bug are fine (and useful for future maintainers).+ assert "seen.add(" not in code_only, (+ "OOM fix regressed: _build_poll_coroutine reintroduced "+ "``seen.add(msg_id)``. The rebase-to-current pattern makes "+ "this redundant AND unbounded — drop it."+ )++class TestPollingTriggerWorkflowBody:"""The workflow's signal/sleep/spawn loop runs deterministically."""@@ -242,6 +351,11 @@ class TestPollingTriggerWorkflowBody:from temporalio import workflow as temporal_workflowasync def fake_execute_activity(name, payload, **kwargs):+ # Trigger-status broadcasts (idle/waiting around each spawn)+ # are side-effect-only; they're driven by the workflow body+ # but don't consume the scripted poll-cycle responses.+ if name == "broadcast_trigger_status_activity":+ return Nonereturn next(responses)sleep_calls: List[Any] = []@@ -312,6 +426,10 @@ class TestPollingTriggerWorkflowBody:call_count = [0]async def fake_execute_activity(name, payload, **kwargs):+ # Trigger-status broadcasts are side-effect-only; don't+ # count them against the poll-cycle script.+ if name == "broadcast_trigger_status_activity":+ return Nonecall_count[0] += 1if call_count[0] == 2: # second call failsraise RuntimeError("transient API timeout")diff --git a/server/tests/test_task_trigger_canary_producer.py b/server/tests/test_task_trigger_canary_producer.pyindex 85fec1e1..d655a98f 100644--- a/server/tests/test_task_trigger_canary_producer.py+++ b/server/tests/test_task_trigger_canary_producer.py@@ -1,11 +1,12 @@-"""Wave 12 C1 rollout #2: taskTrigger producer dual-emit invariant.+"""taskTrigger producer canary-emit invariant.-Locks the contract: ``nodes.agent._events._broadcast_task_event`` must-route delegated-task completion to BOTH the legacy-``broadcaster.send_custom_event`` (which dispatches via event_waiter)-AND the Temporal-durable ``services.events.dispatch.emit`` path. Without-the second call, canary-enabled taskTrigger consumers miss every-child-agent completion.+Locks the contract: ``nodes.agent._events._broadcast_task_event`` routes+delegated-task completion through the canary CloudEvents path+(:func:`services.events.dispatch.emit`) ONLY. The legacy+``broadcaster.send_custom_event`` path (which internally dispatched to+``event_waiter``) was removed in Wave 13 — taskTrigger is canary-registered,+the deployment manager skips ``setup_event_trigger``, and the legacy+collector has zero consumers.Cross-cutting factory: this rollout uses ``WorkflowEvent.task_completed``which stays central per RFC §6.4 (parametrised by task_id + agent +@@ -19,8 +20,8 @@ import inspectimport reimport sysimport types-from typing import Any, Dict, List-from unittest.mock import AsyncMock, MagicMock+from typing import Any, List+from unittest.mock import MagicMockimport pytest@@ -37,54 +38,44 @@ _SEND_CUSTOM_EVENT_PATTERN = re.compile(r"send_custom_event\s*\(")_EVENTS_EMIT_PATTERN = re.compile(r"\bemit\s*\(")-class TestTaskTriggerProducerDualEmit:- """Producer wrapper emits BOTH legacy and Temporal envelopes."""+class TestTaskTriggerProducerCanaryEmit:+ """Producer wrapper emits via the canary CloudEvents path only."""def test_broadcast_helper_is_async(self):from nodes.agent._events import _broadcast_task_eventassert inspect.iscoroutinefunction(_broadcast_task_event)- def test_helper_routes_both_legacy_and_temporal(self):+ def test_helper_uses_canary_path_only(self):from nodes.agent import _eventssrc = inspect.getsource(_events._broadcast_task_event)- assert _SEND_CUSTOM_EVENT_PATTERN.search(src), (- "_broadcast_task_event must still call "- "broadcaster.send_custom_event(_LEGACY_EVENT_TYPE, payload) "- "so canary-flag-off deployments keep their existing dispatch."- )assert _EVENTS_EMIT_PATTERN.search(src), ("_broadcast_task_event must call "- "services.events.dispatch.emit(envelope, ...) for the "- "Temporal-durable canary path. Without it, taskTrigger "- "TriggerListenerWorkflows receive nothing when the "- "event_framework_enabled flag is on."+ "services.events.dispatch.emit(envelope, ...) — the canary "+ "CloudEvents path Signals running TriggerListenerWorkflow "+ "consumers AND broadcasts to FE on the task_completed wire key."+ )+ assert not _SEND_CUSTOM_EVENT_PATTERN.search(src), (+ "_broadcast_task_event must NOT call "+ "broadcaster.send_custom_event — taskTrigger is canary-registered "+ "and the legacy event_waiter dispatch path (which "+ "send_custom_event drives internally) has zero consumers "+ "(removed in Wave 13).")@pytest.mark.asyncio- async def test_completed_event_routes_both_paths(self, monkeypatch):+ async def test_completed_event_emits_canary_envelope(self, monkeypatch):from nodes.agent import _eventsfrom services.events import dispatch as dispatch_mod- custom_calls: List[Dict[str, Any]] = []emit_calls: List[Any] = []- broadcaster = MagicMock()-- async def fake_send_custom_event(event_type, payload):- custom_calls.append({"event_type": event_type, "payload": payload})-- broadcaster.send_custom_event = fake_send_custom_event-async def fake_emit(event, **kwargs):emit_calls.append({"event": event, **kwargs})return event- # Patch the get_status_broadcaster singleton.- from services import status_broadcaster as sb- monkeypatch.setattr(sb, "get_status_broadcaster", lambda: broadcaster)monkeypatch.setattr(dispatch_mod, "emit", fake_emit)await _events.broadcast_agent_task_completed(@@ -96,40 +87,31 @@ class TestTaskTriggerProducerDualEmit:result="done",)- # Legacy path: wire key = task_completed, payload carries- # the existing fields the taskTrigger filter reads.- assert len(custom_calls) == 1- assert custom_calls[0]["event_type"] == "task_completed"- legacy_payload = custom_calls[0]["payload"]- assert legacy_payload["task_id"] == "task-1"- assert legacy_payload["status"] == "completed"- assert legacy_payload["result"] == "done"-- # Temporal path: envelope.type discriminates on succeeded vs- # failed; subject is the task_id; wire_routing_key preserved.+ # Single type (``.completed``) so the listener's EventType SA+ # matches via dispatch.emit's Visibility query. Status+ # discrimination lives in data.status.assert len(emit_calls) == 1envelope = emit_calls[0]["event"]- assert envelope.type == "com.machinaos.agent.task.succeeded"+ assert envelope.type == "com.machinaos.agent.task.completed"assert envelope.subject == "task-1"+ assert envelope.data["status"] == "completed"+ assert envelope.data["result"] == "done"assert emit_calls[0]["wire_routing_key"] == "task_completed"@pytest.mark.asyncio- async def test_failed_event_routes_both_paths_with_error_type(self, monkeypatch):- """Type discriminates between succeeded/failed; subject still task_id."""+ async def test_failed_event_shares_type_with_status_discriminator(self, monkeypatch):+ """Success and failure share a single CloudEvents type so the+ listener's EventType Search Attribute matches both branches;+ ``status='error'`` lives in the payload."""from nodes.agent import _eventsfrom services.events import dispatch as dispatch_modemit_calls: List[Any] = []- broadcaster = MagicMock()- broadcaster.send_custom_event = AsyncMock()-async def fake_emit(event, **kwargs):emit_calls.append(event)return event- from services import status_broadcaster as sb- monkeypatch.setattr(sb, "get_status_broadcaster", lambda: broadcaster)monkeypatch.setattr(dispatch_mod, "emit", fake_emit)await _events.broadcast_agent_task_failed(@@ -143,7 +125,7 @@ class TestTaskTriggerProducerDualEmit:assert len(emit_calls) == 1envelope = emit_calls[0]- # Status='error' maps to the .failed type variant per the- # central factory's discrimination.- assert envelope.type == "com.machinaos.agent.task.failed"+ assert envelope.type == "com.machinaos.agent.task.completed"assert envelope.subject == "task-2"+ assert envelope.data["status"] == "error"+ assert envelope.data["error"] == "timeout"diff --git a/server/tests/test_telegram_trigger_canary_producer.py b/server/tests/test_telegram_trigger_canary_producer.pyindex 2d5bf133..b3317df9 100644--- a/server/tests/test_telegram_trigger_canary_producer.py+++ b/server/tests/test_telegram_trigger_canary_producer.py@@ -1,12 +1,11 @@-"""Wave 12 C1 rollout #3: telegramReceive producer dual-emit invariant.+"""telegramReceive producer canary-emit invariant.Locks the contract: ``nodes.telegram._events.dispatch_telegram_message_received``-must route incoming Telegram messages to BOTH the legacy-``event_waiter.dispatch`` and the Temporal-durable-``services.events.dispatch.emit`` path. Without the second call, a-canary-enabled telegramReceive deployment silently misses every-incoming message — the TriggerListenerWorkflow has no other event-source.+routes incoming Telegram messages through the canary CloudEvents path+(:func:`services.events.dispatch.emit`) ONLY. The legacy+``event_waiter.dispatch`` path was removed in Wave 13 — telegramReceive+is canary-registered, the deployment manager skips+``setup_event_trigger``, and the legacy collector has zero consumers.Same regex-introspection + runtime smoke pattern used for thewebhookTrigger / chatTrigger / taskTrigger producers in this series.@@ -18,8 +17,8 @@ import inspectimport reimport sysimport types-from typing import Any, Dict, List-from unittest.mock import AsyncMock, MagicMock+from typing import Any, List+from unittest.mock import MagicMockimport pytest@@ -37,51 +36,44 @@ _EVENT_WAITER_DISPATCH_PATTERN = re.compile(r"event_waiter\.dispatch\s*\(")_EVENTS_EMIT_PATTERN = re.compile(r"\bemit\s*\(")-class TestTelegramProducerDualEmit:- """Producer wrapper emits BOTH legacy and Temporal envelopes."""+class TestTelegramProducerCanaryEmit:+ """Producer wrapper emits via the canary CloudEvents path only."""def test_dispatcher_is_async(self):from nodes.telegram._events import dispatch_telegram_message_receivedassert inspect.iscoroutinefunction(dispatch_telegram_message_received)- def test_dispatcher_routes_both_legacy_and_temporal(self):+ def test_dispatcher_uses_canary_path_only(self):from nodes.telegram import _eventssrc = inspect.getsource(_events.dispatch_telegram_message_received)- assert _EVENT_WAITER_DISPATCH_PATTERN.search(src), (- "dispatch_telegram_message_received must still call "- "event_waiter.dispatch(_MESSAGE_LEGACY_EVENT_TYPE, ...) for "- "the in-process waiter path. Without it, canary-flag-off "- "deployments lose their existing telegram dispatch."- )assert _EVENTS_EMIT_PATTERN.search(src), ("dispatch_telegram_message_received must call "- "services.events.dispatch.emit(envelope, ...) for the "- "Temporal-durable canary path. Without it, telegramReceive "- "TriggerListenerWorkflows receive nothing when the "- "event_framework_enabled flag is on."+ "services.events.dispatch.emit(envelope, ...) — the canary "+ "CloudEvents path Signals running TriggerListenerWorkflow "+ "consumers AND broadcasts to FE on the "+ "telegram_message_received wire key."+ )+ assert not _EVENT_WAITER_DISPATCH_PATTERN.search(src), (+ "dispatch_telegram_message_received must NOT call "+ "event_waiter.dispatch — telegramReceive is canary-registered "+ "and the legacy collector path has zero consumers (removed "+ "in Wave 13).")@pytest.mark.asyncio- async def test_runtime_dual_emit_calls_both_paths(self, monkeypatch):+ async def test_runtime_emits_canary_envelope(self, monkeypatch):from nodes.telegram import _events- from services import event_waiterfrom services.events import dispatch as dispatch_mod- legacy_calls: List[Dict[str, Any]] = []emit_calls: List[Any] = []- def fake_dispatch(event_type, data):- legacy_calls.append({"event_type": event_type, "data": data})- return 0-async def fake_emit(event, **kwargs):emit_calls.append({"event": event, **kwargs})return event- monkeypatch.setattr(event_waiter, "dispatch", fake_dispatch)monkeypatch.setattr(dispatch_mod, "emit", fake_emit)result = await _events.dispatch_telegram_message_received({@@ -91,11 +83,7 @@ class TestTelegramProducerDualEmit:"timestamp": "2026-05-14T00:00:00",})- assert result == 0-- assert len(legacy_calls) == 1- assert legacy_calls[0]["event_type"] == "telegram_message_received"- assert legacy_calls[0]["data"]["chat_id"] == 12345+ assert result is Noneassert len(emit_calls) == 1envelope = emit_calls[0]["event"]@@ -118,6 +106,6 @@ class TestTelegramPluginSelfRegistersCanary:assert canary_registry.is_canary_trigger_type("telegramReceive"), ("Importing nodes.telegram should call "- "register_canary_trigger_type('telegramReceive') — see "+ "register_canary_trigger_type('telegramReceive', ...) — see ""the __init__.py bottom section.")diff --git a/server/tests/test_trigger_listener_workflow.py b/server/tests/test_trigger_listener_workflow.pyindex 8095fbdb..03ba0a25 100644--- a/server/tests/test_trigger_listener_workflow.py+++ b/server/tests/test_trigger_listener_workflow.py@@ -277,6 +277,10 @@ class TestSpawnChildRun:recorded.append({"name": workflow_name, **kwargs})return MagicMock()+ async def fake_execute_activity(*args, **kwargs):+ # Status-broadcast activities — no-op for the spawn test.+ return None+from temporalio import workflow as temporal_workflowmonkeypatch.setattr(@@ -284,6 +288,11 @@ class TestSpawnChildRun:"start_child_workflow",fake_start_child_workflow,)+ monkeypatch.setattr(+ temporal_workflow,+ "execute_activity",+ fake_execute_activity,+ )wf = tlw.TriggerListenerWorkflow()event = {@@ -327,6 +336,9 @@ class TestSpawnChildRun:async def boom(*args, **kwargs):raise RuntimeError("temporal unreachable")+ async def fake_execute_activity(*args, **kwargs):+ return None+from temporalio import workflow as temporal_workflowmonkeypatch.setattr(@@ -334,6 +346,11 @@ class TestSpawnChildRun:"start_child_workflow",boom,)+ monkeypatch.setattr(+ temporal_workflow,+ "execute_activity",+ fake_execute_activity,+ )wf = tlw.TriggerListenerWorkflow()event = {"id": "evt-1", "type": "x", "data": {}}diff --git a/server/tests/test_whatsapp_trigger_canary_producer.py b/server/tests/test_whatsapp_trigger_canary_producer.pyindex 2fdaeb1c..e2c3f7c9 100644--- a/server/tests/test_whatsapp_trigger_canary_producer.py+++ b/server/tests/test_whatsapp_trigger_canary_producer.py@@ -92,8 +92,13 @@ class TestWhatsappProducerDualEmit:},)- # WS broadcasts still fire (legacy raw + typed sibling).- assert broadcaster.broadcast.await_count == 2+ # Legacy raw frame still goes out so the existing FE message-list+ # handler at WebSocketContext.tsx (which reads ``data.*``)+ # keeps working until the Wave 12 D4 follow-up migrates it to+ # envelope-shape. The duplicate typed-envelope WS broadcast was+ # dropped in Wave 13 — dispatch.emit broadcasts the envelope on+ # the same wire key once.+ assert broadcaster.broadcast.await_count == 1# And the canary path fires exactly once.assert len(emit_calls) == 1@@ -132,10 +137,12 @@ class TestWhatsappProducerDualEmit:{"chat_id": "[email protected]", "text": "hi"},)- # WS broadcasts still fire for the observation channel.- assert broadcaster.broadcast.await_count == 2+ # Single legacy raw broadcast for the FE message-list+ # observation channel — duplicate typed-envelope sibling dropped+ # in Wave 13.+ assert broadcaster.broadcast.await_count == 1- # But NO canary fan-out (no trigger consumes outbound).+ # No canary fan-out (no trigger consumes outbound).assert emit_calls == []