Skip to content

Self-improving AI Employees for everyone

docs: reflect Wave 12.B daemon framework + Stripe OAuth refinements

trohitg55c0b274fec8
diff --git a/CLAUDE.md b/CLAUDE.md
index bd303090..a44bb550 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -25,7 +25,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire
| **[Pricing Service](./docs-internal/pricing_service.md)** | Centralized cost tracking for LLM tokens and API services (Twitter, Google Maps) with HTTPX event hooks |
| **[Proxy Service](./docs-internal/proxy_service.md)** | Residential proxy provider management with template-based URL formatting, health scoring, and transparent HTTP node injection |
| **[Email Service](./docs-internal/email_service.md)** | IMAP/SMTP integration via Himalaya CLI with EmailService orchestrator, provider presets, custom credential fallback, and polling triggers |
-| **[Stripe Service](./docs-internal/stripe_service.md)** | Stripe CLI integration — `stripeAction` (CLI pass-through) + `stripeReceive` (signed-webhook trigger). Reference plugin for the Wave 12 event framework AND the **CLI-managed-auth pattern**: browser OAuth driven by `stripe login --non-interactive` + auto-installed binary (`ensure_stripe_cli` from GitHub releases) + marker-token reuse of `auth_service.store_oauth_tokens` + generic `credential_catalogue_updated` broadcast — zero per-provider hardcoding in the frontend. |
+| **[Stripe Service](./docs-internal/stripe_service.md)** | Stripe CLI integration — `stripeAction` (CLI pass-through) + `stripeReceive` (signed-webhook trigger). Reference plugin for the Wave 12 event framework AND the **CLI-managed-auth pattern**: browser OAuth via `stripe login --non-interactive` + `--complete <url>` (URL extracted from `next_step` shell command via `shlex.split`) + auto-installed binary (`ensure_stripe_cli` from GitHub releases) + marker-token reuse of `auth_service.store_oauth_tokens` + CloudEvents-shaped `broadcast_credential_event("credential.oauth.connected" \| ".disconnected", provider=...)` — zero per-provider hardcoding in the frontend. Daemon stdout/stderr ingested via `ProcessService`'s `line_handler` callback (no log-file tailing); credential gate routed through `await self.has_credential()` so non-api-key auth (`is_logged_in()`) plugs in via subclass override. |
| **[CI/CD Pipeline](./docs-internal/ci_cd.md)** | GitHub Actions workflows, predeploy validation, release publishing, and composite setup action |
| **[Workflow Schema](./docs-internal/workflow-schema.md)** | JSON schema for workflows, edge handle conventions, config node architecture |
| **[Execution Engine Design](./docs-internal/DESIGN.md)** | Architecture patterns, design standards, and implementation details for the workflow execution engine |
diff --git a/docs-internal/node_creation.md b/docs-internal/node_creation.md
index 26800464..5769e227 100644
--- a/docs-internal/node_creation.md
+++ b/docs-internal/node_creation.md
@@ -152,17 +152,19 @@ reused:
)
```
-2. **Generic broadcast on state change.** Plugin emits the existing
- `credential_catalogue_updated` event — handled in
- [`WebSocketContext.tsx`](../client/src/contexts/WebSocketContext.tsx)
- line 671 since before Stripe shipped. The frontend invalidates
- the catalogue, refetches, and the modal re-renders. No
- per-provider broadcast type, no `case 'stripe_status'` —
+2. **CloudEvents-shaped broadcast on state change.** Plugin emits a
+ `WorkflowEvent` (CloudEvents v1.0) via the canonical helper —
+ wrapped under the existing `credential_catalogue_updated`
+ wire-format type. Same shape `save_api_key` / `twitter_logout` /
+ `google_logout` use; locked by
+ `tests/credentials/test_credential_broadcasts.py`. The frontend
+ invalidates the catalogue, refetches, and the modal re-renders.
+ No per-provider broadcast type, no `case 'stripe_status'` —
**zero node-specific code in the frontend**.
```python
- await get_status_broadcaster().broadcast(
- {"type": "credential_catalogue_updated"}
+ await get_status_broadcaster().broadcast_credential_event(
+ "credential.oauth.connected", provider="stripe", # or .disconnected on logout
)
```
diff --git a/docs-internal/plugin_system.md b/docs-internal/plugin_system.md
index 779a209f..3840a1e6 100644
--- a/docs-internal/plugin_system.md
+++ b/docs-internal/plugin_system.md
@@ -1001,21 +1001,32 @@ The shared dispatch path lives in `routers/webhook.py` — it consults
For plugins that wrap a long-lived CLI tool or SDK loop (Stripe CLI,
future GitHub-CLI / Cloudflare-Wrangler / etc.). Delegates lifecycle
to `ProcessService` (battle-tested PATHEXT-aware launching, kill_tree
-cleanup, log capture):
+cleanup, log capture, Terminal-tab broadcast). The base subscribes to
+ProcessService's per-line callback hook (`line_handler`), so plugins
+just provide the parser:
```python
+import shlex
from services.events import DaemonEventSource, WorkflowEvent
class StripeListenSource(DaemonEventSource):
type = "stripe.listen"
process_name = "stripe-listen"
- binary_name = "stripe"
+ binary_name = "" # see "binary_name" note below
workflow_namespace = "_stripe"
install_hint = "https://stripe.com/docs/stripe-cli#install"
credential = StripeCredential
def build_command(self, secrets: Dict) -> str:
- return f"stripe listen --print-secret --api-key {secrets['api_key']}"
+ # `shlex.quote` the binary so the path round-trips through
+ # ProcessService's POSIX-mode `shlex.split` unchanged.
+ binary = shlex.quote(str(stripe_cli_path() or "stripe"))
+ return f"{binary} listen --forward-to ... --print-secret"
+
+ async def has_credential(self) -> bool:
+ # Override the default ``secrets["api_key"]`` gate when auth lives
+ # outside MachinaOs (e.g. the CLI's own ~/.config/stripe/config.toml).
+ return is_logged_in()
def parse_line(self, stream: str, line: str) -> Optional[WorkflowEvent]:
# called per-line for both stdout and stderr
@@ -1025,8 +1036,19 @@ class StripeListenSource(DaemonEventSource):
```
The base provides `start()` / `stop()` / `restart()` / `status()` /
-`has_credential()` / lifecycle locking / pre-flight `shutil.which`
-check / log tailing.
+lifecycle locking / pre-flight `shutil.which` check (skip via
+`binary_name = ""` if the plugin resolves the binary itself).
+
+**Output ingestion**: `DaemonEventSource` registers `self._on_line` as
+the `line_handler` callback on `ProcessService.start()`. ProcessService
+runs the single `stream.readline()` loop per stdout/stderr; on each
+decoded line it writes the log file, broadcasts to the Terminal tab,
+and invokes the callback — which calls `parse_line(stream, line)`. No
+log-file tailing.
+
+**Credential gate**: `start()` consults `await self.has_credential()`
+before spawning. Default implementation tests `secrets["api_key"]`;
+subclasses override for non-api-key auth (Stripe → `is_logged_in()`).
### Webhook verifiers
@@ -1172,14 +1194,15 @@ project that publishes pre-built binaries via GitHub releases.
| HTTP webhook ingress | `routers/webhook.py` consults `WEBHOOK_SOURCES` registry | `register_webhook_source(MySource())` |
| Event dispatch into workflows | `event_waiter.dispatch(source.type, event)` from `WebhookSource.handle` | provider-specific `shape()` returning `WorkflowEvent` |
| Trigger waiting + filtering | `WebhookTriggerNode.build_filter` (CloudEvents glob) | optional `_extra_filter(params)` |
-| Daemon lifecycle | `DaemonEventSource.start/stop/restart` via `ProcessService` | `build_command(secrets)` + `parse_line(stream, line)` |
+| Daemon lifecycle | `DaemonEventSource.start/stop/restart` via `ProcessService` | `build_command(secrets)` + `parse_line(stream, line)` (subscribed via `ProcessService.start(line_handler=...)` — no log-file tailing) |
+| Daemon credential gate | `DaemonEventSource.start` consults `await self.has_credential()` before spawning | optional override when auth is non-api-key (Stripe → `is_logged_in()`); default tests `secrets["api_key"]` |
| Lifecycle WebSocket commands | `make_lifecycle_handlers(prefix, source, extra=…)` | provider-specific extra handlers |
| Status refresh on WS connect | `make_status_refresh(source, status_key, broadcast_type)` | nothing — auto-derived from source |
| CLI subprocess invocation | `run_cli_command(binary=…, argv=…, credential=…)` | nothing — credential injection is automatic |
| Credentials Modal panel | `server/config/credential_providers.json` (read by `services.credential_registry`) | one provider entry: name, category, color, `kind: "oauth"`, `icon_ref`, `status_hook`, `ws.{login,logout,status}` handler names, fields list, instructions string. Frontend modal renders it automatically — no React file edits. |
| AI tool surface | `services/ai.py` `DEFAULT_TOOL_NAMES` + `DEFAULT_TOOL_DESCRIPTIONS` | one row per dual-purpose ActionNode mapping `<nodeType>` → `<snake_case_of_node_type>` |
| Skill (LLM teaching markdown) | `server/skills/<agent>/<skill-name>/SKILL.md` (auto-discovered by `SkillLoader`) | the markdown itself, plus the linkage in `visuals.json` (`"<nodeType>": { ..., "skill": "<skill-name>" }`) |
-| Connection state surfaced to the modal (CLI-managed auth) | `auth_service.store_oauth_tokens(status_hook, "cli-managed", "cli-managed")` + the generic `credential_catalogue_updated` broadcast | one `_mark_logged_in` / `_mark_logged_out` helper pair calling `store_oauth_tokens` / `remove_oauth_tokens`; one `broadcast({"type": "credential_catalogue_updated"})` after every state change |
+| Connection state surfaced to the modal (CLI-managed auth) | `auth_service.store_oauth_tokens(provider, "cli-managed", "cli-managed")` + `StatusBroadcaster.broadcast_credential_event` (CloudEvents v1.0 envelope wrapping `WorkflowEvent`; locked by `tests/credentials/test_credential_broadcasts.py`) | `_mark_logged_in` / `_mark_logged_out` helper pair; one `broadcaster.broadcast_credential_event("credential.oauth.connected", provider="<id>")` after login and `…disconnected` after logout. Same shape Twitter / Google logout use. |
| Auto-install of an external CLI binary | `_install.py` with `ensure_<cli>_cli()` + GitHub-releases asset map | pinned `_VERSION` constant, `(system, machine) -> (asset, kind, member)` table, and an override on `DaemonEventSource.start()` that `await`s the helper before `super().start()` (also set `binary_name = ""` to skip the framework's `shutil.which` pre-check) |
### Tool / skill / visuals naming contract
diff --git a/docs-internal/stripe_service.md b/docs-internal/stripe_service.md
index c2c9656c..c3fe8a52 100644
--- a/docs-internal/stripe_service.md
+++ b/docs-internal/stripe_service.md
@@ -142,8 +142,14 @@ handle_stripe_login()
│ │
│ └── get_listen_source().start() ← daemon auto-starts
│
- └── _broadcast_catalogue_updated() ← generic broadcast
- broadcaster.broadcast({"type": "credential_catalogue_updated"})
+ └── _broadcast_credential_event("credential.oauth.connected")
+ → broadcaster.broadcast_credential_event(
+ "credential.oauth.connected", provider="stripe")
+ (CloudEvents v1.0 envelope wrapped under the
+ 'credential_catalogue_updated' wire-format type.
+ Same shape twitter_logout / google_logout /
+ save_api_key use; locked by
+ tests/credentials/test_credential_broadcasts.py.)
frontend's existing case-handler invalidates the catalogue
query → modal sees provider.stored = true → connection
indicator flips immediately
@@ -166,25 +172,32 @@ handle_stripe_logout()
StripeListenSource.start() (lock-protected, idempotent)
│
├── ensure_stripe_cli() ← override before super().start()
- ├── has_credential() → is_logged_in() ← override
- │ (file check on ~/.config/stripe/config.toml)
+ ├── has_credential() → is_logged_in() ← override of the daemon gate
+ │ (filesystem check on ~/.config/stripe/config.toml)
├── binary_name = "" ← framework PATH check skipped
├── ProcessService.start(
│ name="stripe-listen",
- │ command="<resolved-binary> listen
+ │ command="<shlex-quoted-binary> listen
│ --forward-to http://localhost:{port}/webhook/stripe
│ --print-secret", # no --api-key
│ workflow_id="_stripe_global",
│ working_directory={workspace}/_stripe,
+ │ line_handler=self._on_line, # ← per-line callback
│ )
- │ (CLI reads credentials from its config file)
+ │ (CLI reads credentials from its config file; binary path is
+ │ shlex.quote'd so Windows backslashes survive ProcessService's
+ │ POSIX-mode shlex.split round-trip)
│
- └── tail stderr.log
- on match r"whsec_[A-Za-z0-9_]+":
- auth_service.store_api_key("stripe_webhook_secret", …)
+ └── ProcessService loops `stream.readline()` per stdout/stderr,
+ decodes UTF-8, writes to stdout.log / stderr.log, broadcasts to
+ the Terminal tab, AND calls our line_handler:
+ _on_line(stream, line) → parse_line(stream, line)
+ on stderr match r"whsec_[A-Za-z0-9_]+":
+ auth_service.store_api_key("stripe_webhook_secret", …)
+ (No file-tailing; we hook into the same readline loop ProcessService
+ already runs.)
StripeListenSource.stop()
- ├── cancel tail task
└── ProcessService.stop("stripe-listen", "_stripe_global")
```
@@ -203,7 +216,8 @@ StripeListenSource.stop()
| `server/nodes/stripe/stripe_action.py` | `StripeActionNode` — pass-through over the CLI via `run_cli_command`. |
| `server/nodes/stripe/stripe_receive.py` | `StripeReceiveNode(WebhookTriggerNode)` — filter overrides + output reshape. |
| `server/services/events/__init__.py` | Public framework surface — exports every base class + helper. |
-| `server/services/events/daemon.py` | `DaemonEventSource` — supervises subprocess via `ProcessService`, tails logs. |
+| `server/services/events/daemon.py` | `DaemonEventSource` — supervises subprocess via `ProcessService`. Subscribes to ProcessService's per-line callback (`line_handler`) instead of re-tailing the on-disk log files. Credential gate is `await self.has_credential()` so non-api-key auth (Stripe → `is_logged_in()`) plugs in via subclass override. |
+| `server/services/process_service.py` | Spawns + supervises long-lived subprocesses. Loops `stream.readline()` per stdout/stderr, writes to `.log`, broadcasts to Terminal, and forwards each decoded line to the optional `line_handler` async callback (Wave 12.B addition for typed event-source subscribers — see `DaemonEventSource._on_line`). |
| `server/services/events/webhook.py` | `WebhookSource` + `WEBHOOK_SOURCES` registry + `register_webhook_source`. |
| `server/services/events/triggers.py` | `WebhookTriggerNode` + `BaseTriggerParams`. |
| `server/services/events/cli.py` | `run_cli_command` helper. |
@@ -251,8 +265,9 @@ Storage:
### `StripeListenSource(DaemonEventSource)`
Supervises `stripe listen` as a long-lived process. Inherits the
-full `DaemonEventSource` lifecycle (start / stop / restart / status /
-log tailing). The Stripe-specific overrides are minimal:
+full `DaemonEventSource` lifecycle (start / stop / restart / status)
+plus the per-line callback subscription via `ProcessService`'s
+`line_handler` hook. The Stripe-specific overrides are minimal:
| Method / attr | Purpose |
|---|---|
@@ -262,9 +277,9 @@ log tailing). The Stripe-specific overrides are minimal:
| `install_hint` | Surfaced in the "install failed" error path when the auto-installer can't reach GitHub releases. |
| `credential = StripeCredential` | Resolved by the framework before `build_command` is called. |
| `start()` (override) | `await ensure_stripe_cli()` then `super().start()`. Caches the resolved binary path so `build_command` (sync) can pick it up. |
-| `build_command(secrets)` | Returns `<resolved-binary> listen --forward-to … --print-secret` (no `--api-key` — CLI reads its own config file; binary path comes from `stripe_cli_path()` cache populated by `start()`). |
-| `has_credential()` (override) | Returns `is_logged_in()` — a filesystem check on `~/.config/stripe/config.toml`. The framework's auto-reconnect path uses this as the gate for restarting the daemon when a WS client connects. |
-| `parse_line(stream, line)` | On each stderr line, regex-matches `whsec_…`; on first match, persists the secret via `auth_service.store_api_key("stripe_webhook_secret", …)`. The Stripe daemon doesn't emit events itself — events arrive via the webhook receiver. |
+| `build_command(secrets)` | Returns `<shlex.quote'd-binary> listen --forward-to … --print-secret`. The binary path is `shlex.quote`d so it round-trips through `ProcessService`'s POSIX-mode `shlex.split` unchanged. No `--api-key`: CLI reads its own config file. |
+| `has_credential()` (override) | Returns `is_logged_in()` — a filesystem check on `~/.config/stripe/config.toml`. Consulted by `DaemonEventSource.start` (the credential gate) and by `make_status_refresh` on every WS-client connect (auto-reconnect). |
+| `parse_line(stream, line)` | Invoked once per decoded stdout/stderr line via `ProcessService`'s `line_handler` callback. On `whsec_…` match, persists the secret via `auth_service.store_api_key("stripe_webhook_secret", …)`. The Stripe daemon doesn't emit workflow events itself — they arrive via the webhook receiver. |
### `StripeWebhookSource(WebhookSource)`
@@ -413,11 +428,12 @@ the modal's Connect button are `stripe_login` and `stripe_logout`:
Background `_complete_login(binary, next_step)` flow:
-1. `run_cli_command([binary, "login", "--complete", next_step], timeout=600s)` blocks until OAuth completes.
-2. `is_logged_in()` flips true (the CLI wrote `_api_key` lines into `config.toml`).
-3. `_mark_logged_in()` writes `auth_service.store_oauth_tokens("stripe", "cli-managed", "cli-managed")`.
-4. `get_listen_source().start()` spawns the supervised `stripe listen` daemon (which captures the `whsec_…` from its banner).
-5. `_broadcast_catalogue_updated()` emits `{type: "credential_catalogue_updated"}` — the existing generic broadcast handler in `WebSocketContext.tsx` invalidates the catalogue, the modal refetches, and `provider.stored = true` for stripe flips the connection indicator.
+1. `next_step` from step 1 is a literal shell command (`stripe login --complete '<URL>'`); extract the auth URL with `shlex.split(next_step)[-1]` before passing it to `--complete`.
+2. `run_cli_command([binary, "login", "--complete", complete_url], timeout=600s)` blocks until OAuth completes.
+3. `is_logged_in()` flips true (the CLI wrote `_api_key` lines into `config.toml`).
+4. `_mark_logged_in()` writes `auth_service.store_oauth_tokens("stripe", "cli-managed", "cli-managed")`.
+5. `get_listen_source().start()` spawns the supervised `stripe listen` daemon; the `whsec_…` banner is captured via the `line_handler` callback.
+6. `_broadcast_credential_event("credential.oauth.connected")` emits a CloudEvents v1.0 envelope (`WorkflowEvent`) via `StatusBroadcaster.broadcast_credential_event`, wrapped under the `credential_catalogue_updated` wire-format type. The frontend invalidates the catalogue and `provider.stored = true` flips the connection indicator.
## AI tool surface (`stripe_action`)
@@ -569,15 +585,25 @@ without any provider-specific code in the catalogue handler. The
`auth_service.store_oauth_tokens` API doesn't validate the strings;
the marker exists purely to flip the existence check.
-### 2. The generic `credential_catalogue_updated` broadcast
+### 2. The CloudEvents-shaped credential broadcast
-After `_mark_logged_in()` / `_mark_logged_out()`, the plugin
-broadcasts:
+After `_mark_logged_in()` / `_mark_logged_out()`, the plugin emits a
+CloudEvents v1.0 envelope via the canonical helper:
-```json
-{ "type": "credential_catalogue_updated" }
+```python
+await get_status_broadcaster().broadcast_credential_event(
+ "credential.oauth.connected", provider="stripe", # or .disconnected on logout
+)
```
+`StatusBroadcaster.broadcast_credential_event` wraps a `WorkflowEvent`
+(from `services.events.envelope`) and ships it under the
+`credential_catalogue_updated` wire-format type — the same shape
+`save_api_key`, `delete_api_key`, `twitter_logout`, `google_logout`
+already use. The contract is locked by
+`tests/credentials/test_credential_broadcasts.py` (`inspect.getsource`
+introspection over each handler).
+
`WebSocketContext.tsx` already has a generic case for this event
(line 671 — predates Stripe). Its handler calls `invalidateCatalogue`
on the TanStack Query client; the catalogue refetches; the modal