Self-improving AI Employees for everyone
Email Service
IMAP/SMTP email integration via the Himalaya CLI. Supports any IMAP/SMTP provider: Gmail, Outlook/Office 365, Yahoo, iCloud, ProtonMail (Bridge), Fastmail, and custom/self-hosted servers. Three workflow nodes (emailSend, emailRead, emailReceive) with dual-purpose workflow + AI tool integration for send/read, and a polling trigger for receive.
Architecture
┌──────────────────────────────────────────────┐
│ EmailService (singleton) │
│ │
plugin ──────────► resolve_credentials(params) -> dict │
@Operation │ preset (email_providers.json) > stored │
(nodes/email/...) │ > stored API keys (AuthService) │
│ │
│ send(params) -> dict │
│ read(params) -> dict (7 operations) │
│ poll_ids(creds, folder) -> set[str] │
│ fetch_detail(creds, msg_id, folder) -> dict │
│ resolve_poll_params(params) -> dict │
└───────────┬───────────────────┬──────────────┘
│ │
▼ ▼
HimalayaService AuthService
(CLI wrapper) (get_api_key,
│ store_api_key)
│ subprocess
▼
`himalaya` CLI binary
generates temp TOML config,
calls IMAP/SMTP backends
EmailService and HimalayaService are both singletons living in the
self-contained email plugin folder (server/nodes/email/_service.py and
server/nodes/email/_himalaya.py). EmailService exposes HimalayaService
via its himalaya property (lazy import).
Request Flow (emailSend)
emailSend node
│
▼
EmailSendNode @Operation("send") (server/nodes/email/email_send/__init__.py)
│
▼
EmailService.send(params)
│
├── resolve_credentials(params)
│ email_providers.json preset > stored "email_*" API keys
│
▼
HimalayaService.send_email(creds, to, subject, body, ...)
│
├── _account_name(creds) = email prefix, sanitized
├── _generate_config() writes TOML to tempfile
├── compose MIME message (RFC 2822)
├── subprocess: himalaya -c <tmp> -a <acct> --output json message send
│ (stdin = MIME message)
├── parse JSON stdout, delete tempfile
▼
{"success": True, "result": {...}, "execution_time": 0.34}
Credentials Resolution Precedence
For every field in the returned credentials dict:
| Priority | Source | Example |
|---|---|---|
| 1 (highest) | Provider preset from email_providers.json |
providers.gmail.imap_host = "imap.gmail.com" |
| 2 (lowest) | Stored API key via AuthService.get_api_key() |
email_imap_host |
provider itself is the one field still read from node parameters, because it
is declared on all three Params models.
Credentials are not readable from node parameters, by design. There used to
be a documented third tier above these two (params["imap_host"],
params["password"], …). It never worked: the keys were not declared on any
Params model and all three set extra="ignore", so Pydantic stripped them
before model_dump() and every params.get(...) returned None. Declaring
them to revive the tier is not an option either — ToolNode.as_tool_schema
dumps Params.model_json_schema() wholesale with no field-exclusion hook, so a
declared password becomes an argument the LLM can pass to a callable tool.
The custom provider's preset is entirely blank, which is what makes the
stored keys reachable for a self-hosted server. Do not put default ports or
encryption values back into it: resolve_credentials falls through an or
chain, so a non-empty preset value shadows the stored key and makes it
unsettable. That is exactly what pinned self-hosted servers to 993/465/tls.
Defaults live in _himalaya._toml_port / _toml_encryption instead.
Key Files
Self-contained plugin folder (Wave 11.I) — everything email-specific lives under server/nodes/email/.
| File | Description |
|---|---|
server/nodes/email/__init__.py |
Self-registration: register_filter_builder("emailReceive", build_email_filter) + register_canary_trigger_type("emailReceive", "com.opencompany.email.message.received") + re-export of dispatch_email_received. |
server/nodes/email/_himalaya.py |
HimalayaService (ServiceSingleton). Subprocess-based invocation, temp TOML config generation, JSON output parsing. Singleton via get_himalaya_service(). |
server/nodes/email/_service.py |
EmailService (ServiceSingleton). Credential resolution, operation dispatch, polling helpers. Exposes HimalayaService via the himalaya property. Singleton via get_email_service(). |
server/nodes/email/email_send/__init__.py |
EmailSendNode(ActionNode) — dual-purpose send plugin (group = ("email", "tool"), usable_as_tool = True). |
server/nodes/email/email_read/__init__.py |
EmailReadNode(ActionNode) — dual-purpose read/search/manage plugin (7 operations). |
server/nodes/email/email_receive/__init__.py |
EmailReceiveNode(PollingTriggerNode) — polling trigger; baseline + diff loop with poll_ids / fetch_detail hooks. |
server/nodes/email/_filters.py |
build_filter (registered as build_email_filter) — server-side filter closure for the emailReceive trigger. |
server/nodes/email/_events.py |
email_message_received WorkflowEvent factory + dispatch_email_received (single dispatch.emit, CloudEvents type com.opencompany.email.message.received). |
server/nodes/email/email_{send,read,receive}/icon.svg + meta.json |
Per-plugin icon (served at /api/schemas/nodes/<type>/icon) + color metadata. |
server/config/email_providers.json |
Provider presets (IMAP/SMTP host/port/encryption per provider) + defaults + polling config. Cached on first load. |
server/constants.py |
EMAIL_TYPES, EMAIL_TOOL_TYPES, plus emailReceive in POLLING_TRIGGER_TYPES and WORKFLOW_TRIGGER_TYPES. |
client/src/components/credentials/panels/EmailPanel.tsx + panels/schemas/email.ts |
Email credentials panel (provider dropdown, email/password/display-name inputs, conditional custom IMAP/SMTP + encryption section). |
AI tool schema is derived automatically from each plugin's Params Pydantic model — there is no hand-written EmailSendSchema / EmailReadSchema. Dual-purpose dispatch goes through the generic plugin fast-path in server/services/handlers/tools.py (instance.execute_as_tool(...)), not a per-email _execute_email_tool branch.
EmailService API
File: server/nodes/email/_service.py
async resolve_credentials(params: Dict) -> Dict
Async. Builds the credentials dict consumed by HimalayaService (reads stored keys via AuthService.get_api_key). Required before every operation. Raises ValueError if the stored email address or password is missing.
Returned keys:
email(str, required) — account address, used as IMAP/SMTP loginpassword(str, required) — raw password or app passworddisplay_name(str) — from the storedemail_display_namekey; written to TOML only when non-emptyimap_host,imap_port,imap_encryption— resolved via precedence chainsmtp_host,smtp_port,smtp_encryption— resolved via precedence chain
Stored port values arrive as strings and go through _coerce_port, which
returns None rather than raising on garbage. Every port and encryption key
is always present in the returned dict, sometimes valued None — which is
why _generate_config must use _toml_port(creds.get("imap_port"), 993) and
not creds.get("imap_port", 993): the key is present, so dict.get's
default never fires and backend.port = None renders as unparseable TOML.
display_name resolves from the stored email_display_name key (written by the
credentials panel), not from node parameters.
send(params: Dict) -> Dict
Resolves credentials, calls HimalayaService.send_email(), and returns the result merged with {"from": creds["email"]}.
read(params: Dict) -> Dict
Operation dispatcher keyed by params["operation"] (default "list"):
| Operation | HimalayaService method | Required params |
|---|---|---|
list |
list_envelopes |
- |
search |
search_envelopes |
query |
read |
read_message |
message_id |
folders |
list_folders |
- |
move |
move_message |
message_id, target_folder |
delete |
delete_message |
message_id |
flag |
flag_message |
message_id |
Return shape: {"operation": ..., "folder": ..., ...data} where data is merged in if it's a dict, or wrapped as {"data": data} otherwise.
resolve_poll_params(params: Dict) -> Dict
Reads polling config from email_providers.json (polling.interval, polling.min_interval, polling.max_interval) and clamps the user-provided poll_interval into range. Returns {"interval", "folder", "mark_as_read"}.
Every field is coerced, not trusted: emailReceive overrides execute()
and passes raw parameters, so the Pydantic ge=30, le=3600 guard never runs on
that path and poll_interval can arrive as None or a string.
dict.get(key, default) only substitutes when the key is absent, so an
explicit None used to reach min() and raise TypeError. Mirrors
PollingTriggerNode._clamp_interval.
Note this clamp applies only to the interactive Run path. The deployment path
re-clamps through PollingTriggerNode._clamp_interval, so the JSON
polling.interval is not consulted there.
Polling helpers
poll_ids(creds, folder) -> Set[str]— Callslist_envelopeswithbaseline_page_size(from JSON config), extracts envelope IDs as strings for baseline/diff.fetch_detail(creds, msg_id, folder) -> Dict— Callsread_messageand merges{message_id, folder}into the result for downstream consumers.
HimalayaService API
File: server/nodes/email/_himalaya.py
CLI execution model
Every operation follows the same pattern inside execute():
ensure_binary()— locatehimalayaonPATHviashutil.which. Caches the path on the singleton. RaisesRuntimeErrorwith install instructions if missing. Despite the name it installs nothing; see "Installation Requirement" below._generate_config(account_name, credentials)— build TOML config as a single string (no template file; uses Python f-strings).tempfile.NamedTemporaryFile(suffix=".toml", delete=False)— write config to a temp file.asyncio.create_subprocess_exec(binary, -c <tmp>, -a <account>, --output json, <args>)- Pipe
stdin_dataif provided (used bysend_emailto deliver the MIME message). - Delegates to
services.events.cli.run_cli_command, which enforces thecli.timeout_secondsbudget fromemail_providers.jsonAND tree-kills the child on timeout.asyncio.wait_forcancels only the wrapper, so the old inline call left an orphaned process holding the temp config open -- and thefinallyunlink then raised PermissionError on Windows, masking the timeout. - Parse JSON stdout (
json.loads). Fall back to{"raw_output": stdout_str}on JSON decode error. finally: delete the temp config file withPath.unlink(missing_ok=True).
Error handling: non-zero exit raises RuntimeError(f"himalaya error: {stderr}"). The handler layer catches this and returns the standard error-shaped result dict.
High-level methods
| Method | Himalaya subcommand | Notes |
|---|---|---|
send_email(creds, to, subject, body, cc, bcc, body_type) |
message send (stdin) |
Composes MIME via email.mime stdlib. body_type="html" wraps in MIMEMultipart("alternative"); otherwise MIMEText. |
list_envelopes(creds, folder, page, page_size) |
envelope list -f <folder> --page N --page-size M |
|
search_envelopes(creds, query, folder) |
envelope list -f <folder> --query <q> |
|
read_message(creds, message_id, folder) |
message read <id> -f <folder> |
|
move_message(creds, message_id, target_folder, folder) |
message move <id> <target> -f <folder> |
|
delete_message(creds, message_id, folder) |
message delete <id> -f <folder> |
|
flag_message(creds, message_id, flag, action, folder) |
flag add/remove <id> --flag <name> -f <folder> |
action must be "add" or "remove". |
list_folders(creds) |
folder list |
Account naming
_account_name(credentials) derives a consistent TOML section name from the email address:
"[email protected]" -> "jane_doe_bot"
Dots and + are replaced with underscores so the name is valid as a TOML table key. The account name is arbitrary -- Himalaya just needs a consistent label between config and CLI invocation.
Node Catalog
emailSend — Dual-purpose (workflow node + AI tool)
Send email via SMTP. Group: ['email', 'tool']. Two outputs (main, tool).
Parameters:
| Name | Type | Required | Description |
|---|---|---|---|
provider |
options | yes | gmail, outlook, yahoo, icloud, protonmail, fastmail, custom |
to |
string | yes | Recipient(s), comma-separated |
subject |
string | yes | |
body |
string | no | Plain text or HTML (per body_type) |
cc |
string | no | |
bcc |
string | no | |
body_type |
options | no | text (default) or html |
AI tool schema: derived from EmailSendParams (same fields as the node, body_type default "text", cc/bcc optional). When the LLM invokes the tool, the generic plugin tool fast-path (instance.execute_as_tool in handlers/tools.py) runs EmailSendNode's send operation with the LLM args merged over the node params.
emailRead — Dual-purpose (workflow node + AI tool)
Read/search/manage emails via IMAP. Group: ['email', 'tool']. Two outputs (main, tool).
Parameters (all conditional via displayOptions.show on operation):
| Name | Shown when operation is | Description |
|---|---|---|
operation |
always | list / search / read / folders / move / delete / flag |
folder |
list, search, read, move, delete, flag | default INBOX |
query |
search | Himalaya search syntax (from:, subject:, etc.) |
message_id |
read, move, delete, flag | |
target_folder |
move | |
flag |
flag | Seen / Answered / Flagged / Draft / Deleted |
flag_action |
flag | add / remove |
page |
list | default 1 |
page_size |
list | default 20, max 500 |
AI tool schema: derived from EmailReadParams, exposing every operation and all operation-specific fields. The LLM picks an operation and fills the relevant subset.
emailReceive — Polling trigger
Group: ['email', 'trigger']. No inputs. Single output main with the new email data.
Parameters:
| Name | Default | Description |
|---|---|---|
provider |
gmail | Provider preset |
folder |
INBOX | Mailbox to monitor |
poll_interval |
60 | seconds (clamped 30..3600) |
filter_query |
"" |
Case-insensitive substring match over subject / from / to / body, applied in _filters.build_filter |
mark_as_read |
false | If true, adds Seen flag to new messages after fetch |
Baseline detection: On first execution the handler calls poll_ids(creds, folder) to capture the set of currently-existing envelope IDs. The poll loop then diffs against this baseline -- only newly appearing IDs trigger the workflow. This avoids firing on existing historical mail.
Standalone event dispatch: When a new message arrives during an interactive
Run, EmailReceiveNode.execute() (a plain override, not an @Operation):
- Fetches full detail via
fetch_detail - Optionally flags it as read
- Calls
dispatch_email_received(email_data)(in_events.py), which builds aWorkflowEvent(type="com.opencompany.email.message.received") and routes it throughdispatch.emit - Returns the first new email's result
Deployment Mode (Continuous Polling)
When a workflow containing emailReceive is deployed through workflow control,
its generation's WorkflowControlWorkflow owns continuous polling. It invokes
the activity generated by PollingTriggerNode.as_poll_activity(), maintains the
deduplication baseline in durable workflow state, and starts MachinaWorkflow
only for new messages. Pause gates new polls and graph starts; Resume continues
the same controller execution.
EmailReceiveNode(PollingTriggerNode) declares the polling hooks (poll_ids,
fetch_detail); the plugin system registers the corresponding Temporal poll
activity. The older deployment poll-coroutine registry and
PollingTriggerWorkflow remain compatibility paths for uncontrolled deployments
and replay of existing histories, not the authority for new controlled runs.
The poll loop:
- Resolves credentials once via
EmailService.resolve_credentials - Establishes the baseline via
svc.poll_ids(creds, folder) - On each iteration (clamped interval from
resolve_poll_params):- Polls for current IDs and computes
new_ids = current - seen - For each new ID, oldest UID first: fetches detail, optionally marks read, dispatches the event. The envelope returned to the canvas is the FIRST message; earlier revisions dispatched only one and dropped the rest.
- Handles
asyncio.CancelledErrorcleanly on teardown
- Polls for current IDs and computes
All credential resolution and IMAP access still delegate to EmailService —
nothing is duplicated in the controller. See
Temporal Execution Engine RFC for the
control and trigger lifecycle.
Credentials Storage
API key names
Stored via AuthService.store_api_key() / read via .get_api_key(). All keys live in the EncryptedAPIKey table (separate from OAuth tokens).
Required keys (any provider):
| Key | Stored by | Read by |
|---|---|---|
email_provider |
Credentials Modal | resolve_credentials (defaults to gmail) |
email_address |
Credentials Modal | resolve_credentials |
email_password |
Credentials Modal | resolve_credentials |
Optional keys (custom provider only):
| Key | Purpose |
|---|---|
email_imap_host |
Fallback IMAP hostname when preset is empty |
email_imap_port |
Fallback IMAP port (stored as string, coerced to int) |
email_imap_encryption |
tls / start-tls / none |
email_smtp_host |
Fallback SMTP hostname |
email_smtp_port |
Fallback SMTP port |
email_smtp_encryption |
tls / start-tls / none |
These custom keys are only used when the preset for the selected provider has empty host/port fields (i.e., provider == 'custom'). For named providers like Gmail, the preset always wins before the stored custom keys are consulted.
Credentials Modal UI
File: client/src/components/credentials/panels/EmailPanel.tsx
The Email category appears between Productivity and Android in the sidebar. The panel provides:
- Provider dropdown (7 options mirroring
email_providers.json) - Email address input
- Password input (secret, with "leave blank to keep existing" placeholder when already stored)
- Per-provider auth note (e.g., "Use an App Password from Google Account > Security > 2-Step Verification")
- Conditional custom IMAP/SMTP block shown only when
provider === 'custom':- IMAP host (text) + IMAP port (number, default 993)
- SMTP host (text) + SMTP port (number, default 465)
- Save button — writes
email_provider,email_address, and conditionallyemail_password(only if user typed a new one) + the four custom IMAP/SMTP keys when applicable. - Remove button — clears all ten
email_*keys. Switching away from thecustomprovider also clears the six server-specific keys, which otherwise stranded a stale host that then won theorchain.
Status is shown via getSpecialStatus(item) returning { connected: !!emailStored, label: emailAddress || 'Not configured' }. No WebSocket connection is needed because email credentials are pure API keys (no live session like Telegram or WhatsApp).
Configuration File
File: server/config/email_providers.json
{
"cli": {
"binary": "himalaya",
"homepage": "https://github.com/pimalaya/himalaya",
"install_url": "https://github.com/pimalaya/himalaya/releases",
"timeout_seconds": 60
},
"defaults": {
"provider": "gmail",
"folder": "INBOX",
"body_type": "text",
"page_size": 20,
"flag": "Seen",
"flag_action": "add"
},
"polling": {
"interval": 60,
"min_interval": 30,
"max_interval": 3600,
"baseline_page_size": 50
},
"providers": {
"gmail": {
"name": "Gmail",
"imap_host": "imap.gmail.com",
"imap_port": 993,
"imap_encryption": "tls",
"smtp_host": "smtp.gmail.com",
"smtp_port": 465,
"smtp_encryption": "tls",
"auth_note": "Use App Password from Google Account > Security > 2-Step Verification"
},
"outlook": { "...": "..." },
"yahoo": { "...": "..." },
"icloud": { "...": "..." },
"protonmail": {
"imap_host": "127.0.0.1", "imap_port": 1143, "imap_encryption": "none",
"smtp_host": "127.0.0.1", "smtp_port": 1025, "smtp_encryption": "none",
"auth_note": "Requires ProtonMail Bridge running locally"
},
"fastmail": { "...": "..." },
"custom": {
"imap_host": "", "imap_port": "", "imap_encryption": "",
"smtp_host": "", "smtp_port": "", "smtp_encryption": "",
"auth_note": "Enter your mail server details"
}
}
}
Loaded lazily by the module-level _load_config() in _service.py and cached in a module-level _CONFIG variable (surfaced through EmailService.config / .defaults / .polling properties). To add a new provider, edit only this JSON — no code changes required.
Config-driven, with named exceptions. resolve_poll_params reads the polling defaults, EmailService.read() reads defaults.page_size / defaults.flag, and HimalayaService.execute reads cli.timeout_seconds. Three values deliberately live in code rather than here: the port and encryption fallbacks in _himalaya._toml_port / _toml_encryption (they must NOT be in the custom preset, which would shadow the stored keys), and EmailReceiveNode.poll_interval_clamp, which the deployment path uses instead of the JSON clamp. The provider dropdown options and auth notes are duplicated in panels/schemas/email.ts with no test asserting they stay in sync with the providers keys.
Installation Requirement
The himalaya CLI must be installed and on PATH. Install via:
# macOS
brew install himalaya
# Linux / macOS with Rust toolchain
cargo install himalaya
# Pre-built binaries (Linux, macOS, Windows)
# https://github.com/pimalaya/himalaya/releases
HimalayaService.ensure_binary() caches the resolved path on the singleton after first detection. If missing, the handler returns:
{
"success": false,
"error": "himalaya CLI not found in PATH. Install via: cargo install himalaya, brew install himalaya, ..."
}
Provider-Specific Notes
| Provider | Password type | Notes |
|---|---|---|
| Gmail | App Password | Requires 2-Step Verification enabled first |
| Outlook / Office 365 | Account or App Password | STARTTLS on port 587 |
| Yahoo | App Password | Requires "Allow apps that use less secure sign-in" or App Password |
| iCloud | App-Specific Password | Generate from appleid.apple.com |
| ProtonMail | Bridge password | Requires running ProtonMail Bridge locally. IMAP localhost:1143, SMTP localhost:1025, encryption none (bridge handles TLS internally). |
| Fastmail | App Password | Generate from Settings > Privacy & Security |
| Custom | Whatever the server accepts | Must fill IMAP/SMTP host + port in the Credentials Modal's custom block. |
Related Docs
- Node Creation Guide — canonical plugin recipe (covers dual-purpose nodes;
emailSend/emailReadare live examples) - Event Waiter System — trigger registration for
emailReceive - New Service Integration — end-to-end integration pattern (use Google Workspace as a richer OAuth example)
- Credentials Encryption — how
email_*keys are encrypted on disk
Security notes
TOML generation escapes everything. _generate_config routes every
interpolated credential through _toml_str / _toml_port /
_toml_encryption. Do not add a raw f-string interpolation: an unescaped "
in a password breaks the config, and " plus a newline injects arbitrary
himalaya keys — including a backend.host pointing at someone else's server.
App passwords are user-chosen, so that path needs no prior compromise.
tests/nodes/test_email_service.py parses the generated config with tomllib
and asserts the key set, which is the actual regression guard.
CLI errors are scrubbed before they leave execute. himalaya echoes the
config path — and on a parse failure, config content — into stderr, and that
string becomes the RuntimeError message, the node error envelope, a persisted
node output, and a WebSocket broadcast. _scrub runs before both the log call
and the raise.
The password still touches disk. Each invocation writes a 0600 +
O_EXCL temp TOML and unlinks it in a finally (now actually effective on the
timeout path, since the child is killed first). Removing that entirely means
switching to himalaya's backend.auth.cmd:
backend.auth.type = "password"
backend.auth.cmd = "<command that prints the password>"
Not adopted yet: cmd delegates to pimalaya's process crate, which is sh -c
on POSIX and cmd /C on Windows, and getting a value to round-trip without
trailing-newline corruption on both shells needs verification against a real
binary. Recorded here so the next person does not re-derive it.