Skip to content

Self-improving AI Employees for everyone

fix(telegram): deliver the messages Telegram held during a restart

trohitg40c73dafb4a5
diff --git a/CLAUDE.md b/CLAUDE.md
index 93f7697f..7b5ac913 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -2814,7 +2814,7 @@ This function:
- **Typed CloudEvents factory pattern for new broadcasts** (b6aecd3): every new server→FE lifecycle broadcast follows the recipe locked by `broadcast_credential_event`, `broadcast_agent_progress`, and `broadcast_agent_capability`. (1) Add a typed factory — convention: `source = "opencompany://services/<area>"`, reverse-DNS `type = "com.opencompany.<area>.<event>"`, and `subject = <exact primary entity id>`. Put snake_case workflow/execution scope inside `data`; CloudEvents extension attribute names MUST be lowercase-alphanumeric, so the historical top-level `workflow_id`/`trigger_node_id`/`correlation_id` fields are compatibility debt and are forbidden on new contracts. (2) Add a broadcaster wrapper that emits `broadcast({type: "<wire_key>", data: event.model_dump(mode="json", exclude_none=True)})`. (3) FE validates `specversion`, ID/source/type, subject ownership, and event-specific payload invariants, deduplicates by `(source,id)`, then writes only the exact scoped entity. (4) Add contract tests in `test_events.py` and keep `tests/test_status_broadcasts.py`'s raw-callsite allowlist empty. Raw `node_status` remains allowed only as the established high-frequency latest-state/reconnect projection paired with a typed lifecycle occurrence; it is not the event of record.
- **Plugin-folder location for plugin-specific CloudEvents factories** (RFC §6.4): when a typed event is owned by one plugin (telegram, whatsapp, android, email, google, master_skill, webhook_trigger, agent), the factory + `broadcast_<event>` wrapper live in [`server/nodes/<plugin>/_events.py`](./server/nodes), NOT on `WorkflowEvent` / `StatusBroadcaster` directly. Telegram's [`_events.py`](./server/nodes/telegram/_events.py) is the canonical template — `broadcast_telegram_status` constructs the envelope via a plain factory function and calls `get_status_broadcaster().broadcast(...)` itself. Latest example: [`server/nodes/skill/master_skill/_events.py`](./server/nodes/skill/master_skill/_events.py) (`broadcast_skill_lifecycle("created" \| "updated" \| "deleted" \| "content_saved", name=..., **data_extra)`) — replaces four raw-dict broadcasts in `handle_create_user_skill` / `handle_update_user_skill` / `handle_delete_user_skill` / `handle_save_skill_content` (the last one was previously silent). FE routes wire key `skill_lifecycle` to invalidate `userSkills` + `folderSkills` queries and drop the `skillContent` cache on delete + content_saved so the Master Skill panel refreshes live across every connected client. Cross-cutting events (credential, workflow_lifecycle, agent_progress, agent_capability, node_parameters_updated) keep their `WorkflowEvent` classmethods + `StatusBroadcaster` wrappers — only plugin-scoped events move to the plugin folder. A core service's own event keeps its factory beside it, so core never imports a plugin `_events.py`: `services/workflow_ops.py` (`workflow_ops_apply`), `services/chat_thread.py` (`chat.updated`) and `services/chat/events.py` (the owner's chat message to the chatTrigger listeners, which moved there from the plugin, and `chat_run_event`).
- **Operator-metadata fields in credential panels** (f746ddf): when a credential panel needs a non-credential follow-up field (e.g. Telegram bot token + owner chat id), declare it as a second entry in the provider's `fields` array in `server/config/credential_providers.json`. The first field stays the validate/connect target; subsequent fields render below as plain text inputs via `SecondaryFieldRow` in `ApiKeyPanel.tsx` (shadcn `Input` + `Label` + `<ActionButton intent="save">`). Each field carries an optional `help` slot (`FieldDef.help` / `ServerFieldDef.help`) for always-visible explanatory text under the input. Save writes via `panel.actions.save(field.key, value)` → `auth_service.store_api_key` (which is permissive about provider string). The credential class must declare the new key in `extra_fields` (`Credential` subclass attribute) so the storage layer accepts it. Reference: `telegram_owner_chat_id` field.
-- **Telegram owner detection — three layers** (f746ddf): the bot owner is captured by THREE independent paths so the realistic setup flows all converge to a working state. (1) **Explicit field**: secondary `FieldDef` in the credentials modal lets the user paste their Telegram user ID directly (recommended; works without DM'ing the bot). (2) **Pre-poll peek** in `connect()` via `_capture_owner_from_pending_updates()` — calls `bot.get_updates(timeout=0)` BEFORE `start_polling(drop_pending_updates=True)` discards the queue, scans for any historical private DM, captures + persists atomically. (3) **Atomic write-through in `_on_message_received`** — persists FIRST, sets in-memory ONLY on success. Invariant: "in-memory has owner ⇒ DB has owner" so a process restart can re-capture cleanly. Failure logged at `ERROR` with `exc_info=True` (was `WARNING` previously, masking persist failures). The lazy fallback in `_send.py` `resolve_chat_id` (read DB → `service.set_owner`), which `telegram_send.py` calls, sits on top of all three.
+- **Telegram owner detection — three layers** (f746ddf): the bot owner is captured by THREE independent paths so the realistic setup flows all converge to a working state. (1) **Explicit field**: secondary `FieldDef` in the credentials modal lets the user paste their Telegram user ID directly (recommended; works without DM'ing the bot). (2) **Pre-poll peek** in `connect()` via `_capture_owner_from_pending_updates()` — calls `bot.get_updates(timeout=0)` before polling starts, scans the pending queue for a historical private DM, captures + persists atomically, so the owner is known before polling delivers that queue. Polling starts with `drop_pending_updates=False` (RFC-0005 Phase 0.5): the messages Telegram held while the backend was down (up to 24 h) are delivered, once each, instead of discarded. (3) **Atomic write-through in `_on_message_received`** — persists FIRST, sets in-memory ONLY on success. Invariant: "in-memory has owner ⇒ DB has owner" so a process restart can re-capture cleanly. Failure logged at `ERROR` with `exc_info=True` (was `WARNING` previously, masking persist failures). The lazy fallback in `_send.py` `resolve_chat_id` (read DB → `service.set_owner`), which `telegram_send.py` calls, sits on top of all three.
- **Supervisor Job Object loud-failure (cli/tree.py + cli/supervisor.py)** (8a64eb9): the Windows process-tree mechanism depends entirely on a Job Object with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` (no fallback — `start_new_session` is POSIX-only). `pywin32>=308` is a hard supervisor dependency on Windows; the declaration in the root `pyproject.toml` (where `opencompany-cli` lives — `cli/pyproject.toml` was removed as a stale duplicate) carries an inline comment so future maintainers don't drop it. `_JobObject.__init__` splits failures into explicit `ImportError` (with reinstall command) and `Exception` (with `repr`) branches — both write to `stderr` instead of swallowing. `_JobObject.add()` verifies enrollment via `win32job.IsProcessInJob(handle, self._handle)` after assignment; on mismatch it queries `IsProcessInJob(handle, None)` so the warning explicitly names the wrapper-job hypothesis (npm / bun / conhost wrappers occasionally place us in their own non-nesting Job). `supervisor._spawn_once` checks `add_to_job` return value and emits a yellow `WARN: pid=N not enrolled in Job Object` per child if False. Without these guards, a stale pywin32 install or a wrapping Job Object silently leaks orphan Python processes on every Ctrl-C and they accumulate to the point of holding SQLite locks that block subsequent backend startup.
- **Credentials envelope-shape invariant** (dc94cde): the `useCredentialPanel` query stores `{values: CredentialFormValues, hadStored: boolean}` (an envelope, NOT a flat dict). The earlier `writeValues` called `setQueryData<CredentialFormValues>` with a spread `(prev) => ({...prev, [key]: value})`, which at runtime merged the typed character at the envelope level next to `values` and `hadStored` instead of inside `.values` — the input selector `panel.values[field.key]` re-rendered with the original (server) value on every keystroke and the input felt frozen. Fix: `writeValues` now preserves the envelope shape and updates only the inner `values` dict; `hadStored` is preserved verbatim (it reflects real server state, not local edits). When introducing other queries shaped as `{data, meta}` envelopes, follow the same pattern — never spread `prev` as if it were the inner payload.
- **`core.paths` is the SSOT for on-disk locations**: only generic helpers (`data_path` / `packages_dir` / `package_dir(name)` / `daemons_dir` / `workspaces_dir` / `workspace_dir(workflow_id)` / `example_workflows_dir`). Plugin-specific subpaths are composed inline at the plugin's call site — adding a new package never requires touching `core.paths`. Reference subtree under `opencompany_root()` (= `~/.opencompany/` per default `DATA_DIR`): `workspaces/<slug>/` (per-workflow scratch), `daemons/` (supervised event-source daemon cwds — `stripe listen` etc.; shared root by default, no per-namespace subdir), `claude/` (Claude Code's `CLAUDE_CONFIG_DIR` state — composed as `data_path("claude")` in `nodes/agent/claude_code_agent/_oauth.py`), `packages/` (OpenCompany-managed install root). `packages/` holds a **single shared, bun-managed packages tree** — one `package.json` + `bun.lock` + `node_modules/` covering every OpenCompany-managed npm-registry package (`@anthropic-ai/claude-code`, `edgymeow`, `cf`, `vercel`). Each plugin's `_install.py` calls `core.js_runtime.add_package(spec, trust=...)`, which runs `bun add --cwd <packages_dir> --no-progress [--trust] <spec>` and extends the shared tree idempotently (`ensure_shared_tree()` writes the minimal private manifest first); `bun_binary()` resolves bun from `OPENCOMPANY_BUN_BIN` or PATH, and `shared_tree_bin(name)` returns the shim at `node_modules/.bin/<name>` on POSIX or `<name>.exe` (plus a `<name>.bunx` file) on Windows. `installed_version(name)` reads `node_modules/<name>/package.json`: an existing shim says nothing about its version and `company clean` keeps `packages/`, so a pinned plugin compares against it and re-runs `add_package` on a mismatch (Cloudflare does; the other installers check only that the shim exists, so their pin bumps reach fresh installs only). The shims run on the bun runtime — no Node or npm anywhere on this path (bun ignores a package's `engines.node`, and shims `node` for the lifecycle scripts it is allowed to run). bun blocks dependency lifecycle scripts unless trusted, so `trust=True` is passed only where a postinstall materialises something the plugin needs: claude-code (stages its native launcher), edgymeow (downloads the Go binary). The Browser runtime does not use this npm tree: its CLI installer uses `packages/browser-use/`; only explicit testing mode installs Chrome under `packages/chrome-for-testing/` (see [Browser](./docs-internal/browser.md)). Binaries that are not npm-registry packages (Stripe CLI, Temporal CLI) sit under sibling subdirs `packages/stripe/`, `packages/temporal/` via `package_dir(name)`. Also under `<DATA_DIR>/`: `whatsapp/` (WhatsApp session DB), `credentials.db` / `workflow.db` / `temporal.db`, and the model-registry caches `local_models.json` (models registered from the user's own LLM servers) / `litellm_models.json` (LiteLLM's model table, RFC-0003 D16). Pre-fix `packages_dir()` routed through `platformdirs.user_cache_path("OpenCompany")` (`~/.cache/OpenCompany/` etc.) and the Temporal CLI sat in its own `pooch.os_cache("opencompany-temporal")` namespace — operators reported both as "not local". The WhatsApp Go bridge (`edgymeow`) used to be a top-level pnpm dep at `<repo>/node_modules/edgymeow/`; now OpenCompany-managed via `nodes/whatsapp/_install.py` like the other CLIs. Daemon cwds used to live under `workspaces/_<namespace>/` and polluted per-workflow scratch with framework state. Out of scope: globally-installed binaries (Himalaya — system package manager). Shipped seed workflows are the lone exception — `example_workflows_dir()` is hardcoded to `<repo>/.opencompany/workflows/` (git-tracked seeds, NOT under `DATA_DIR`) so they survive `company clean` and stay at the same path across `DATA_DIR=~/.opencompany` vs `DATA_DIR=.opencompany` configs.
diff --git a/server/nodes/telegram/_service.py b/server/nodes/telegram/_service.py
index 2a94962d..1ce9e5f9 100644
--- a/server/nodes/telegram/_service.py
+++ b/server/nodes/telegram/_service.py
@@ -530,20 +530,13 @@ class TelegramService(ServiceSingleton):
except (TypeError, ValueError):
pass
- # Pre-poll peek for historical owner capture.
- # Polling starts with drop_pending_updates=True
- # (correct hygiene for operational events -- we do
- # not want stale group spam replaying on every
- # restart) but that ALSO drops any DM the user sent
- # before the bot connected. Without this peek the
- # only path to owner capture is "DM the bot AFTER
- # polling has started", which is the surprising UX
- # that produces "Bot owner not detected" on every
- # fresh setup. get_updates() with no offset returns
- # the same pending queue without advancing it, so
- # the subsequent start_polling drop-pass is a no-op
- # for these messages -- they were going to be
- # dropped anyway, we just inspect them first.
+ # Pre-poll peek for historical owner capture: find the
+ # owner in a DM the user sent before the bot connected,
+ # before polling delivers the pending queue, so a
+ # workflow fired by an earlier message in that queue can
+ # already send to the owner. get_updates() with no offset
+ # returns the queue without advancing it, so polling
+ # still delivers every one of these messages.
if self._owner_chat_id is None:
await self._capture_owner_from_pending_updates()
@@ -659,8 +652,12 @@ class TelegramService(ServiceSingleton):
try:
logger.info("[Telegram] Starting polling loop...")
await self._application.start()
+ # Keep the updates Telegram held while the backend was down (up
+ # to 24 hours): dropping them lost every message sent during a
+ # restart. Each update is delivered once; the offset advances
+ # as polling confirms it.
await self._application.updater.start_polling(
- drop_pending_updates=True,
+ drop_pending_updates=False,
allowed_updates=Update.ALL_TYPES,
error_callback=self._on_polling_error,
)
@@ -680,8 +677,8 @@ class TelegramService(ServiceSingleton):
async def _capture_owner_from_pending_updates(self) -> None:
"""Inspect the bot's pending update queue (24h Telegram retention)
for any private message and capture the owner if found. Called
- from connect() before polling starts -- after polling begins the
- ``drop_pending_updates=True`` flag throws these away unread.
+ from connect() before polling starts, so the owner is known before
+ polling delivers the queued messages.
Uses the same atomic-write-through invariant as
``_on_message_received``: persist FIRST, set in-memory only on
diff --git a/server/tests/nodes/test_telegram_service.py b/server/tests/nodes/test_telegram_service.py
index 50d0cc4d..ff5c3946 100644
--- a/server/tests/nodes/test_telegram_service.py
+++ b/server/tests/nodes/test_telegram_service.py
@@ -501,6 +501,27 @@ def test_transient_and_fatal_polling_messages_are_distinguishable():
assert "bot disconnected" not in callback_src
+async def test_polling_keeps_the_messages_telegram_held_during_a_restart():
+ """Telegram holds updates for 24 hours. Polling used to start with
+ ``drop_pending_updates=True``, which discarded every message sent while
+ the backend was down (RFC-0005 Phase 0.5)."""
+ import asyncio
+
+ service = TelegramService()
+ started = asyncio.Event()
+ updater = SimpleNamespace(start_polling=AsyncMock(side_effect=lambda **_: started.set()))
+ service._application = SimpleNamespace(start=AsyncMock(), updater=updater)
+
+ task = asyncio.create_task(service._run_polling())
+ await asyncio.wait_for(started.wait(), timeout=5)
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+
+ assert updater.start_polling.await_args.kwargs["drop_pending_updates"] is False
+
+
# ---------------------------------------------------------------------------
# shutdown: release the getUpdates slot
# ---------------------------------------------------------------------------