The fastest, cheapest, most efficient open-source agent harness. Run more than 500 agents on a $10 VPS.
Rust agent-orchestrationai-agentsai-assistantdesktopllmlocal-firstmcppersonal-aiprivacyrustsecond-braintauri
//! Centralised error reporting for the core, plus a Sentry
//! `before_send` filters that drop deterministic provider noise:
//! per-attempt transient-upstream failures, budget-exhausted user-state,
//! and transient updater failures.
//!
//! Wraps `tracing::error!` (which the global subscriber forwards to Sentry via
//! `sentry-tracing`) inside a `sentry::with_scope` so each captured event
//! carries consistent tags identifying the failing domain/operation plus any
//! callsite-specific context (session id, request id, tool name, …).
//!
//! Why this helper exists: errors that bubble up as `Result::Err` without ever
//! being logged at error level never reach Sentry. The agent-turn path is the
//! canonical example — `run_single` used to publish a `DomainEvent::AgentError`
//! and return `Err(_)`, but Sentry never saw it. Funnel error sites through
//! `report_error` so they show up tagged and grep-friendly in Sentry.
pub(crate) mod progress_tracing;
pub(crate) mod telemetry;
#[path = "observability_channel_message.rs"]
mod channel_message;
#[cfg(feature = "crash-reporting")]
pub use channel_message::is_channel_message_not_found_event;
use std::fmt::Display;
#[cfg(feature = "crash-reporting")]
#[path = "observability_release_tag.rs"]
mod release_tag;
#[cfg(feature = "crash-reporting")]
pub use release_tag::is_stale_release_event;
#[path = "observability_availability.rs"]
mod availability;
use availability::log_module_unavailable;
pub use availability::{
is_api_key_rejected_message, is_backend_unavailable_message, is_module_unavailable_message,
API_KEY_REJECTED_PREFIX, BACKEND_UNAVAILABLE_PREFIX,
};
/// A `(key, value)` pair attached as a Sentry tag. Tags are short, indexed,
/// and filterable in the Sentry UI — prefer them over free-form fields for
/// anything you'd want to facet on (`error_kind`, `tool_name`, `method`).
pub type Tag<'a> = (&'a str, &'a str);
/// HTTP status codes that represent transient backend / integration transport
/// failures rather than application bugs. Keep this as strings because Sentry
/// tags are strings, and the before_send classifiers match tag values exactly.
pub const TRANSIENT_HTTP_STATUSES: &[&str] = &["408", "429", "502", "503", "504", "520"];
/// Transport-layer phrases observed from reqwest / hyper for temporary
/// upstream interruptions. Keep these specific so rare configuration failures
/// still reach Sentry.
pub const TRANSIENT_TRANSPORT_PHRASES: &[&str] = &[
"timeout",
"operation timed out",
"connection forcibly closed",
"connection reset",
"tls handshake eof",
"error sending request",
];
/// HTTP statuses from updater probes that are expected GitHub/network noise:
/// unauthenticated GitHub API rate-limit / policy 403s plus gateway/server
/// hiccups. Scoped to updater domains/messages by [`is_updater_transient_event`].
const UPDATER_TRANSIENT_HTTP_STATUSES: &[u16] = &[403, 500, 502, 503, 504];
/// Message fragments observed from Tauri/core updater transient failures.
/// Keep these updater-specific so unrelated GitHub or generic transport
/// failures still reach Sentry.
///
const UPDATER_TRANSIENT_MESSAGE_PHRASES: &[&str] = &[
"failed to check for updates: error sending request",
"github api error: 403",
"github api error: 5",
"error sending request for url (https://github.com/tinyhumansai/openhuman/releases/",
"update endpoint did not respond with a successful status code",
];
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ExpectedErrorKind {
LocalAiDisabled,
ApiKeyMissing,
NetworkUnreachable,
TransientUpstreamHttp,
LocalAiBinaryMissing,
BackendUserError,
/// Third-party provider (composio, gmail OAuth, …) surfaced a user-state
/// validation failure: a trigger registry mismatch, a toolkit that was
/// never enabled, an OAuth scope that the user did not grant, or a
/// required field that was left blank. The UI already shows an
/// actionable error and Sentry has no remediation path — see
/// [`is_provider_user_state_message`] for the exact body shapes.
///
/// Drops OPENHUMAN-TAURI-3R / -3S / -33 / -34 / -97 (~54 events): the
/// composio backend wraps several of these as HTTP 500 with the real
/// 4xx body embedded, which would otherwise escape the
/// [`is_backend_user_error_message`] 4xx-only matcher.
ProviderUserState,
/// A user-configured custom cloud provider (`custom_openai` → DeepSeek
/// / OpenRouter / Moonshot / …) rejected the request because of the
/// user's **model / parameter configuration**: an OpenHuman abstract
/// tier alias leaked to a provider that only speaks its native ids
/// (#2079), an unknown / stale model pin (#2202), or a model-specific
/// temperature constraint (#2076 — Moonshot Kimi K2). The provider
/// HTTP layer (`provider::http_error::api_error`) already demotes its own
/// per-attempt event; this catches the *re-report* when the same
/// error is raised again by `agent.run_single` /
/// `web_channel.run_chat_task` under `domain=agent` / `web_channel`.
/// Deterministic user-config state surfaced in the UI — Sentry has no
/// remediation path (OPENHUMAN-TAURI-WJ / -QW / -HB / -NH, ~273
/// events). See
/// [`crate::inference::provider::is_provider_config_rejection_message`]
/// for the polarity contract and exact body shapes.
ProviderConfigRejection,
LocalAiCapabilityUnavailable,
BudgetExhausted,
SessionExpired,
/// Boot-window failure where the in-process core HTTP listener
/// (`127.0.0.1:<port>`) is not yet accepting connections, so a sibling
/// component (frontend RPC relay, integrations client) sees a TCP
/// connect refused. The condition self-resolves once the core finishes
/// binding — typically within a few seconds of app launch — and no retry
/// on the calling side can do better than waiting it out.
///
/// Distinct from [`ExpectedErrorKind::NetworkUnreachable`] (which covers
/// real user-environment network problems — VPN drop, captive portal,
/// ISP block) because:
///
/// - The remediation is internal lifecycle (the core's own startup), not
/// user action. Sentry has nothing to act on either way, but conflating
/// the two buckets makes "which class of transport failure is
/// spiking?" un-answerable.
/// - Loopback URLs (`127.0.0.1:` / `localhost:`) carry no PII, so the
/// demoted breadcrumb can stay sparse (debug level, metadata-only
/// fields) instead of warn-level with the full body included.
///
LoopbackUnavailable,
PromptInjectionBlocked,
ContextWindowExceeded,
// (WhatsApp structured-ingest SQLite busy/corrupt classifiers were removed
// when that store moved to the Tauri shell; the store itself is gone now —
// its only writer was the CDP scanner deleted in #5478 — so no build
// produces the envelope these matched.)
/// Host disk is full — the filesystem returned `ENOSPC` to a write,
/// `mkdir`, or `open` syscall. The user cannot recover from this without
/// freeing space on their machine, and Sentry has no remediation path
/// because the failing path is bound to the user's local FS. Surfaces
/// from many call sites once the disk fills up (auth profile lock
/// creation, SQLite WAL grows, log rotation, `tokio::fs::write` for
/// state snapshots) — every one of them emits the same canonical errno
/// rendering.
DiskFull,
/// A user-supplied filesystem path failed an RPC-level validation
/// check — e.g. `openhuman.vault_create` was called with a
/// `root_path` that doesn't exist or points at a file rather than a
/// directory. The UI already shows the typed error to the user, and
/// Sentry has no remediation path (we can't `mkdir -p` a folder the
/// user hasn't actually picked yet). User-supplied paths can also
/// embed PII fragments (the home-directory segment leaks the OS
/// username), so demoting these out of the Sentry event stream is a
/// small privacy win on top of the noise reduction.
///
/// Drops Sentry TAURI-RUST-4QH (`root_path is not a directory:
/// /Users/<user>/Documents/<vault>`, observed on
/// `[email protected]`) and preempts the symmetric
/// `hosted path is not a directory:` shape from
/// `openhuman_rpc::http_host::path_utils` once it starts surfacing.
/// See [`is_filesystem_user_path_invalid_message`] for the polarity
/// contract — the safety-guard variant in `skills::ops_install`
/// (`{path} is not a directory — refusing to remove`) is
/// deliberately not matched because that's an `rm -rf` invariant
/// violation, not user input.
FilesystemUserPathInvalid,
/// The provider/model completed a turn with a completely empty body
/// (`text_chars=0 thinking_chars=0 tool_calls=0`), so the agent harness
/// bailed with the user-facing `"The model returned an empty response.
/// Please try again."` string
/// (`agent::session_host::turn`). This is a model/user-config
/// condition — a quirky or broken local fine-tune that returns nothing,
/// a provider that dropped the stream — not a code bug. The UI already
/// surfaces the typed error and the user can retry; Sentry has no
/// remediation path.
///
/// `agent::run_single` already suppresses the **agent-layer** Sentry
/// event for this condition via the typed
/// `AgentError::EmptyProviderResponse` + `AgentError::skips_sentry()`
/// (PR #2790, TAURI-RUST-4JX). But `web_chat::
/// run_chat_task` **re-reports** the same failure under
/// `domain=web_channel operation=run_chat_task` after the typed error
/// has been flattened to a `String` at the native-bus boundary — so the
/// typed suppression can't reach it and it escapes as a fresh Sentry
/// event (TAURI-RUST-4Z1). This string classifier closes that second
/// emit site, mirroring how `MaxIterationsExceeded` is handled at both
/// layers. See [`is_empty_provider_response_message`].
///
/// Although the immediate trigger is the `web_channel.run_chat_task`
/// re-report, this classifier runs in the central `expected_error_kind`
/// dispatcher, so any caller of `report_error_or_expected`
/// (`channels/runtime/dispatch.rs`, `channels/runtime/supervision.rs`,
/// any future channel provider) whose error chain contains `"model
/// returned an empty response"` is also demoted — no per-channel typed
/// suppression needed.
EmptyProviderResponse,
/// The core has no hosted backend registered (built and run without
/// `openhuman-tinyhumans`), so a hosted-backend call could not be sent
/// at all. Expected build state, not a defect: the core runs agents,
/// memory and tools without any TinyHumans connection, and every
/// backend-touching surface degrades to this typed error. Messages carry
/// the [`BACKEND_UNAVAILABLE_PREFIX`] sentinel.
BackendUnavailable,
/// A cached native-module load failure handed back to another caller;
/// reported once at resolution. See [`is_module_unavailable_message`].
ModuleUnavailable,
/// Channel supervisor (`channels::runtime::supervision::spawn_supervised_listener`)
/// caught a transient error from a channel listener and restarted it. The
/// wrapper shape `"Channel <name> error: <inner>; restarting"` is the
/// signature; the underlying inner error can be anything — reqwest transport
/// errors, OS-localized WSAETIMEDOUT messages, TLS handshake failures, gateway
/// disconnect strings — all of which are self-resolving via the supervisor's
/// own backoff/retry loop. Sustained outages still surface via
/// `health.bus` / `FAIL_ESCALATE_THRESHOLD` (separate path, not affected by
/// this kind).
///
/// Drops Sentry TAURI-RUST-15 (~11.4 k events Discord gateway) and -BB
/// (~815 events Chinese-Windows variant) where the English-only
/// `is_network_unreachable_message` anchors miss the inner OS message.
ChannelSupervisorRestart,
ConfigLoadTimedOut,
/// A config-file READ failed because the OS refused access to a file that
/// exists (ACL-denied, held open by another process, OneDrive placeholder).
/// Unpreventable user-environment state — zero local lever to make the file
/// readable. Demoted only for the access-denied / locked io kinds; a
/// `NotFound` after the `exists()` check stays a paging defect. See
/// [`is_config_read_io_failure_message`]. Drops TAURI-RUST-DME
/// (`inference_downloads_progress` re-reads config every poll → 36k events /
/// 1 Windows user).
ConfigReadIoFailure,
/// Windows `ERROR_FILE_SYSTEM_LIMITATION` (os error 665) — the NTFS file
/// system cannot complete the operation because the filesystem is too
/// fragmented, the USN journal has overflowed, or a filesystem filter
/// driver has hit its resource cap. This is a persistent host-filesystem
/// condition that the user must resolve by restarting, running a defrag,
/// or freeing space — Sentry has no remediation path.
///
/// Also covers `ERROR_DISK_FULL` (112) / `ERROR_HANDLE_DISK_FULL` (39),
/// although those are more reliably caught by the `DiskFull` arm above.
///
/// Matched on the locale-stable `(os error 665)` suffix (the Italian
/// locale rendering observed in Sentry TAURI-RUST-QT0 — 6,050 events from
/// 1 Windows user — is `".. (os error 665)"`, matching any locale).
WindowsFileSystemLimitation,
/// The user invoked "Import Codex CLI login" (Connections → API keys → LLM → Codex auth)
/// but the Codex CLI auth at `~/.codex/auth.json` is absent or unusable:
/// the file doesn't exist (the user never ran `codex login`), can't be
/// parsed, or carries no tokens / no access token. The import RPC already
/// returns an actionable error string ("Run `codex login` first, then try
/// Codex auth again.") that the frontend surfaces inline
/// (`AIPanel.tsx` → `setCodexAuthError`), and Sentry has no remediation
/// path — we can't run `codex login` for the user. The error strings also
/// embed the absolute `~/.codex/auth.json` path (home dir / username), so
/// demoting these out of the event stream is a privacy win on top of the
/// noise reduction (mirror `FilesystemUserPathInvalid` / `DiskFull`).
///
/// Drops Sentry TAURI-RUST-83A (~430 events / 40 users on
/// `[email protected]`). Anchored to the `codex cli auth` /
/// `.codex/auth.json` envelope produced by
/// [`crate::security::credentials::openai_oauth::store::import_codex_cli_auth_from_path`]
/// — a genuine keyring/persist failure in `upsert_profile` carries neither
/// anchor, so a real defect in the import code still reaches Sentry.
CodexCliAuthUnavailable,
/// The managed backend (#870) stamped a stable `errorCode` on this
/// inference error response — so the backend **owns** it (it already paged
/// its own 5xx, or the code is expected user-state: rate limit, out of
/// credits, upstream unavailable, model/routing misconfig, payload too
/// large, context overflow, user-param rejection). The provider HTTP layer
/// (`api_error`) already demotes its own per-attempt event; this catches
/// the **re-report** when the same flattened error is raised again under
/// `domain=web_channel` / `agent` (the path
/// `web_chat::run_chat_task` →
/// `report_error_or_expected`). The FE surfaces actionable copy via
/// `classify_inference_error`; Sentry must not double-report (F2/F4).
///
/// The single exception — a backend-flagged **malformed** `BAD_REQUEST` —
/// is NOT classified here (it is a client-built payload the backend
/// couldn't parse, and the FE *does* page for it, F8). See
/// [`crate::inference::provider::backend_error_code_skips_sentry`].
BackendErrorCodeOwned,
/// A provider embedding call (Cohere `/v2/embed`, OpenAI/Voyage embed,
/// custom OpenAI-compatible embed) returned a **403/Forbidden gateway
/// HTML page** instead of the provider's JSON error envelope — the
/// signature of an edge/CDN/WAF or regional block sitting *in front of*
/// the provider API (the request never reached the provider app). The
/// endpoint is correct and a key was sent (the empty-key fast-fail guard
/// already passed), so there is no local lever: retry won't clear an edge
/// policy keyed on the user's network reputation / geo / IP. The
/// embedding caller already degrades gracefully and the UI surfaces the
/// failure; Sentry has no remediation path.
///
/// Anchored on HTML gateway markers (`<!doctype html`, `<title>403`,
/// `403 forbidden`) **and the absence of a JSON envelope** so an
/// actionable JSON 4xx (Cohere's `{"message": …}`, a real request-shape
/// bug in our client) still classifies `None` and reaches Sentry. See
/// [`is_upstream_edge_block_message`].
///
/// Drops Sentry TAURI-RUST-8S3 (~1.7 k events / 3 users on
/// [email protected], `Cohere embed API error (403 Forbidden):
/// <!doctype html>…<title>403</title>…`).
UpstreamEdgeBlock,
/// `approval_decide` (`crates/openhuman-core/src/security/approval/rpc.rs`) resolved a request_id
/// whose pending row was **already decided, lazily expired, or superseded**
/// — `store::decide` updated 0 rows because `decided_at` was already set,
/// and `store::get_decision` confirms a persisted decision exists. The
/// inline-approvals design spec
/// classifies "no pending approval found" as a **benign** outcome: the
/// frontend `ApprovalRequestCard.decide` and the Telegram callback both call
/// `approval_decide` without server-confirmed dedupe, so double-taps, two
/// operators racing, and expiry-while-live all land here harmlessly.
///
/// Drops the benign half of Sentry TAURI-RUST-5EH (~1,995 events / 8 users
/// on `[email protected]`). Anchored to the benign `"no pending approval
/// found"` wording emitted ONLY when `get_decision` confirms the row was
/// resolved; a genuine **never-registered** id raises the distinct
/// `"no pending approval ever registered"` string (no anchor) so a real
/// lost-registration defect still reaches Sentry.
ApprovalNoPendingRace,
/// A remote MCP server answered the connect handshake with HTTP 401 — it
/// needs OAuth sign-in, not a code fix. TinyMCP returns the typed
/// unauthorized outcome over `tinymcp-bus`, and
/// `mcp::registry::connections::connect` already classifies it and stores a
/// `needs_auth` flag so the UI prompts the user to authenticate (the
/// `needs_auth` UX shipped in #3733 / #3719). But `mcp_clients_connect`
/// still returns `Err(e.to_string())`, which propagates to the RPC
/// dispatcher (`jsonrpc` → `report_error_or_expected`) where no arm matched
/// it — so the same user-state condition the UI already handles was being
/// captured as a full Sentry error (TAURI-RUST-CGP: ~1.2k events / 79 users
/// on `[email protected]`). This arm closes that ship gap: the connect-time
/// 401 is preventable user-state with no Sentry-actionable signal, so demote
/// it to info. Anchored on the canonical `McpUnauthorizedError` Display body
/// (`"MCP unauthorized for "` + `"(HTTP 401"`) so an unrelated MCP transport
/// failure still reaches Sentry.
McpServerNeedsAuth,
/// The wallet has not been set up yet, and something that needs a
/// wallet-derived key said so.
///
/// A wallet is an **optional** feature. Not having one is the default
/// state for every user who has not opted in, the UI already renders a
/// "set up wallet" prompt, and there is no local lever that makes the call
/// succeed until the user creates one — so this is user-state, not a
/// defect.
///
/// `openhuman-rpc/src/server/http/rpc_handler.rs` already demoted the *bare* message via
/// `is_wallet_not_configured_error`, but that predicate is exact equality,
/// so it stops matching the moment any caller adds context — and callers
/// do: `format!("{context}: {e}")` appears ~800 times in `src/`. One such
/// wrap (`self_identity key_status: {e}`) was enough to route an
/// expected state to Sentry 55 times in 72 minutes on an ordinary local
/// session, while a genuine turn-killing failure in the same session
/// emitted nothing (#5805 / #5804).
///
/// Classifying it **here** rather than at the wrap site is deliberate:
/// this classifier is substring-based, so it holds for any wrapper, at any
/// nesting depth, on any reporting path that goes through
/// [`report_error_or_expected`] — not just the RPC boundary, and not just
/// the one method that happened to be observed. Matched against the shared
/// [`crate::web3::wallet::WALLET_NOT_CONFIGURED_MESSAGE`]
/// constant so producer and classifier cannot drift.
WalletNotConfigured,
}
include!("observability_expected_error_classification.rs");
include!("observability_plan_and_session_errors.rs");
include!("observability_provider_local_errors.rs");
include!("observability_network_provider_errors.rs");
include!("observability_reporting.rs");
include!("observability_event_classification.rs");
include!("observability_backend_event_classification.rs");
#[cfg(test)]
#[path = "observability_tests.rs"]
mod tests;