The fastest, cheapest, most efficient open-source agent harness. Run more than 500 agents on a $10 VPS.
desktop
Core-side logic that exists to serve the desktop client: the app-state
snapshot the shell polls, the notification center,
the model-health dashboard, the provider respond queue, and opt-in native
desktop control. A headless or embedded host has little use for any of it,
which is why these domains are grouped here and tagged DomainGroup::Desktop.
This is not the Tauri shell. The shell lives in crates/openhuman-app/ and
owns windows, the tray, and native OS integration. This folder is the
business logic those windows reach over JSON-RPC and Socket.IO. The browser
SPA uses the same RPC surface; the TUI (crates/openhuman-tui) uses little of
it.
How it works
The members are independent. They share a host-facing role, not a data model. Each one meets the frontend in one of two ways:
React shell / mascot window / browser SPA
| ^
JSON-RPC | | Socket.IO
(openhuman.*)| | (openhuman-rpc server/socketio.rs)
v |
+--------------------------+ +-------------------------------+
| controllers (core/all.rs)| | in-process broadcast channels |
| app_state_* | | notifications::subscribe_ |
| notification_* | | core_notifications |
| dashboard_* | | |
| provider_surfaces_* | | |
| desktop_* (internal) | +-------------------------------+
+--------------------------+ ^
| | publish
v |
peer domains (config, auth, DomainEvent bus -> notifications::bridge
local AI, service, keyring)
Request and response flows (the app-state snapshot, the notification list,
model health, the respond queue, desktop status) are ordinary controllers
registered under DomainGroup::Desktop in core/all.rs. Push flows use
process-global tokio::sync::broadcast channels that the Socket.IO server in
crates/openhuman-rpc subscribes to and forwards. Notifications get onto
their channel through NotificationBridgeSubscriber (notifications::bridge),
which translates selected DomainEvents; it is registered at startup from
core/runtime/subscribers.rs.
Desktop control
control/ is different from the rest: it lets the agent drive native desktop
apps through the tinycomputer module's accessibility layer, with Jev picking
actions. It is opt-in per computer. The enabled flag lives in
<workspace>/state/desktop-control.json and fails closed when the file is
missing or corrupt. Control is only supported on macOS (the TinyComputer engine has no
Windows or Linux desktop backend yet, #7164), and only when the core's HTTP listener is bound to loopback
(set_listener_is_loopback, set from core/runtime/builder.rs once the
address is known). The tools refuse to run on an unsupported platform, when the listener is not
loopback, or when control is disabled.
The agent tools are desktop_list_apps, desktop_list_windows,
desktop_launch, desktop_snapshot, desktop_find, desktop_goal, and
desktop_continue_goal (DesktopTool with a DesktopToolKind, registered in
tools/ops.rs). A desktop_goal call hands one bounded task to the module,
which runs and verifies several actions in that call. When the operator turns
desktop approvals on (config.desktop.approvals_enabled), the module can stop
on a pending action. confirmation.rs keeps those one-use decisions for ten
minutes and exposes them through desktop_pending and desktop_confirm. With
approvals off (the default), approvals_disabled_for lets these seven tools
skip only the approval park; permission caps, denies, OS grants, and the
action budget still apply. The agent-facing instructions are in
control/WORKFLOW.md.
Layout
| Path | What it does |
|---|---|
mod.rs |
Declares the members. Only control is feature-gated. |
app_state/ |
The snapshot the shell polls (app_state_snapshot): stored credential and user, local-AI and service status, onboarding tasks, keyring status, config-recovery notice. Owns state/app-state.json and update_local_state. |
control/ |
Opt-in native desktop control: enablement and status (ops.rs), pending-action decisions (confirmation.rs), the desktop_* agent tools (tools.rs), and internal-only controllers (schemas.rs). modules feature only. Agent guidance in WORKFLOW.md. |
dashboard/ |
Read-only per-model health table for Settings, built from model_registry and the dashboard.model_health thresholds. Also the dashboard_model_health agent tool. |
notifications/ |
Two pipelines: the event bridge that turns DomainEvents into CoreNotificationEvents, and the integration-notification store (SQLite) with background triage and per-provider settings. |
provider_surfaces/ |
An early scaffold: a normalized ProviderEvent model and an in-memory respond queue (soft cap 500) over embedded provider webviews. |
Key types and entry points
app_state::snapshot()(app_state/ops/snapshot.rs) assemblesAppStateSnapshotfrom peer domains. It never calls the backend; the user it reports is what the host passed in throughauth.set_credential.notifications::register_notification_bridge_subscriber(config)andnotifications::publish_core_notification/subscribe_core_notifications(notifications/bus.rs) are the push path for the notification center.control::status,control::set_enabled, andcontrol::probe(control/ops.rs) back the Connections page.DesktopToolandDesktopToolKind(control/tools.rs) are the desktop agent tools, re-exported throughtools/mod.rs.
RPC / CLI surface
Method names are openhuman.<namespace>_<function>.
| Namespace | Functions | Notes |
|---|---|---|
app_state |
snapshot, update_local_state |
Polled by the shell every few seconds. |
notification |
ingest, list, mark_read, dismiss, mark_acted, settings_get, settings_set, stats, core_list, core_mark_read |
Integration notifications plus persisted core notifications. |
dashboard |
model_health |
Errors when dashboard.model_health.enabled is false. |
provider_surfaces |
ingest_event, list_queue |
In-memory only. |
desktop |
status, set_enabled, probe, pending, confirm |
Internal-only (build_internal_only_controllers): callable by the shell, hidden from agent tool listings. modules feature only. |
Boundaries
- Windows, tray, mascot window, and OS integration belong to the Tauri shell
(
crates/openhuman-app/). The mascot window host iscrates/openhuman-app/src/mascot_native_window.rs. - Socket.IO forwarding belongs to
crates/openhuman-rpc/src/server/socketio.rs. - OS accessibility (AX and IOKit FFI, the Swift helper, focus, permissions,
the Globe key) executes inside the compiled TinyComputer module in
vendor/tinycomputer(upstreamtinyhumansai/tinycomputer). Desktop observation, interaction, and Jev-driven goals belong to thetinycomputermodule; this folder calls it throughtinycomputer-bus. - The live current user belongs to the host's session owner
(
openhuman_tinyhumans::session), not toapp_state.
Gotchas
- Only
controlis feature-gated (modules). Everything else compiles unconditionally. The runtime filter isDomainSet::desktop(core/runtime/builder.rs). There is nodesktopCargo feature yet; the grouping exists so one can drop the family as a unit later. (Themod.rsdoc points atdocs/specs/2026-08-02-core-kernel-domain-reorg.md, which is not in the repository.) controlchecks loopback on every tool call and every confirmation, not just at enable time. A core bound to a non-loopback address reportssupported: false.- The notification channel drops events when no one is subscribed. It is not a queue.
provider_surfaceskeeps its queue in process memory, so it is empty after a restart.
Tests
Each member keeps sibling *_tests.rs files.
cargo test -p openhuman desktop::
pnpm debug rust desktop::