Soul-driven AI agent with permission-hardened tools, token budgets, and multi-channel access. Runs 24/7 from CLI, Telegram or More.
Optimus
Mercury uses your computer like a person: it sees the screen, moves the mouse, types, and opens apps, on macOS, Windows and Linux. Use it by chat, or out loud with a talk key.
Setup
mercury plugins install optimus
Setup asks:
- Let Optimus control this computer? Nothing works until you say yes.
- How do you want to talk to Optimus? Chat, Listening, or Both (recommended).
- Key to start and stop talking (default
ctrl+alt+m: Control+Option+M on a Mac). - Who listens and speaks? OpenAI is used automatically when Mercury has an OpenAI key; otherwise pick Gemini, Grok, ElevenLabs, Azure, Qwen, or your own Mercury model with Deepgram/Cartesia/etc. Keys you added for the WhatsApp plugin work here too.
Change any of it later in /plugins optimus.
- macOS: Optimus builds
Optimus.appin its data folder and signs it with a certificate made on your Mac (no Apple account involved), then asks for three permissions in System Settings → Privacy & Security: Accessibility (mouse and keyboard), Screen Recording (seeing the screen) and, if you talk to it, the Microphone. The permissions survive updates. macOS asks you to confirm Screen Recording again about once a month. - Windows: Windows never asks per app. The microphone works when Settings → Privacy & security → Microphone has "Microphone access" and "Let desktop apps access your microphone" on (Windows 10: Settings → Privacy → Microphone, "Allow desktop apps…"). Windows running as administrator (Task Manager, installers) can't be controlled yet.
- Linux: works fully on X11. On Wayland, screenshots go through the desktop's screen-sharing prompt, and input and the talk key reach only X11 apps for now (use chat, or the
listentool). Linux has no microphone permission: Optimus reaches the microphone through PipeWire or PulseAudio in your desktop session.
Microphone & permissions in Mercury Desktop (Settings → AI → Voice) shows each of these for your OS with a green, amber or red dot, the exact steps to fix one that's off, an Allow button where the OS can prompt (macOS) and a button that opens the right settings page. Pick which microphone Optimus listens with, turn the microphone off for Optimus altogether, and press Test for a live level meter (grey when silent, green for speech, orange when loud). A blocked microphone is reported when a session starts, instead of Optimus hearing silence.
Talking to it
- Chat: ask Mercury in any chat (TUI, dashboard, WhatsApp, Telegram): "look at my screen", "open Safari and search for…".
- Listening: press the talk key (you hear a rising chime), talk, and Mercury answers out loud while it works on your screen, saying each step as it happens. A soft tick means it's still working.
- The talk key: while Optimus is working it's the stop button (everything halts, you keep talking); while Mercury is speaking it interrupts; when all is quiet it ends the session (falling chime).
- Mute: Control+Option+Shift+M (or say "mute") stops listening while Mercury keeps working and talking; nothing you say is sent. Press it again, or the talk key, to unmute. The status bar shows 🔇.
- Saying "stop" ("cancel", "ruko", "bas") halts every action and the task at once; "bye" does the same and ends the session. Silence for 60 seconds ends it too.
- Links and details go to your Mercury chat. Approvals are asked out loud.
- With speakers, the mic is muted while Mercury talks so it doesn't hear itself. With headphones, turn on "I use headphones" in
/plugins optimus→ 🎙 Voice to interrupt just by talking.
What it can do
| Tool | What it does |
|---|---|
look |
Screenshot of the display the mouse is on (or display N), with an optional zoom |
act |
Click, double-click, right-click, drag, type, key combos, scroll, wait; returns a fresh screenshot |
act_consequential |
The same, for steps that submit, send, pay, delete, install or change settings: you approve first |
open |
Open an app by name, a URL, or a file |
windows |
List open windows |
permissions |
Check (and request) the OS permissions |
listen |
Start a voice conversation from chat ("let's talk") |
Safety
- Off until you allow it in setup.
- Anything that commits something goes through Mercury's approval prompt.
- Move the mouse while Optimus is working and it stops, handing control back.
- It never types passwords, card numbers or 2FA codes: it asks you to.
- Screenshots go to the model for the current step only; they are not saved in chat history or logs.
Build
cd plugins/optimus/companion && cargo build --release
node plugins/optimus/package.mjs companion/target/release/mercury-optimus darwin-arm64 dist
Linux needs libxcb1-dev libxrandr-dev libdbus-1-dev libpipewire-0.3-dev libwayland-dev libegl-dev libgbm-dev libasound2-dev clang to build.