Andee lives on your phone or Mac and uses any app like you do—no integrations, no root, any model; memory and passwords never leave your device | Andee 住进你的手机或 Mac,像你一样操作任何 App——无需集成、无需 Root、支持任意模型,记忆和密码绝不离开设备。
fix(mac): type into apps that hide their focused field, without the shell
booboosui1866d2f30b23
diff --git a/CLAUDE.md b/CLAUDE.mdindex 4255904..039fdd0 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -661,6 +661,7 @@ meetings, gear, friends) and lists exactly the tools some provider handles. Whatlistens, double-tap stops and types, long-press opens the window; a sideways drag on the chat ball switchescharacter.- **Notifications come from `usernoted`'s SQLite store, and where it is depends on the macOS version** (`notifCandidates` in `AndeeMac.swift`): macOS 15 keeps it in `~/Library/Group Containers/group.com.apple.usernoted/db2`, earlier systems under `confstr(_CS_DARWIN_USER_DIR)` + `com.apple.notificationcenter/db2` (never derive that from `$TMPDIR` — the JVM replaces it). Only the first was tried, and a store that was not there was reported as missing Full Disk Access, so the user granted it and was told the same thing forever. `andee_notif_latest_id` now answers -2 for "no store anywhere" (`State.NOT_FOUND`, no permission button) and -1 for "exists but refuses us" (`NO_ACCESS`), and reads in process through `libsqlite3` (read-only, busy timeout, since `usernoted` is writing the WAL) instead of forking `/usr/bin/sqlite3`, which swallowed the reason.+- **`type_text` pastes even when the front app reports no focused element** (`MacScreen.typeBlind`). Mac WeChat answers `focused: false` to accessibility, and refusing there sent the model to the shell (`osascript` + ⌘V), which a notification's turn may not use — so a message Andee could read, it could not answer. The blind path sends ⌘V only (no ⌘A: with focus unknown, select-all may take the chat history), refuses when Andee's own window is active, and returns `unverified` with a note to check the screenshot and not to reach for the shell. Verified in WeChat's 文件传输助手 by reading the box back with ⌘A ⌘C. The pasteboard is still not restored afterwards — that was a deliberate choice (see `andee_set_pasteboard`).- **A prompt card appears beside the ball** (`PromptWindow`), which may be on a screen the user is not lookingat. A friend's request therefore plays a sound and waves the ball first — the first one timed out unseen.diff --git a/desktop/src/main/kotlin/net/kuafuai/andee/desktop/mac/MacScreen.kt b/desktop/src/main/kotlin/net/kuafuai/andee/desktop/mac/MacScreen.ktindex 5e14472..34acb33 100644--- a/desktop/src/main/kotlin/net/kuafuai/andee/desktop/mac/MacScreen.kt+++ b/desktop/src/main/kotlin/net/kuafuai/andee/desktop/mac/MacScreen.kt@@ -252,7 +252,7 @@ class MacScreen(*/private fun type(text: String): JSONObject {val before = Bridge.focusedValue()- if (!before.optBoolean("focused")) throw IllegalStateException("no field has focus — click into the field first")+ if (!before.optBoolean("focused")) return typeBlind(text)// Keys go to the focused app, not to a point on screen: fading our windows changes nothing for them.var via = "paste"if (Bridge.setPasteboard(text)) {@@ -278,6 +278,36 @@ class MacScreen(}}+ /**+ * The app in front reports no focused element to accessibility — Mac WeChat is the case that forced this.+ * Refusing here ("no field has focus") was a dead end the model could only leave through the shell+ * (`osascript` to set the clipboard and press ⌘V), and the shell is closed to a notification's turn, so+ * a message arrived that Andee could read and could not answer. The keystroke never needed the focused+ * element: ⌘V goes to whatever the front app has focused, exactly as a person's would.+ *+ * So: paste, and only paste. No ⌘A — with no idea where focus is, select-all may select the chat history+ * rather than the box. Unverified by construction, and the result says what the screenshot must show.+ */+ private fun typeBlind(text: String): JSONObject {+ // ⌘V lands in the frontmost app, and if that is Andee's own window it lands in Andee's own input.+ if (java.awt.Window.getWindows().any { it.isActive }) {+ throw IllegalStateException("Andee's own window is in front, so a paste would land in Andee. Click into the other app's input box first.")+ }+ val app = Bridge.frontmost().optString("name").ifEmpty { throw IllegalStateException("no app is in front — open the app and click into its input box first") }+ if (!Bridge.setPasteboard(text)) throw IllegalStateException("could not put the text on the pasteboard")+ Bridge.key(Bridge.Keys.V, Bridge.Keys.CMD)+ Thread.sleep(350)+ return JSONObject().put("performed", true).put("unverified", true).put("via", "paste_blind").put("len", text.length)+ .put("app", app)+ .put(+ "note",+ "$app does not tell the accessibility layer which box has focus, so I pasted into whatever it has focused " ++ "(⌘V, once) and cannot read it back. Look at a screenshot: the text must be in the input box before you " ++ "submit. If the box already had text, mine was added at the cursor, not in place of it. Do NOT type it " ++ "again and do NOT use the shell to paste — this is the way to type into $app.",+ )+ }+private fun submit(): JSONObject {Bridge.key(Bridge.Keys.RETURN)return JSONObject().put("performed", true).put("via", "return_key")diff --git a/desktop/src/main/kotlin/net/kuafuai/andee/desktop/shell/ShellPolicy.kt b/desktop/src/main/kotlin/net/kuafuai/andee/desktop/shell/ShellPolicy.ktindex 6327e66..f28f86e 100644--- a/desktop/src/main/kotlin/net/kuafuai/andee/desktop/shell/ShellPolicy.kt+++ b/desktop/src/main/kotlin/net/kuafuai/andee/desktop/shell/ShellPolicy.kt@@ -74,17 +74,22 @@ object ShellPolicy {fun decide(tier: Tier, turn: Turn, mode: Mode, alwaysAllowed: Boolean, criticalConfirm: Boolean): Decision {when (turn) {Turn.NOTIFY -> return Decision.Refuse(- "This turn started from a notification somebody else sent, so no command can run from it. " +- "Tell the user what you were about to run and let them ask for it themselves.",+ "This turn started from a notification somebody else sent, so no command can run from it — not in this turn " ++ "and not in a todo you set from it. To type into an app (a reply), use type_text, which needs no command. " ++ "Otherwise tell the user what you were about to run and let them ask for it themselves; do not promise a " ++ "later turn will run it.",)Turn.SILENT -> return Decision.Refuse("A silent review may only use the notebook; it must not run commands.")Turn.WAKE_OTHER -> if (tier != Tier.SAFE) return Decision.Refuse("This todo was not set by the user, and nobody is here to approve a command — only read-only " +- "commands may run now. Tell the user what you wanted to run when they are back.",+ "commands may run now. To type into an app, use type_text, which needs no command. Tell the user what " ++ "you wanted to run when they are back; do not promise another automatic turn will run it.",)Turn.WAKE_OWN -> if (mode != Mode.FULL && tier != Tier.SAFE) return Decision.Refuse("A todo is running and the user may not be at the Mac to approve anything, so only read-only " +- "commands may run now. Tell the user what you wanted to run when they are back.",+ "commands may run now (full access lifts this). To type into an app, use type_text, which needs no " ++ "command. Tell the user what you wanted to run when they are back; do not promise another automatic " ++ "turn will run it.",)Turn.USER -> Unit}