Andee lives on your phone or Mac and uses any app like you do—no integrations, no root, any model; memory and passwords never leave your device | Andee 住进你的手机或 Mac,像你一样操作任何 App——无需集成、无需 Root、支持任意模型,记忆和密码绝不离开设备。
fix(screen): wait for the system to notice the fold before reading the tree
booboosuid3b797951455
diff --git a/CLAUDE.md b/CLAUDE.mdindex e8d6f3b..73a6075 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -85,7 +85,7 @@ The mirror-image problem is **accessibility** visibility, and it is not solved bWhat makes `hideFromAccessibility()` actually work is **`accessibilityDataSensitive`, not `importantForAccessibility`** — and the difference is invisible until you look. `accessibility_service_config.xml` asks for `flagIncludeNotImportantViews`, which makes `View.includeForAccessibility()` return true no matter what `importantForAccessibility` says. So `NO_HIDE_DESCENDANTS` on its own hides our overlays from every service *except the one that needed it*: measured, with the vault form open, `get_screen_element` returned the whole form, labels and phone number included. `ACCESSIBILITY_DATA_SENSITIVE_YES` (API 34+, inherits down the subtree) is honoured against us because we do not declare `android:isAccessibilityTool` — and is strictly kinder than the blunt flag, since TalkBack does declare it and keeps the ball. Both are set; the old one is the pre-34 fallback. Don't casually drop `flagIncludeNotImportantViews` to "simplify" this — it is what makes unlabelled app views visible at all.-Hiding the two **focusable** cards (`SettingsUi`, `VaultUi`) then creates a second-order problem: they are the active window, and a hidden active window dumps as zero nodes — the exact signature `reviveNote()` reads as "the accessibility connection went stale, have the user toggle Andee off and on". That message is aimed at a human, so it has to be true. `OwnCard` in `ui/OverlayA11y.kt` (incremented in those two classes' `show`/`hide`; counted, because the vault opens on top of settings) lets `annotate` say the real thing instead — the user has our own card open, nothing is broken, wait or ask them to close it. Same rule as the meeting recorder's `stopped_by` and `device.look`'s grace message.+Hiding the two **focusable** cards (`SettingsUi`, `VaultUi`) then creates a second-order problem: they are the active window, and a hidden active window dumps as zero nodes — the exact signature `reviveNote()` reads as "the accessibility connection went stale, have the user toggle Andee off and on". That message is aimed at a human, so it has to be true. `OwnCard` in `ui/OverlayA11y.kt` (incremented in those two classes' `show`/`hide`; counted, because the vault opens on top of settings) lets `annotate` say the real thing instead — the user has our own card open, nothing is broken, wait or ask them to close it. Same rule as the meeting recorder's `stopped_by` and `device.look`'s grace message. The *unfolded* card produces the same zero by a different mechanism and wants a different answer — it is not focusable and hides nothing, it simply covers — see `FullscreenCard` under "Getting out of the way is the default".### Coordinate protocol: the model never does math@@ -370,7 +370,13 @@ Four things about that spread are not cosmetic:### Getting out of the way is the default-`CommandDispatcher.dispatch` folds the card into the corner ball before running **any** `screen.*` command — by prefix, so a new method there inherits it. It is not a tool anymore: `reveal_screen` is deleted, because a brain that forgot to call it left the user staring at our face instead of the app being operated, and its `seconds` deadline meant the assistant deciding how long the user gets to look. The fold has no deadline; the user taps the corner ball to talk (`dispatcher` does not expand), and long-press unfolds (`dispatcher.expand()`). **The service starts *unfolded*** — `onServiceConnected` calls `window.setCompact(false)` after `show()`, so the first thing the user sees is the full card, not a ball in the corner. It used to start folded, on the reasoning that a freshly enabled service should not freeze the tablet with a full-screen touchable window; what answers that worry is `ensureCompact()`, which folds the card before any `screen.*` command drives another app, and it was never touched. The only remaining unfold paths are therefore the service's own birth and a long-press on the ball. If you add a `screen.*` method that does *not* drive the device, keep it out of the agent-driving path.+`CommandDispatcher.dispatch` folds the card into the corner ball before running **any** `screen.*` command — by prefix, so a new method there inherits it. It is not a tool anymore: `reveal_screen` is deleted, because a brain that forgot to call it left the user staring at our face instead of the app being operated, and its `seconds` deadline meant the assistant deciding how long the user gets to look. The fold has no deadline; the user taps the corner ball to talk (`dispatcher` does not expand), and long-press unfolds (`dispatcher.expand()`). **The service starts *unfolded*** — `onServiceConnected` calls `window.setCompact(false)` after `show()`, so the first thing the user sees is the full card, not a ball in the corner. It used to start folded, on the reasoning that a freshly enabled service should not freeze the tablet with a full-screen touchable window; what answers that worry is `ensureCompact()`, which folds the card before any `screen.*` command drives another app — but that change is also what made the next paragraph reachable, so `ensureCompact()` did not survive it untouched. The only remaining unfold paths are therefore the service's own birth and a long-press on the ball. If you add a `screen.*` method that does *not* drive the device, keep it out of the agent-driving path.++**Folding the card is two events, and waiting for the wrong one reads as a dead accessibility service.** Reported as "之前能抓带微信的ui树,现在抓不到了,好像重启个什么权限就行了" — and the user believed that because our own tool hint told them so. The card is `FLAG_NOT_FOCUSABLE`, so it never takes focus and the app behind it stays `active`, `focused` and rooted; it merely *covers* the display, and every node of an occluded window answers `isVisibleToUser == false`. `ScreenController.buildNode` drops those **before** `st.total++`, so the dump is `nodes_total: 0` from a perfectly healthy screen — byte-for-byte the signature `reviveNote()` reports as a stale connection. Measured on this tablet: card up, Chrome present/`active`/`focused`/`root: true`, 0 nodes; card folded, 75 from the same screen.++The bug was that `ensureCompact()` waited on the fold *request*. `updateViewLayout` only asks; the system recomputes occlusion a frame or two later, **measured at 4 polls / ~140 ms**, against the couple of milliseconds `ensureCompact`'s two `ui.post` turns take to return (`COMPACT_SETTLE_MS` is an `await` cap, never a floor). So the dump ran ~140 ms early, repeatably. It now polls `ScreenController.activeWindowVisible()` to `UNCOVER_BUDGET_MS` (600 ms, several times the measurement because overrunning it costs an empty tree) — a poll and not a longer sleep, for the same reason the typing readback is one: the settle is tens of ms on a free main thread and much longer on a busy one, and paying the worst case on every tool call taxes all of them. It is only reached when a fold actually happened, so the usual path — a card already folded, which is most of a task — pays nothing. `ensureCompact` also asks `window.isCompact()` rather than trusting its `compacted` cache; that one is belt-and-braces, not the fix (`ensureOverlays()`'s only caller follows it with `expand()`, so no live desync path exists today).++A missed settle is survivable rather than silent: `ui.FullscreenCard` (set by `FloatingWindowUi` alone, derived from `root != null && !compact` so a card hidden while unfolded cannot keep claiming to cover) gives `annotate()` a branch ahead of the two `reviveNote` ones that names the card and says nothing is broken. It carries **no** `revive` — that note asks for a permission toggle, and asking for one when nothing is wrong is precisely how this went wrong. A dump that still reaches it is a passive observer (`driving = false` on the 9008 socket, by design) or a fold that failed, and both want naming rather than a guess about accessibility.Handlers otherwise must **not** block for the length of a timed effect — `reveal_screen` did, and a ten-second tool call reads to the brain as work still in progress, so it sat silent while the user stared at the thing the reveal was for. `screen.wait` is the one deliberate exception, because there the waiting *is* the request and returning early would answer the wrong question. It pays for that by being bounded: `WAIT_MAX_MS` is 20s against `body_hub`'s 30s request timeout, and the element dump plus screenshot that follow the sleep are inside the same request (~2.5s measured, more on a page that never settles). Anything with a duration that is *not* a wait still arms a `Handler` and returns.diff --git a/app/src/main/java/net/kuafuai/andee/net/CommandDispatcher.kt b/app/src/main/java/net/kuafuai/andee/net/CommandDispatcher.ktindex baf77aa..db0cef2 100644--- a/app/src/main/java/net/kuafuai/andee/net/CommandDispatcher.kt+++ b/app/src/main/java/net/kuafuai/andee/net/CommandDispatcher.kt@@ -1060,11 +1060,17 @@ class CommandDispatcher(}private fun ensureCompact() {- val wait = !compacted+ // The window, not the cache. [compacted] is written on the main thread+ // by everything that folds, but the card can be unfolded by paths that+ // never come back through here — birth, `ensureOverlays`, a card that+ // was hidden and re-shown — and a cache that says "already folded"+ // about a fullscreen card leaves it covering the app for the whole tool+ // call. `isCompact()` is a field read, so asking is free.+ val wait = !compacted || !window.isCompact()val settled = CountDownLatch(1)ui.post {touchBusy()- if (!compacted) {+ if (wait) {window.setCompact(true)compacted = true// Counted down a turn later rather than inline: `setCompact`@@ -1079,7 +1085,37 @@ class CommandDispatcher(}// Bounded, and only a hint: a busy main thread costs us the wait, not// the command.- if (wait) runCatching { settled.await(COMPACT_SETTLE_MS, TimeUnit.MILLISECONDS) }+ if (!wait) return+ runCatching { settled.await(COMPACT_SETTLE_MS, TimeUnit.MILLISECONDS) }+ awaitUncovered()+ }++ /**+ * Wait for the *system* to notice the fold, which is a separate event from+ * the fold itself and the one that actually matters.+ *+ * `updateViewLayout` only asks; occlusion is recomputed a frame or two+ * later. Until it is, the app behind the card still reports every node as+ * `isVisibleToUser == false` and the dump comes back with zero nodes from a+ * screen that is fine — the same signature as a stale accessibility+ * connection, which is how this shipped as "WeChat's tree cannot be read+ * any more" and sent the user off to toggle a permission that was working.+ *+ * A poll rather than a longer sleep, for the same reason the typing readback+ * is one: the settle is a few tens of ms when the main thread is free and+ * much longer when it is not, and paying the worst case on every tool call+ * is a tax on all of them. Bounded, and a miss is not fatal — the dump's own+ * hint can then name the card (see [net.kuafuai.andee.ui.FullscreenCard]).+ *+ * Only reached when a fold actually happened, so the usual path — a card+ * that was already folded, which is most of a task — pays nothing.+ */+ private fun awaitUncovered() {+ val deadline = System.currentTimeMillis() + UNCOVER_BUDGET_MS+ while (!screen.activeWindowVisible()) {+ if (System.currentTimeMillis() >= deadline) return+ runCatching { Thread.sleep(UNCOVER_POLL_MS) }+ }}/**@@ -1309,6 +1345,17 @@ class CommandDispatcher(/** How long to let the main thread take to apply the fold. */const val COMPACT_SETTLE_MS = 200L+ /**+ * And how long to then let the *system* take to notice it — see+ * [awaitUncovered]. Measured on this tablet at 4 polls / ~140 ms,+ * repeatably, against the handful of milliseconds the fold itself takes+ * to return: that gap is the whole bug. The budget is several times the+ * measurement because overrunning it costs an empty tree, and it is paid+ * once per unfold rather than per tool call.+ */+ const val UNCOVER_BUDGET_MS = 600L+ const val UNCOVER_POLL_MS = 30L+/*** Not a timeout on thinking — [setBusy] has none, on purpose. This only* catches a brain that walks away mid-task (crash, dropped socket) sodiff --git a/app/src/main/java/net/kuafuai/andee/screen/ScreenController.kt b/app/src/main/java/net/kuafuai/andee/screen/ScreenController.ktindex 63774dc..a0f1cd4 100644--- a/app/src/main/java/net/kuafuai/andee/screen/ScreenController.kt+++ b/app/src/main/java/net/kuafuai/andee/screen/ScreenController.kt@@ -1124,6 +1124,32 @@ class ScreenController(}}+ /**+ * Whether the window in front reports itself as visible to us **right+ * now** — the one question [net.kuafuai.andee.net.CommandDispatcher]'s fold+ * has to be able to wait on.+ *+ * A fullscreen overlay of ours occludes the app, and every node of an+ * occluded window answers `isVisibleToUser == false`; [buildNode] drops+ * those before it counts them, so the whole tree reads as empty. Folding+ * the card fixes it, but `updateViewLayout` only *asks* — the system+ * recomputes occlusion a frame or two later, measured here at ~140 ms after+ * the fold call has already returned. A dump that beats that recomputation+ * gets zero nodes from a healthy screen.+ *+ * False is deliberately also the answer when there is no root at all: the+ * caller polls to a deadline, and a screen mid-transition is worth the same+ * short wait as one still being uncovered.+ */+ fun activeWindowVisible(): Boolean {+ val root = runCatching { service.rootInActiveWindow }.getOrNull() ?: return false+ return try {+ runCatching { root.isVisibleToUser }.getOrDefault(false)+ } finally {+ runCatching { root.recycle() }+ }+ }+/*** Dump the active window's UI tree, with enough diagnostics that an empty* tree can say *why* it is empty.@@ -1683,16 +1709,20 @@ class ScreenController(* if the capture failed, say so instead of promising an image that is not* there.*- * The branches are not all the same verdict. Only the first three are+ * The branches are not all the same verdict. Only the first four are* genuinely empty; the last has nodes that simply have no names, and since* [rebuildIndexAndList] started rendering nesting and centres for every* node rather than sorting the labelled ones to the top, that case still* has a usable outline and must not be told to tap by coordinate.*- * [reviveNote] is appended to the first two only — the two that mean "no- * nodes AT ALL". That is the whole discrimination: a self-drawn pane still- * reports its containers, so a mini-program never reaches those branches- * and never nags the user about restarting anything.+ * [reviveNote] is appended to the two genuinely-empty branches only — the+ * two that mean "no nodes AT ALL" and have no other explanation. That is+ * the whole discrimination: a self-drawn pane still reports its containers,+ * so a mini-program never reaches those branches and never nags the user+ * about restarting anything. The two branches ahead of them are the cases+ * where a window of *ours* explains the zero, and they must not carry it —+ * that note asks for a permission toggle, and asking for one when nothing+ * is wrong is how this last went wrong.*/private fun annotate(out: JSONObject, verbose: Boolean) {if (verbose) return@@ -1722,6 +1752,21 @@ class ScreenController("whatever is behind the card.",)+ // Same zero, a different window of ours, and a different answer:+ // the big card is not focusable, so the app below is still active+ // and still has a root — it is merely *covered*, and an occluded+ // window reports every node invisible. Nothing about accessibility+ // is wrong here, so this has to come ahead of [reviveNote] too.+ total == 0 && net.kuafuai.andee.ui.FullscreenCard.covering ->+ out.put(+ "hint",+ "Andee's own card is unfolded over the whole screen, so the app behind it " ++ "is covered and reports every node as not visible — that is why this is " ++ "empty, and nothing is broken. Folding the card is automatic before any " ++ "screen tool, so retry this call once and it should read normally. Do " ++ "not tell the user to restart anything, and do not tap blindly.",+ )+out.has("error") && total == 0 ->out.put("hint",diff --git a/app/src/main/java/net/kuafuai/andee/ui/FloatingWindowUi.kt b/app/src/main/java/net/kuafuai/andee/ui/FloatingWindowUi.ktindex 2da9525..8fa5bec 100644--- a/app/src/main/java/net/kuafuai/andee/ui/FloatingWindowUi.kt+++ b/app/src/main/java/net/kuafuai/andee/ui/FloatingWindowUi.kt@@ -399,6 +399,7 @@ class FloatingWindowUi(wm.addView(v, p)root = vparams = p+ syncCover()applyCompactAppearance()// Only now can [winYOrigin] be measured, and the resting position// depends on it — the y above is a first guess that is correct only@@ -689,6 +690,7 @@ class FloatingWindowUi(root = nullparams = nullball = null+ syncCover()true} catch (_: Throwable) {false@@ -875,11 +877,25 @@ class FloatingWindowUi(ui.post {if (this.compact == compact) return@postthis.compact = compact+ syncCover()if (compact) enterCompact() else exitCompact()applyWindow()}}+ /**+ * Publish "this window is covering the display" for the two readers that+ * cannot be handed a reference to it — see [FullscreenCard].+ *+ * Derived from both facts rather than mirrored off [setCompact], because+ * [hide] leaves `compact` alone: a card torn down while unfolded would+ * otherwise keep claiming to cover a screen it is no longer on. Main thread+ * only, like everything that touches these two fields.+ */+ private fun syncCover() {+ FullscreenCard.covering = root != null && !compact+ }+/*** Whether the card is folded, read from outside.*diff --git a/app/src/main/java/net/kuafuai/andee/ui/OverlayA11y.kt b/app/src/main/java/net/kuafuai/andee/ui/OverlayA11y.ktindex 379ca1f..33fcb53 100644--- a/app/src/main/java/net/kuafuai/andee/ui/OverlayA11y.kt+++ b/app/src/main/java/net/kuafuai/andee/ui/OverlayA11y.kt@@ -105,3 +105,32 @@ object OwnCard {depth.updateAndGet { if (it > 0) it - 1 else 0 }}}++/**+ * Whether the assistant's own card is unfolded over the whole display.+ *+ * A second way to get zero nodes, by a different mechanism than [OwnCard] and+ * with a different answer. The card is `FLAG_NOT_FOCUSABLE`, so it never+ * becomes the active window and the app behind it stays focused and keeps its+ * root — but it *covers* the display, and every node of an occluded window+ * reports `isVisibleToUser == false`. [ScreenController.buildNode] drops those+ * before it counts them, so `nodes_total` is 0 with a perfectly healthy tree+ * underneath. Measured on this device: with the card unfolded, Chrome's window+ * is present, `active`, `focused`, `root: true` and yields 0 nodes; folding the+ * card returns 75 from the same screen.+ *+ * It exists because that is the same signature [ScreenController.reviveNote]+ * reads as a stale accessibility connection, and it sent a user off to toggle a+ * permission that was working fine — the dump has to be able to say the real+ * reason. Normally unreachable now that [net.kuafuai.andee.net.CommandDispatcher]+ * waits for the fold to land, so a dump that still sees this is either a passive+ * observer (`driving = false`) or the fold failing, and both want naming rather+ * than a guess about accessibility.+ *+ * Set by [FloatingWindowUi] alone, from every place that changes whether the+ * card is on screen at full size.+ */+object FullscreenCard {+ @Volatile+ var covering: Boolean = false+}