Skip to content

armgit / avibe

Public Imported from GitHub
0

Make your computer a home for your AI partner. Avibe is the local-first Agent OS: Vibey, its built-in AI partner, works out of the box, and Claude Code, Codex & OpenCode run unchanged beside it — reach them from your browser, phone, or any chat app.

docs(computer-use): verify activation before retrying no-effect input (#3074)

Max5f296e8f64e3
diff --git a/skills/computer-use/SKILL.md b/skills/computer-use/SKILL.md
index 07985ee66..94d17c2d6 100644
--- a/skills/computer-use/SKILL.md
+++ b/skills/computer-use/SKILL.md
@@ -56,6 +56,26 @@ its frame, with neither usable visual content nor useful controls, is not ready
yet: observe again while the state keeps changing, and report what you observed
once it stops changing or the timeout ends.
+## No-effect recovery and viewport content
+
+- Default to background input. If a second observation shows no effect, do not
+ repeat the same foreground click immediately: first `await win.activate()`,
+ then re-observe that this exact window is on screen and focused before
+ retrying that one action. Use `delivery_mode: "foreground"` for the retry if
+ background input still has no effect. Foreground input restores the previous
+ frontmost window; `activate()` leaves the target frontmost.
+- If no on-screen window matches, or activation is unverified, reports
+ `focused=false`, or leaves the window off-screen: stop and ask the owner to
+ show the window. Do not loop `launch()` or `activate()` or invent a reopen
+ path for a hidden window.
+- Treat driver `status: unverifiable` as unknown, not proof of effect. After
+ input, compare a second observation against a stable signature (title,
+ selected row, or visible text). Allow only one retry after verified activation;
+ if the effect is still absent or unknown, stop.
+- For virtualized content, the tree and screenshot describe the current viewport,
+ not the full document. Scrolling replaces that content: observe and capture
+ what you need as you scroll; do not assume earlier rows remain in the tree.
+
## SDK cheat-sheet
```js
@@ -76,12 +96,9 @@ await avibe.notify("Draft is ready");
await avibe.confirm("Send this message to Ana?"); // throws ConfirmRefused on no
```
-- Input uses background delivery by default and leaves the owner's frontmost
- window and pointer alone. Some apps ignore background input: when an
- observation shows no effect, repeat that one action with
- `delivery_mode: "foreground"`, which fronts the window, acts, and restores the
- previous frontmost window. Use `activate()` when the window must stay in
- front across several actions; it does not restore the previous window.
+- Background delivery leaves the owner's frontmost window and pointer alone.
+ For no-effect input, follow the activation and observation procedure above
+ before retrying; foreground delivery alone is not proof of effect.
- Window selectors: a title shared by off-screen siblings resolves with
`onScreen: true`; `windowId` picks an exact id from `observe windows`;
`index` orders matches front to back.
diff --git a/tests/test_computer_use_backends.py b/tests/test_computer_use_backends.py
index 6fdd49660..27cc0222e 100644
--- a/tests/test_computer_use_backends.py
+++ b/tests/test_computer_use_backends.py
@@ -105,3 +105,47 @@ def test_computer_use_skill_states_readiness_and_restrictions_not_permissions()
for retired in ("require_bind", "Never take focus", "focus_shortcut_forbidden"):
assert retired not in body
assert "verify, `await avibe.confirm(...)`, verify\n again, then act" in body
+
+
+def test_computer_use_skill_bounds_no_effect_recovery_and_viewport_evidence() -> None:
+ """Guard no-effect recovery; readiness coverage alone misses blind retries."""
+
+ body = Path("skills/computer-use/SKILL.md").read_text(encoding="utf-8")
+ first_screen = " ".join("\n".join(body.splitlines()[:80]).split())
+ assert "## No-effect recovery and viewport content" in first_screen
+ recovery = " ".join(
+ body.split("## No-effect recovery and viewport content", 1)[1]
+ .split("\n## ", 1)[0]
+ .split()
+ )
+ assert recovery in first_screen
+ assert "Default to background input" in recovery
+
+ # A no-effect observation gates activation, exact-window verification, then retry.
+ steps = (
+ "second observation shows no effect",
+ "do not repeat the same foreground click immediately",
+ "await win.activate()",
+ "re-observe that this exact window is on screen and focused",
+ "retrying that one action",
+ )
+ positions = [recovery.index(step) for step in steps]
+ assert positions == sorted(positions)
+ assert 'Use `delivery_mode: "foreground"` for the retry if background' in recovery
+ assert "Foreground input restores the previous frontmost window" in recovery
+ assert "`activate()` leaves the target frontmost" in recovery
+
+ # Missing visibility or unverified focus requires owner recovery, not a loop.
+ stop_rule = recovery.split("- If ", 1)[1].split("- Treat ", 1)[0]
+ for condition in ("no on-screen window", "unverified", "focused=false", "off-screen"):
+ assert condition in stop_rule
+ assert "stop and ask the owner to show the window" in stop_rule
+ assert "Do not loop `launch()` or `activate()`" in stop_rule
+
+ assert "`status: unverifiable` as unknown, not proof of effect" in recovery
+ assert "compare a second observation against a stable signature" in recovery
+ assert "only one retry after verified activation" in recovery
+ assert "if the effect is still absent or unknown, stop" in recovery
+ assert "current viewport, not the full document" in recovery
+ assert "observe and capture what you need as you scroll" in recovery
+ assert "do not assume earlier rows remain in the tree" in recovery