Skip to content

Computer use for the Pi coding agent — Windows apps, browsers and desktops through native UI trees, screenshots and real input.

docs: tool_search loads computer_view with its own group, computer view viewport (LPT-931)

alexshpunt-clanker[bot]57575c3bc9a2
diff --git a/.pi/skills/computer-use/SKILL.md b/.pi/skills/computer-use/SKILL.md
index bbd09eb..9716914 100644
--- a/.pi/skills/computer-use/SKILL.md
+++ b/.pi/skills/computer-use/SKILL.md
@@ -5,7 +5,7 @@ description: Read and operate authorized Windows app windows, displays and brows
# Computer use
-Use **Search → Read → inspect → plan → act → check**. Browsers are ordinary app windows. Read [the resource guide](../../../docs/computer-resources.md) before using computer addresses; the same guide is available as `docs:computer`. Read only looks, Search only discovers, Delete closes and releases, and each action is a `computer_*` tool. Load the tools with `tool_search`, one group per query: `computer open window`, `computer wait browser`, `computer files clipboard record`, `computer click hover drag scroll`, `computer type press select timeline`. Without Pi Agent IDE, use `computer_read`, `computer_search` and `computer_delete` wherever this skill says Read, Search or Delete.
+Use **Search → Read → inspect → plan → act → check**. Browsers are ordinary app windows. Read [the resource guide](../../../docs/computer-resources.md) before using computer addresses; the same guide is available as `docs:computer`. Read only looks, Search only discovers, Delete closes and releases, and each action is a `computer_*` tool. Load the tools with `tool_search`, one group per query: `computer open window`, `computer wait browser`, `computer files clipboard record`, `computer click hover drag scroll`, `computer type press select timeline`, `computer view viewport`. Without Pi Agent IDE, use `computer_read`, `computer_search` and `computer_delete` wherever this skill says Read, Search or Delete.
## Observe and inspect
diff --git a/docs/computer-resources.md b/docs/computer-resources.md
index e92a894..07a241e 100644
--- a/docs/computer-resources.md
+++ b/docs/computer-resources.md
@@ -7,7 +7,7 @@ Computer use reads and operates Windows app windows, displays and browser pages
Everything else is a `computer_*` tool with typed fields, the same in both modes. Read only looks, Search only discovers, and the `computer_*` tools act. Write and Insert do not act on computer resources: they refuse and name the tool to use.
-Every `computer_*` tool is deferred: a session that never uses the computer pays nothing for them. `tool_search` loads at most 8 best matches per query, so load one group at a time: `computer open window`, `computer wait browser`, `computer files clipboard record`, `computer click hover drag scroll`, `computer type press select timeline`. Computer use turns `tool_search` on.
+Every `computer_*` tool is deferred: a session that never uses the computer pays nothing for them. `tool_search` loads at most 8 best matches per query, so load one group at a time: `computer open window`, `computer wait browser`, `computer files clipboard record`, `computer click hover drag scroll`, `computer type press select timeline`, `computer view viewport`. Computer use turns `tool_search` on.
The `/computer-mode`, `/computer-debug` and `/computer-setup` commands belong to the user. When a part is missing, the refusal names the `/computer-setup` command that installs it; ask the user to run it.
diff --git a/src/provider.ts b/src/provider.ts
index cb2bcfe..7358fef 100644
--- a/src/provider.ts
+++ b/src/provider.ts
@@ -39,7 +39,7 @@ export function computerEditRefusal(tool: "write" | "insert", input: Record<stri
let action: unknown = tool === "write" ? "type_text" : undefined;
if (tool === "insert" && typeof input.text === "string") try { action = (JSON.parse(input.text) as { action?: unknown })?.action; } catch { /* no action to map */ }
const replacement = typeof action === "string" ? REPLACED.find(([pattern]) => pattern.test(action as string))?.[1] : undefined;
- return `${tool === "write" ? "Write" : "Insert"} does not act on computer resources; nothing was sent. ${replacement ? `Use ${replacement}. ` : ""}Each operation has its own computer_* tool: tool_search loads them with "computer open window", "computer click hover drag scroll", "computer type press select timeline", "computer wait browser" or "computer files clipboard record".`;
+ return `${tool === "write" ? "Write" : "Insert"} does not act on computer resources; nothing was sent. ${replacement ? `Use ${replacement}. ` : ""}Each operation has its own computer_* tool: tool_search loads them with "computer open window", "computer click hover drag scroll", "computer type press select timeline", "computer wait browser", "computer files clipboard record" or "computer view viewport".`;
}
/** Present native resources without dumping provenance; full receipts remain in structured data. */
diff --git a/test/extension-modes.integration.test.ts b/test/extension-modes.integration.test.ts
index 5fb5218..004e1b2 100644
--- a/test/extension-modes.integration.test.ts
+++ b/test/extension-modes.integration.test.ts
@@ -38,12 +38,13 @@ test("without Pi Agent IDE, tool_search finds the deferred computer tools and th
{ id: "more", name: "tool_search", arguments: { query: "computer wait browser" } },
{ id: "display", name: "computer_display", arguments: { display: "computer:display/primary", width: 1920, height: 1080 } },
{ id: "files", name: "tool_search", arguments: { query: "computer files clipboard record" } },
+ { id: "view", name: "tool_search", arguments: { query: "computer view viewport" } },
{ id: "list", name: "computer_files", arguments: { machine: "computer:session", action: "list" } },
{ id: "sequence", name: "computer_record", arguments: { at: "computer:machine/local/display/primary", action: "sequence" } },
{ id: "click", name: "computer_click", arguments: { window: "computer:machine/local", target: "1:a" } },
{ id: "delete", name: "computer_delete", arguments: { path: "computer:session" } },
]);
- expect(getToolCallNames(result)).toEqual(["tool_search", "tool_search", "tool_search", "computer_read", "computer_read", "computer_delete", "tool_search", "tool_search", "computer_display", "tool_search", "computer_files", "computer_record", "computer_click", "computer_delete"]);
+ expect(getToolCallNames(result)).toEqual(["tool_search", "tool_search", "tool_search", "computer_read", "computer_read", "computer_delete", "tool_search", "tool_search", "computer_display", "tool_search", "tool_search", "computer_files", "computer_record", "computer_click", "computer_delete"]);
for (const tool of ["computer_click", "computer_hover", "computer_drag", "computer_scroll", "computer_type", "computer_press", "computer_select", "computer_timeline"])
expect(["find", "input", "keys"].map(id => getToolResultText(result, id)).join("\n")).toContain(`- ${tool}:`);
for (const id of ["find", "guide", "session", "close", "delete"])
@@ -51,7 +52,7 @@ test("without Pi Agent IDE, tool_search finds the deferred computer tools and th
expect(getToolResultText(result, "find")).toContain("computer_read");
// tool_search loads the best matches of each query, at most 8 by default, so the guide names one query per cluster.
// A query lists only tools it newly loads; together the guide's cluster queries load every computer tool.
- const loaded = ["find", "open", "more", "files", "input", "keys"].map(id => getToolResultText(result, id)).join("\n");
+ const loaded = ["find", "open", "more", "files", "view", "input", "keys"].map(id => getToolResultText(result, id)).join("\n");
for (const tool of ["computer_search", "computer_delete", "computer_open", "computer_window", "computer_view", "computer_display",
"computer_wait", "computer_browser", "computer_files", "computer_clipboard", "computer_record", "computer_click", "computer_hover", "computer_drag",
"computer_scroll", "computer_type", "computer_press", "computer_select", "computer_timeline"]) expect(loaded).toContain(`- ${tool}:`);
@@ -89,6 +90,7 @@ test("with Pi Agent IDE, computer use extends Read, Search and Delete, refuses W
{ id: "open", name: "tool_search", arguments: { query: "computer open window" } },
{ id: "more", name: "tool_search", arguments: { query: "computer wait browser" } },
{ id: "files", name: "tool_search", arguments: { query: "computer files clipboard record" } },
+ { id: "view", name: "tool_search", arguments: { query: "computer view viewport" } },
{ id: "input", name: "tool_search", arguments: { query: "computer click hover drag scroll" } },
{ id: "keys", name: "tool_search", arguments: { query: "computer type press select timeline" } },
{ id: "session", name: "read", arguments: { path: "computer:session" } },
@@ -101,7 +103,7 @@ test("with Pi Agent IDE, computer use extends Read, Search and Delete, refuses W
// The force option reaches computer use through the IDE's resolver, which then refuses it for a machine.
expect(getToolResultText(result, "force")).toContain("?force=true applies to window addresses only.");
// The IDE's own tools read and delete computer resources; the cluster queries load every typed tool.
- const loaded = ["find", "open", "more", "files", "input", "keys"].map(id => getToolResultText(result, id)).join("\n");
+ const loaded = ["find", "open", "more", "files", "view", "input", "keys"].map(id => getToolResultText(result, id)).join("\n");
expect(loaded).not.toContain("computer_read");
expect(loaded).not.toContain("computer_delete");
for (const tool of ["computer_open", "computer_window", "computer_view", "computer_display", "computer_wait", "computer_browser", "computer_files", "computer_clipboard", "computer_record",