Computer use for the Pi coding agent — Windows apps, browsers and desktops through native UI trees, screenshots and real input.
TypeScript accessibilityagent-toolsai-agentbrowser-automationcomputer-usecuadesktop-automationelectronpipi-coding-agentpi-extensionpi-packageplaywrighttypescriptui-automationvisionwebview2windowswindows-automationwsl
Commits on main
A running watch reports its state; a stopped watch has no pending slots (LPT-949)
ea77822 alexshpunt-clanker[bot] committed 24 days ago
Merge pull request #80 from alexshpunt/fix/lpt-951-ref-line
6098b3b alexshpunt-clanker[bot] committed 24 days ago
Read results say what they grant for input: image points, no refs, or none for frames (LPT-951)
ded0af2 alexshpunt-clanker[bot] committed 24 days ago
Merge pull request #79 from alexshpunt/lpt-948-docs
44566fc alexshpunt-clanker[bot] committed 24 days ago
Merge pull request #78 from alexshpunt/lpt-948-tests
1a17a4f alexshpunt-clanker[bot] committed 24 days ago
Tests: native multi-clicks, unit and live (LPT-948)
2f23d09 alexshpunt-clanker[bot] committed 24 days ago
Merge pull request #77 from alexshpunt/lpt-948-code
d404043 alexshpunt-clanker[bot] committed 24 days ago
Triple clicks and right or middle double clicks on native windows (LPT-948)
e940bed alexshpunt-clanker[bot] committed 24 days ago
Merge pull request #76 from alexshpunt/feat/lpt-930-watch
c3e9d71 alexshpunt-clanker[bot] committed 25 days ago
Document watches, add the live watch check, polish stop texts (LPT-930)
befcf2d alexshpunt-clanker[bot] committed 25 days ago
Watch a window or viewport in the background while acting (LPT-930)
f74fd94 alexshpunt-clanker[bot] committed 25 days ago
Merge pull request #75 from alexshpunt/feat/lpt-931-viewports
031eed3 alexshpunt-clanker[bot] committed 25 days ago
docs: tool_search loads computer_view with its own group, computer view viewport (LPT-931)
57575c3 alexshpunt-clanker[bot] committed 25 days ago
test: integration checks use element crops for window detail and load computer_view (LPT-931)
c3ed372 alexshpunt-clanker[bot] committed 25 days ago
feat!: a window has no region view; detail of part of a window comes from a viewport; docs (LPT-931)
c977df4 alexshpunt-clanker[bot] committed 25 days ago
fix: quiet captures keep the agent's image as the space of its points; viewport OCR reads enlarged; live check of viewports (LPT-931)
d60339c alexshpunt-clanker[bot] committed 25 days ago
feat: named viewports of a window with computer_view (LPT-931)
758537c alexshpunt-clanker[bot] committed 25 days ago
Merge pull request #74 from alexshpunt/feat/lpt-928-change-overview
fc37c16 alexshpunt-clanker[bot] committed 25 days ago
test: the Paint stroke live check expects the outlined overview first (LPT-928)
d237d2a alexshpunt-clanker[bot] committed 25 days ago
feat: a navigation shows the whole page; the page start comes from the topmost web element; live checks of the change list (LPT-928)
35dabd3 alexshpunt-clanker[bot] committed 25 days ago
docs: the change list outlines numbered areas, puts the input first and summarises browser chrome (LPT-928)
8cdfde9 alexshpunt-clanker[bot] committed 25 days ago
feat: page actions show their pixel change from Playwright screenshots in CSS pixels (LPT-928)
a7e2200 alexshpunt-clanker[bot] committed 25 days ago
feat: the pixel change list outlines and numbers the changed areas on the window, puts the area at the input first, enlarges small crops and summarises browser chrome (LPT-928)
ceabe77 alexshpunt-clanker[bot] committed 25 days ago
Merge pull request #73 from alexshpunt/feat/lpt-942-real-input
dbe5fb3 alexshpunt-clanker[bot] committed 25 days ago
docs: the bundled runtime is pi-runtime-0.37.0-c920b66b (LPT-942)
3f2f235 alexshpunt-clanker[bot] committed 25 days ago
Pin runtime c920b66b: click and scroll take uia_only; live checks of the input routes (LPT-942)
0a86636 alexshpunt-clanker[bot] committed 25 days ago
test: a drag is real input in the capture-drag integration test (LPT-942)
89c8549 alexshpunt-clanker[bot] committed 25 days ago
docs: the target decides the input route; no foreground parameter (LPT-942)
102db46 alexshpunt-clanker[bot] committed 25 days ago
test: integration tests expect real input for points, keys and drags (LPT-942)
b19975b alexshpunt-clanker[bot] committed 25 days ago