Computer use for the Pi coding agent — Windows apps, browsers and desktops through native UI trees, screenshots and real input.
TypeScript accessibilityagent-toolsai-agentbrowser-automationcomputer-usecuadesktop-automationelectronpipi-coding-agentpi-extensionpi-packageplaywrighttypescriptui-automationvisionwebview2windowswindows-automationwsl
Commits on main
feat!: input by target goes through UI Automation only and falls back to the real pointer; points and keys are real input; the foreground parameter is gone (LPT-942)
da479c5 alexshpunt-clanker[bot] committed 25 days ago
Merge pull request #72 from alexshpunt/fix/lpt-941-spent-observation
c0b1e56 alexshpunt-clanker[bot] committed 25 days ago
test: live check that a point click after a refused region Read captures the window again (LPT-941)
9ec5e41 alexshpunt-clanker[bot] committed 25 days ago
fix: point input captures the window again when its observation was spent by a refused detail or the registry limit (LPT-941)
aa0ed3a alexshpunt-clanker[bot] committed 25 days ago
Merge pull request #71 from alexshpunt/feat/lpt-934-stage-5
5c04748 alexshpunt-clanker[bot] committed 25 days ago
fix: an Insert hold names computer_press with hold_ms, a pointer move computer_hover (LPT-934)
db929a8 alexshpunt-clanker[bot] committed 25 days ago
docs: a window sequence spends the refs of the earlier Read (LPT-934)
4c1c610 alexshpunt-clanker[bot] committed 25 days ago
test: the live slider check goes through computer_type replace; the live engine helper keeps semantic actions off foreground delivery (LPT-934)
010e30a alexshpunt-clanker[bot] committed 25 days ago
test: IDE provider integration test reads with IDE tools and acts with computer_* tools; docs name what reaches set_value and select (LPT-934)
ee1f7c9 alexshpunt-clanker[bot] committed 25 days ago
test: coordinate scroll integration test uses computer_scroll (LPT-934)
d431014 alexshpunt-clanker[bot] committed 25 days ago
test: window management, restore and mode integration tests use computer_window and check the Write/Insert refusal (LPT-934)
bfc7feb alexshpunt-clanker[bot] committed 25 days ago
test: extension, unified, safe and shell-desktop integration tests use the computer_* tools (LPT-934)
b77db68 alexshpunt-clanker[bot] committed 25 days ago
test: display and pointer integration tests use the computer_* tools (LPT-934)
f361574 alexshpunt-clanker[bot] committed 25 days ago
test: input, granularity, UI resource and capture-failure integration tests use the computer_* tools (LPT-934)
12a7ea3 alexshpunt-clanker[bot] committed 25 days ago
test: sequence, semantic, detail, hold, held-path and capture-drag integration tests use the computer_* tools (LPT-934)
577f7d0 alexshpunt-clanker[bot] committed 25 days ago
docs: the guide shrinks to concepts; README, setup and the skill name the computer_* tools (LPT-934)
cd7da5b alexshpunt-clanker[bot] committed 25 days ago
fix: the batch refusal names Codemode (LPT-934)
8ac18ab alexshpunt-clanker[bot] committed 25 days ago
feat!: Write and Insert no longer act on computer resources; Read only looks (LPT-934)
7ed711b alexshpunt-clanker[bot] committed 25 days ago
feat: computer_window names windows, computer_open sets WebView2 debugging (LPT-934)
da48b82 alexshpunt-clanker[bot] committed 25 days ago
Merge pull request #70 from alexshpunt/feat/lpt-934-stage-4
d0d58c1 alexshpunt-clanker[bot] committed 25 days ago
fix: chords go as hotkeys, the namespace points to the guide so tool_search ranks tools by their own text (LPT-927)
b41eef4 alexshpunt-clanker[bot] committed 25 days ago
docs, tests: typed input tools in the guide, cluster query and refusals (LPT-927)
c492bf8 alexshpunt-clanker[bot] committed 25 days ago
feat: a native Read and a DOM Read of one window keep each other's observations (LPT-927 part c)
5b128cd alexshpunt-clanker[bot] committed 25 days ago
feat: points, drags, held keys, timelines and pen on browser pages through Playwright (LPT-927 part b)
963b6bb alexshpunt-clanker[bot] committed 25 days ago
feat: typed input tools on native windows and browser elements (LPT-927 part a)
f6c72d6 alexshpunt-clanker[bot] committed 25 days ago
Merge pull request #69 from alexshpunt/feat/lpt-934-stage-3
3085bfc alexshpunt-clanker[bot] committed 25 days ago
fix: recording receipts show where the file was fetched and name the typed tools (LPT-934)
70c9b4f alexshpunt-clanker[bot] committed 25 days ago
feat: typed computer_files, computer_clipboard and computer_record (LPT-934 step 3)
cc8c69f alexshpunt-clanker[bot] committed 25 days ago
Merge pull request #68 from alexshpunt/feat/lpt-934-stage-2
8ab97bb alexshpunt-clanker[bot] committed 25 days ago
fix: back and forward settle on pages restored from the back-forward cache; live computer_browser and computer_wait test (LPT-934)
403aacb alexshpunt-clanker[bot] committed 25 days ago