Let an AI model use a real iPhone with a $7 ESP32: AirPlay eyes, USB HID hands, any OpenAI-compatible vision model. No Mac, no developer account, no jailbreak.
docs: use a clearer UI screenshot as the demo, move the setup photo to Hardware
Henry27f36ce0f5d2
diff --git a/README.md b/README.mdindex bbc24a7..52bc98e 100644--- a/README.md+++ b/README.md@@ -13,7 +13,7 @@ English · [简体中文](README.zh-CN.md)</div>-<p align="center"><img src="docs/images/demo.png" alt="Demo: an XIAO ESP32S3 plugged into an iPhone, and the web UI with the live screen, live log and step screenshots" width="100%"></p>+<p align="center"><img src="docs/images/demo.png" alt="The web UI after a real run: live phone screen, live log, plan, and every step's screenshot with the tap drawn on it" width="100%"></p>The agent sees the phone through AirPlay screen mirroring and touches it through a tiny ESP32-S3 board thatplugs into the phone's USB port as a mouse and keyboard. Any OpenAI-compatible vision model can be the brain —@@ -60,6 +60,8 @@ uses the same paths a person uses — the screen and a pointer — so the phone| **Computer** | macOS or Linux, on the same network as the phone; runs UxPlay and the server. || **Vision model** | Any OpenAI-compatible chat API that accepts images. Running a model locally is optional. |+<p align="center"><img src="docs/images/setup.png" alt="A XIAO ESP32S3 plugged into an iPhone, next to the web UI" width="100%"></p>+Use a USB-C hub with power pass-through if the phone should keep charging while the board is plugged in.## Quick startdiff --git a/README.zh-CN.md b/README.zh-CN.mdindex e39a111..9891564 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -13,7 +13,7 @@</div>-<p align="center"><img src="docs/images/demo.png" alt="演示:插在 iPhone 上的 XIAO ESP32S3,以及网页界面(实时画面、实时日志、步骤截图)" width="100%"></p>+<p align="center"><img src="docs/images/demo.png" alt="一次真实运行后的网页界面:实时画面、实时日志、计划,以及每一步的截图和点按位置" width="100%"></p>Agent 通过 AirPlay 屏幕镜像"看"手机,通过一块插在手机 USB 口上的 ESP32-S3 小板子"摸"手机:板子对手机来说就是一个USB 鼠标加键盘。大脑可以是任何兼容 OpenAI 接口的视觉模型,云端 API 或自己部署的模型都行。@@ -55,6 +55,8 @@ USB 鼠标加键盘。大脑可以是任何兼容 OpenAI 接口的视觉模型| **电脑** | macOS 或 Linux,和手机在同一个局域网;运行 UxPlay 和服务端。 || **视觉模型** | 任何支持图片输入、兼容 OpenAI 接口的对话模型。本地部署是可选的。 |+<p align="center"><img src="docs/images/setup.png" alt="插在 iPhone 上的 XIAO ESP32S3,旁边是网页界面" width="100%"></p>+如果希望插着板子时手机还能充电,用一个支持 PD 充电直通的 USB-C 扩展坞。## 快速开始diff --git a/docs/images/demo.png b/docs/images/demo.pngindex 0fe0d3e..bd1eb54 100644Binary files a/docs/images/demo.png and b/docs/images/demo.png differdiff --git a/docs/images/setup.png b/docs/images/setup.pngnew file mode 100644index 0000000..0fe0d3eBinary files /dev/null and b/docs/images/setup.png differ