Skip to content

Let an AI model use a real iPhone with a $7 ESP32: AirPlay eyes, USB HID hands, any OpenAI-compatible vision model. No Mac, no developer account, no jailbreak.

docs: use a clearer UI screenshot as the demo, move the setup photo to Hardware

Henry27f36ce0f5d2
diff --git a/README.md b/README.md
index bbc24a7..52bc98e 100644
--- a/README.md
+++ b/README.md
@@ -13,7 +13,7 @@ English · [简体中文](README.zh-CN.md)
</div>
-<p align="center"><img src="docs/images/demo.png" alt="Demo: an XIAO ESP32S3 plugged into an iPhone, and the web UI with the live screen, live log and step screenshots" width="100%"></p>
+<p align="center"><img src="docs/images/demo.png" alt="The web UI after a real run: live phone screen, live log, plan, and every step's screenshot with the tap drawn on it" width="100%"></p>
The agent sees the phone through AirPlay screen mirroring and touches it through a tiny ESP32-S3 board that
plugs into the phone's USB port as a mouse and keyboard. Any OpenAI-compatible vision model can be the brain —
@@ -60,6 +60,8 @@ uses the same paths a person uses — the screen and a pointer — so the phone
| **Computer** | macOS or Linux, on the same network as the phone; runs UxPlay and the server. |
| **Vision model** | Any OpenAI-compatible chat API that accepts images. Running a model locally is optional. |
+<p align="center"><img src="docs/images/setup.png" alt="A XIAO ESP32S3 plugged into an iPhone, next to the web UI" width="100%"></p>
+
Use a USB-C hub with power pass-through if the phone should keep charging while the board is plugged in.
## Quick start
diff --git a/README.zh-CN.md b/README.zh-CN.md
index e39a111..9891564 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -13,7 +13,7 @@
</div>
-<p align="center"><img src="docs/images/demo.png" alt="演示:插在 iPhone 上的 XIAO ESP32S3,以及网页界面(实时画面、实时日志、步骤截图)" width="100%"></p>
+<p align="center"><img src="docs/images/demo.png" alt="一次真实运行后的网页界面:实时画面、实时日志、计划,以及每一步的截图和点按位置" width="100%"></p>
Agent 通过 AirPlay 屏幕镜像"看"手机,通过一块插在手机 USB 口上的 ESP32-S3 小板子"摸"手机:板子对手机来说就是一个
USB 鼠标加键盘。大脑可以是任何兼容 OpenAI 接口的视觉模型,云端 API 或自己部署的模型都行。
@@ -55,6 +55,8 @@ USB 鼠标加键盘。大脑可以是任何兼容 OpenAI 接口的视觉模型
| **电脑** | macOS 或 Linux,和手机在同一个局域网;运行 UxPlay 和服务端。 |
| **视觉模型** | 任何支持图片输入、兼容 OpenAI 接口的对话模型。本地部署是可选的。 |
+<p align="center"><img src="docs/images/setup.png" alt="插在 iPhone 上的 XIAO ESP32S3,旁边是网页界面" width="100%"></p>
+
如果希望插着板子时手机还能充电,用一个支持 PD 充电直通的 USB-C 扩展坞。
## 快速开始
diff --git a/docs/images/demo.png b/docs/images/demo.png
index 0fe0d3e..bd1eb54 100644
Binary files a/docs/images/demo.png and b/docs/images/demo.png differ
diff --git a/docs/images/setup.png b/docs/images/setup.png
new file mode 100644
index 0000000..0fe0d3e
Binary files /dev/null and b/docs/images/setup.png differ