Skip to content

armgit / andee

Public Imported from GitHub
0

Andee lives on your phone or Mac and uses any app like you do—no integrations, no root, any model; memory and passwords never leave your device | Andee 住进你的手机或 Mac,像你一样操作任何 App——无需集成、无需 Root、支持任意模型,记忆和密码绝不离开设备。

docs(readme): lead with what it does for a phone, and catch up on recent features

booboosui794c1e978d10
diff --git a/README.md b/README.md
index 3665e68..33ed3aa 100644
--- a/README.md
+++ b/README.md
@@ -25,14 +25,58 @@
<a href="./README.zh-CN.md"><img alt="简体中文文档" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>
</p>
-# Andee
+<h1 align="center">Andee</h1>
-**Andee is not a tool that runs on a tablet. It is an individual that lives on one.**
+<h3 align="center">Turn any Android phone into an AI phone.</h3>
-The tablet is not the point — it is what Andee wears. The same software runs on a phone, and is
-written for **any device with a body worth having**. Everything below is that sentence taken
-seriously in code: the body it uses, the character it has, the notebook it keeps, and the rules it
-holds to when nobody is watching.
+<p align="center">
+ <b>An agent that moves into your phone</b> — it reads the screen, taps through any app, talks back,<br>
+ remembers you, and learns how you like things done.<br>
+ No root. No custom ROM. No computer on a cable. <b>One APK.</b>
+</p>
+
+<!-- images/demo.gif goes here once it is recorded — see the asset list near the end. -->
+
+The phone you already own can do everything an "AI phone" is sold on. What it was missing was
+someone living in it. Andee is that someone: **a floating ball on your screen with a face, a voice
+and a pair of hands**, and an LLM behind it that operates your apps the way you do — by looking at
+the screen and touching it. No app has to integrate with anything. If you can do it with your thumb,
+it can do it.
+
+### Things you can just say
+
+| You say | What happens |
+|---|---|
+| *"Reply to Mom on WeChat: I'll be home by seven."* | It opens WeChat, finds the chat, types, sends — while you watch the ball's face work. |
+| *"Find this cheaper on Taobao and JD."* | It walks both shops itself and comes back with a comparison page. |
+| *"Remind me to call the landlord at nine tomorrow."* | The alarm is set before it says "OK". At nine, it wakes up and tells you. |
+| *"Record this meeting."* | It holds the mic for the hour, then hands you the minutes as a page. |
+| *(a photo)* *"What's wrong with this plant?"* | Photos ride along with what you type; it looks and answers. |
+| *"Practice English with me — every evening at eight."* | It proposes saving that as a **scene**, and from then on steps into it on time. |
+| *(nothing — a message just arrived)* | It reads the notification first and decides: handle it, tell you in one sentence, or stay quiet. |
+
+### Why it is not another assistant app
+
+- **Works in every app, not a list of integrations.** It reads the accessibility tree and the
+ screenshot, and aims on a labelled grid instead of doing coordinate maths — the thing general models
+ are worst at.
+- **Lives on the phone.** The brain runs on the device against any OpenAI-compatible model (DeepSeek
+ by default). No vendor backend, no account, **no telemetry at all**.
+- **Gets to know you.** A private notebook, a real scheduler, a nightly review of its own
+ conversations, and **scenes** — ways of working it proposes after doing the same kind of thing with
+ you a few times.
+- **Has a character.** Five looks, fifteen moods, and a face the model picks itself, sentence by
+ sentence.
+- **Holds lines a prompt cannot talk it out of.** A stranger's notification cannot unlock the vault or
+ grant it new permissions — that refusal is in the dispatcher, not in the prompt.
+
+---
+
+**Andee is not a tool that runs on a phone. It is an individual that lives on one.** The phone is
+what it wears — the same software runs on a tablet, and is written for **any device with a body worth
+having**, up to a robot dog it treats as a leg. Everything below is that sentence taken seriously in
+code: the body it uses, the character it has, the notebook it keeps, and the rules it holds to when
+nobody is watching.
It is one Android app, and it has no conventional UI. An `AccessibilityService` sees and touches the
screen; a `WindowManager` overlay draws the result; **its face is a floating ball** that changes
@@ -96,9 +140,9 @@ like missing credentials.** Both go in ⚙ on the device:
- **Brain** — with `brain = local` (the default), an API key for any OpenAI-compatible endpoint.
Blank means the local brain cannot run at all.
-You do not have to remember this: the **✓** button in the ball's control bar opens the self-check
-list, which says which of the two is missing, and offers the button that fixes it. The first run also
-raises that list by itself.
+You do not have to remember this: the first run is a stepped setup that walks you through both, and
+afterwards the **✓** button in the ball's control bar opens the self-check list, which says which of
+the two is missing and offers the button that fixes it.
**Use `assembleDebug`, not `assembleRelease`.** The release build type has no `signingConfig` and
there is no keystore in the repo, so `assembleRelease` emits `app-release-unsigned.apk`, which will
@@ -113,15 +157,16 @@ network you do not control.
## What it looks like
-There is no home screen and no app drawer icon that does anything useful. What exists:
+There is no home screen. What exists:
- **A floating ball.** The entire UI is `TYPE_APPLICATION_OVERLAY` layers added through
- `WindowManager` (`ui/FloatingWindowUi`). The ball is the face; subtitles, the settings card and the
- self-check page hang off it.
-- **A self-check page** (`ui/SelfCheckActivity`) — the only launcher entry, and it exists because of a
- lesson: the project deliberately shipped without a launcher icon at first, and the cost was that a
- user whose accessibility service was off held a device that did nothing and had nowhere to be told
- why.
+ `WindowManager` (`ui/FloatingWindowUi`). Unfolded, it is a full-screen card: the ball's face on
+ top, the conversation underneath, and a small control bar — **产物** (the pages it has made),
+ **情景** (its scenes), **✓** (self-check), **⚙** (settings). When it starts working another app it
+ folds itself into a ball in the corner and gets out of the way; tap to talk, long-press to unfold.
+- **The launcher icon opens two doors.** With the assistant running, it unfolds the card — exactly
+ what a long-press on the ball does. With accessibility off, it shows the self-check card instead,
+ because that is the one moment the phone has nothing else to tell you why it is doing nothing.
- **`ScreenBodyService`** (`AccessibilityService`) — the whole job. It builds the object graph and
starts both network paths.
@@ -131,63 +176,122 @@ not pages.
## Key features
-**1. A body, not a screen.**
-The device is what Andee wears. Camera as eyes, ASR plus an offline wake-word as ears, the
-accessibility service as hands, and — when one is attached — a robot dog as **a leg**. The prompt
-is explicit that it is not a pet on a remote: *"you have not gained a subordinate, you have gained
-a body part."* Bodies differ, so it checks rather than assumes.
-
-**2. The brain is detachable from the body.**
-It runs on the device itself (`brain = local`, calling any OpenAI-compatible endpoint) or lives
-somewhere else and reaches the device over a WebSocket hub. The wire between the two is a plain
-tool-schema contract, so either half can be replaced without rewriting the other. That is what
-makes "any smart hardware as a body" a fact here rather than an aspiration.
-
-**3. A character, not a theme.**
-Fifteen moods (`ui/ball/Mood.kt`) arbitrated between what it feels, what the voice pipeline is
-doing, and what work is in flight — plus an idle fidget pool that never repeats the same gesture
-twice in a row, *"because a repeat reads as a stuck animation, not a personality."* You swipe the
-ball to change who you are talking to: a look carries a `persona` (how it talks) and a `voice`
-(how it sounds), and the persona lands at the *head* of the system prompt, where a manner has to
-sit to change how the model writes.
-
-**4. Character is delivery, never a licence.**
-§11 of the prompt subordinates the persona to the ten sections above it: it may change wording,
-rhythm, and how it sounds about the task — never what it does, what it is willing to do, or what
-it tells the user is true. And when something is genuinely wrong, the mask comes off.
-
-**5. Its own memory and its own schedule.**
+### Hands — it uses your phone the way you do
+
+**1. Any app, no integrations.**
+It reads the screen as a compact element outline — one indented line per element, with an id it can
+tap — and when the app draws its own pixels (mini-programs, WebViews, games) it falls back to the
+screenshot. General-purpose models are bad at computing coordinates and good at reading them, so it
+**never does the maths**: every screenshot it sees carries a faint labelled 8×12 grid, it taps by
+*cell + which third of it*, and for tight targets it zooms in and sees a red crosshair where the tap
+would land before committing. A per-location guard refuses the fourth tap on the same spot, so a
+confused model stops and asks instead of hammering.
+
+**2. It types like a keyboard, without being one.**
+On Android 13+ it types through the accessibility service's own input channel — the same
+`commitText` path a keyboard uses, into the focused field, **while your own keyboard stays active**.
+Nothing to install, nothing to switch. Older devices fall back to ADBKeyboard.
+
+**3. Sixty-odd tools, held to honest failure.**
+Screen, input, notifications, SMS, calls, contacts, calendar, files, camera, meetings, TTS/ASR, and
+the dog. Every one checks its runtime permission *before* acting and reports `denied("sms")` rather
+than working around a refusal. Tools return what they saw, not what was hoped for.
+
+**4. It can log in as you without ever holding your password.**
+The vault (Keystore AES-GCM, on the device) stores your accounts; the brain can list them and have the
+password **typed** into the focused field, but no tool will ever return it — not even masked. You see
+every fill on the ball as it happens.
+
+### Mind — it gets to know you
+
+**5. Scenes: ways of working it learns with you.** *(new)*
+"Practice English with me", "look after my WeChat", "compare prices for me" — a scene is a goal, a
+voice, rules and a routine it steps into for a while. They are **not shipped, they are learned**:
+after the same kind of session comes up a few times, it proposes one; nothing is saved until you say
+yes. Enter one by saying so, from the 情景 panel, on a schedule, or automatically when you open an
+app. A scene's own rules decide how much it does on its own — "reply to my WeChat for me" really
+means it replies. While you are in one, a chip under the ball names it; its ✕ leaves it.
+
+**6. It reads your notifications before they interrupt you.** *(new)*
+A cheap, tool-less model call (~500 tokens, against ~22,000 for a full turn) classifies each one:
+**act** on it, **tell** you in a single spoken sentence, or **ignore** it and log why. It is biased
+toward silence, and when it cannot decide it falls back to asking you. The text came from a stranger,
+so it is fenced as data — and a notification-driven turn is **mechanically barred** from the vault
+and from creating or entering a scene, whatever the model was talked into.
+
+**7. Its own memory and its own schedule.**
`config/Notebook.kt` is what it remembers about the person, and what it promised to do later. One
test decides what gets written: *"next time I deal with them, would not knowing this make me get
it wrong?"* The notebook is **private** — the tools are `localOnly` and filtered out of the hub
payload, so habits and promises never leave the device. On top of it sits a real scheduler, under
-a hard rule: the moment it says "I'll remind you", the task must already be set and confirmed.
+a hard rule: the moment it says "I'll remind you", the task must already be set and confirmed. And it
+is yours to read: **它记得的事** lists every memory and every scheduled task, each one deletable.
-**6. It behaves the same when nobody is watching.**
+**8. It behaves the same when nobody is watching.**
After a conversation goes quiet the device sends a `(to yourself …)` turn — Andee reviewing the
conversation alone, *"nobody is watching you, nobody is waiting for you to answer."* Two rules:
-make no sound, and do not touch the device. It closes out its own promises, and ends empty-handed
-rather than inventing something to record.
+make no sound, and do not touch the device. It closes out its own promises, notes which habits are
+worth turning into a scene next time, and ends empty-handed rather than inventing something to record.
+
+### Face — a character, not a theme
+
+**9. Five looks, fifteen moods, and a face it chooses.**
+奶油 Cream, 八戒 Pigsy, 贱贱兔 Rascal, 憨憨熊 Bear and 小恶魔 Imp, drawn live in hand-written OpenGL
+ES. Fifteen moods (`ui/ball/Mood.kt`) arbitrate between what it feels, what the voice pipeline is
+doing, and what work is in flight, plus an idle fidget pool that never repeats the same gesture twice
+in a row, *"because a repeat reads as a stuck animation, not a personality."* The model now picks its
+own expression **mid-sentence** — `[happy] Found it, page three. [calm] Want me to open it?` — and the
+face changes in step with the audio. Swipe the ball to change who you are talking to: a look carries a
+`persona` (how it talks) and a `voice` (how it sounds).
+
+**10. Character is delivery, never a licence.**
+§12 of the prompt subordinates the persona to everything above it: it may change wording, rhythm,
+and how it sounds about the task — never what it does, what it is willing to do, or what it tells
+the user is true. And when something is genuinely wrong, the mask comes off.
-**7. Sixty-odd tools, held to honest failure.**
-Screen, input, notifications, SMS, calls, contacts, calendar, files, camera, TTS/ASR, and the dog.
-Every one checks its runtime permission *before* acting and reports `denied("sms")` rather than
-working around a refusal. Tools return what they saw, not what was hoped for.
+### Show and tell
-**8. A robot dog counts as a leg.**
+**11. Talk, type, or show it a photo.**
+Tap the ball, or say "Hey Andee" (fully offline wake word). Prefer typing? The typing field takes
+up to six photos — from the camera or the gallery — with or without words.
+
+**12. Answers you can keep.**
+Anything structured — a comparison, a plan, meeting minutes — comes back as a page it writes and
+puts on screen. The **产物** panel keeps up to a hundred of them to reopen, delete, or revise by
+just saying what should change. Ask it to record a meeting and it holds the microphone for the
+hour, writing a verbatim transcript to disk as it goes, then turns it into minutes.
+
+### Body
+
+**13. A body, not a screen.**
+The device is what Andee wears. Camera as eyes, ASR plus an offline wake-word as ears, the
+accessibility service as hands, and — when one is attached — a robot dog as **a leg**. The prompt
+is explicit that it is not a pet on a remote: *"you have not gained a subordinate, you have gained
+a body part."* Bodies differ, so it checks rather than assumes.
+
+**14. A robot dog counts as a leg.**
The dog is not a smart-home accessory bolted onto a phone app. It runs through a USB OTG dongle
(Arduino Nano + NRF24L01) on the same body, so a single request — *walk to the kitchen and tell me
-what is on the counter* — can move a physical object and look through a camera, in one task. Its
-own safety rules are not negotiable: the original remote must be off, the dog refuses to advance
-inside 20cm of an obstacle, and **if the dongle is unplugged mid-motion the body cannot send a
-stop** — the dog finishes its last action. That last one is a firmware limitation this app does not
-paper over. If you attach a dog, read
+what is on the counter* — can move a physical object and look through a camera, in one task. Turns
+are closed-loop on the phone's own gyroscope, and a tilt past the fall threshold stops it on the
+device without waiting for the brain. Its own safety rules are not negotiable: the original remote
+must be off, the dog refuses to advance inside 20cm of an obstacle, and **if the dongle is unplugged
+mid-motion the body cannot send a stop** — the dog finishes its last action. That last one is a
+firmware limitation this app does not paper over. If you attach a dog, read
[The robot dog](#the-robot-dog-usb-dongle) before the first move.
-**9. No Compose, no DI framework, no coroutines.**
+**15. The brain is detachable from the body.**
+It runs on the device itself (`brain = local`, the default, calling any OpenAI-compatible endpoint)
+or lives somewhere else and reaches the device over a WebSocket hub. The wire between the two is a
+plain tool-schema contract, so either half can be replaced without rewriting the other. That is what
+makes "any smart hardware as a body" a fact here rather than an aspiration.
+
+### Engineering
+
+**16. No Compose, no DI framework, no coroutines.**
Deliberately. Overlays cannot host Compose, so all UI is hand-written `View` code; there is one
service, so dependencies are constructor-injected by hand; concurrency is `Handler` plus dedicated
-executors. 34k+ lines, all first-party — no code copied in from anywhere.
+executors. 42k+ lines of Kotlin, all first-party — no code copied in from anywhere.
## Permissions: why each one, and what happens if you refuse
@@ -280,6 +384,10 @@ party has audited. This project's claims are duller and checkable, in code you c
Habits and promises are physically incapable of being sent.
- **The wake word is fully offline.** Local MFCC + DTW template matching (`app/.../wake/`). It
stores feature vectors, never audio, and it never touches the network.
+- **A message from a stranger cannot hand it permissions.** A turn started by a notification runs
+ with `untrusted = true`, and `CommandDispatcher.dispatch` refuses the whole `device.vault.` prefix
+ and `scene.save` / `scene.enter` / `scene.delete` on it — in code, not in the prompt. The prompt is
+ the first layer; this is the one that holds when the prompt is talked around.
- **The local brain needs no vendor backend.** `brain = local` against any OpenAI-compatible
endpoint is a supported mode, not a degraded one.
- **What is *not* fixed is written down.** [SECURITY.md](SECURITY.md) lists five known limitations,
@@ -534,7 +642,10 @@ The tools the brain gets:
| `dog_sequence` | Several steps in a row (half a turn → pause → forward → back), run in the background, stopping itself at the end |
| `dog_stop` | Emergency stop. The first thing to call when the user says "stop" |
| `dog_turn` | Turn a measured number of degrees using the tablet's gyroscope, and stop on the angle — the one dog command that knows whether it worked. Prefers the legs |
-| `dog_status` | Link state and what the dog is doing. It cannot read the battery — this link only sends |
+| `dog_speed` / `dog_pose` | Speed and stance. Settings, not motions: they take effect from the next move |
+| `dog_status` | Link state, what the dog is doing, and the last telemetry it sent back |
+| `dog_sense` | Refresh the dog's own readings — front distance and battery — riding home on the radio's acknowledgement. Needs the v2 firmware on both ends; without it, it says "cannot see" rather than reporting a zero |
+| `dog_calibrate` | Tell the tablet "this is standing upright", for the fall guard. Rarely needed — every move from rest does it automatically |
### Wiring and first use
@@ -611,12 +722,14 @@ that is matched against system text **can never be translated**.
app/ the Android app (Kotlin, no Compose)
src/main/java/net/kuafuai/andee/
ScreenBodyService.kt the accessibility service — read this first
- brain/ the agent loop — LocalBrain, and LocalPrompt,
- which is the character's rulebook (§§1–10 the job, §11 the persona)
- ui/ball/ the face — Mood (15 moods + idle fidgets), BallLook (looks are characters)
+ brain/ the agent loop — LocalBrain, LocalPrompt (the character's rulebook:
+ §§1–10 the job, §11 scenes, §12 the persona, §13 the active scene),
+ and NotificationTriage
+ ui/ball/ the face — Mood (15 moods + idle fidgets), BallLook (five looks, each a character)
net/ BodyWsServer, BodyWsClient, ToolSchemas (the tool definitions)
device/ screen control, permissions, vault, robot dog
- ui/ the overlay: ball, subtitles, settings, self-check
+ config/ VoiceConfig (settings), Notebook (memories, todos, scenes — encrypted)
+ ui/ the overlay: ball, subtitles, settings, self-check, scenes, pages
i18n/AppLocale.kt how the overlay reaches its strings
src/main/res/values{,-en}/strings.xml the two user-facing string tables
docs/permissions.md why it asks for what it asks for
diff --git a/README.zh-CN.md b/README.zh-CN.md
index a9f6ab7..3581d9c 100644
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@@ -28,13 +28,52 @@
<a href="./README.zh-CN.md"><img alt="简体中文文档" src="https://img.shields.io/badge/简体中文-d9d9d9"></a>
</p>
-# Andee
+<h1 align="center">Andee</h1>
-**Andee 不是一个装在设备上的工具,他是一个住在设备里的个体。**
+<h3 align="center">把任何一台 Android 手机,变成 AI 手机。</h3>
-设备不是重点 —— 设备只是他**穿着的东西**。同一套软件也跑在手机上,并且本就为"任何值得拥有身体的
-设备"而写。下面所有内容,都是这句话被认真写进代码之后的样子:他用的身体、他有的性格、他记的
-笔记本,以及没人看着的时候他给自己定的规矩。
+<p align="center">
+ <b>让 Agent 住进你的手机</b> —— 它看得懂屏幕、能替你点遍任何 App、会开口说话、记得你,<br>
+ 还会慢慢学会你喜欢的做事方式。<br>
+ 不用 root,不用刷机,不用插着电脑。<b>一个 APK 就够。</b>
+</p>
+
+<!-- 录好 images/demo.gif 之后放这里 —— 见下面「从哪里开始看」上方的素材清单。 -->
+
+你手里这台手机,本来就做得到"AI 手机"宣传的那些事 —— 缺的只是有个"人"住在里面。Andee 就是这个
+"人":屏幕上一颗**有脸、有声音、有手**的悬浮球,背后是一个大模型,像你一样用你的 App —— 看屏幕,
+然后动手点。**不需要任何 App 来对接它。你用拇指能做到的,它就能做到。**
+
+### 你可以直接这么说
+
+| 你说 | 会发生什么 |
+|---|---|
+| *「微信回我妈:我七点到家。」* | 它自己打开微信、找到聊天、打字、发送 —— 你看着球的表情在干活。 |
+| *「这个东西帮我在淘宝和京东比比价。」* | 它自己把两家店逛一遍,回来给你一页对比。 |
+| *「明天九点提醒我给房东打电话。」* | 说"好"之前闹钟就已经设好了。九点一到,它自己醒来叫你。 |
+| *「帮我把这个会录下来。」* | 它占着麦克风录完整场,结束后把会议纪要做成一页给你。 |
+| *(一张照片)*「这盆花怎么了?」 | 打字框可以带照片,它看完就回答。 |
+| *「每天晚上八点陪我练英语。」* | 它会提议把这件事存成一个**情景**,以后到点就自己进入。 |
+| *(你什么都没说 —— 只是来了条消息)* | 它先替你看一眼通知,再决定:直接处理、用一句话告诉你、还是不打扰你。 |
+
+### 它和别的助手 App 不一样在哪
+
+- **任何 App 都能用,而不是一张"已接入应用"清单。** 它读无障碍树和截图,在一张带标号的网格上
+ 瞄准,而不是自己算坐标 —— 那恰恰是通用大模型最不擅长的事。
+- **住在手机上。** 大脑就跑在设备本机,接任意 OpenAI 兼容模型(默认 DeepSeek)。没有厂商后端、
+ 不用注册账号、**没有任何遥测**。
+- **会越来越懂你。** 私人笔记本、真的定时任务、没人时自己复盘对话,还有**情景** —— 同一类事陪你
+ 做过几次之后,它会主动提议"要不要存成一个情景"。
+- **有性格。** 五个形象、十五种情绪,表情由模型自己挑,一句话说到一半都能换脸。
+- **有些底线,提示词说不动它。** 陌生人发来的一条通知,打不开保险箱、也不能替你给它新的权限 ——
+ 这道拒绝写在调度器里,不在提示词里。
+
+---
+
+**Andee 不是一个装在手机上的工具,他是一个住在手机里的个体。** 手机只是他**穿着的东西** —— 同一套
+软件也跑在平板上,并且本就为"任何值得拥有身体的设备"而写,一直到一只被他当作腿的机器狗。下面所有
+内容,都是这句话被认真写进代码之后的样子:他用的身体、他有的性格、他记的笔记本,以及没人看着的
+时候他给自己定的规矩。
它就是一个 Android 应用。没有常规界面:一个 `AccessibilityService` 负责看和碰屏幕,一个
`WindowManager` 悬浮层负责画出来 —— 他的脸就是那颗浮动的球。
@@ -57,7 +96,7 @@
| | |
|---|---|
-| **演示视频** | 一段 30~60 秒的真机录屏,正在制作中。在那之前,下面「核心能力」第 1 条和第 8 条是这件事最好的文字说明。 |
+| **演示视频** | 一段 30~60 秒的真机录屏,正在制作中。在那之前,上面「你可以直接这么说」和下面的「核心能力」是这件事最好的文字说明。 |
| **截图** | 那颗球、在手机上跑任务、以及机器狗 —— 三张静帧待补。 |
| **现在就能构建** | **[www.codeflying.net](https://www.codeflying.net)** —— **不用自己打包,也不用自己配令牌。** 描述你想要的 Andee,它交给你一个已经配好令牌的安装包。有免费额度,但额度有限:用完之后需要购买 token。 |
@@ -105,8 +144,8 @@
其实是那一栏空着。其余功能照常,只是既听不见也说不出。
- **大脑** —— `brain = local`(默认)下,任何 OpenAI 兼容接口的 API Key。空着,本机大脑根本跑不起来。
-这些不用记:球的控制条上的 **✓** 会打开自检清单,直接告诉你缺哪一个,并给出去处理的按钮。第一次
-启动也会自己把这份清单弹出来。
+这些不用记:第一次启动会有一个分步引导,带你把这两样填好;之后球的控制条上的 **✓** 会打开自检
+清单,直接告诉你缺哪一个,并给出去处理的按钮。
**打包装 `assembleDebug`,别用 `assembleRelease`。** release 没有 `signingConfig`,仓库里也没有
keystore,`assembleRelease` 产出的是 `app-release-unsigned.apk`,**装不上**。debug 包用 Android
@@ -121,46 +160,105 @@ keystore,`assembleRelease` 产出的是 `app-release-unsigned.apk`,**装不
## 核心能力
-**1. 身体,不是屏幕。**
-设备是他穿着的东西。相机是眼睛,ASR + 离线唤醒词是耳朵,无障碍服务是手;而当有狗接上时,
-那是一条**腿**。提示词说得很直接:*"你得到的不是一个下属,是一个身体部位"*,不是遥控的宠物。
-身体各不相同,所以他会**去查**,而不是假设。
+### 手 —— 它像你一样用手机
-**2. 大脑可以和身体分离。**
-大脑可以跑在设备本机(`brain = local`,接任意 OpenAI 兼容端点),也可以待在别处、通过
-WebSocket hub 连到设备。两者之间只是一份普通的工具 schema 契约 —— 任何一半都能被换掉而不用
-重写另一半。**这才是"以任意智能硬件为身体"在这里是事实而非愿景的原因。**
+**1. 任何 App,不用对接。**
+它把屏幕读成一份精简的元素提纲 —— 每个元素一行、按层级缩进、带一个能直接点的编号;遇到 App 自己
+画像素的地方(小程序、WebView、游戏)就退回看截图。通用大模型**算**坐标很差、**读**坐标很好,所以
+它**从不做这道数学题**:它看到的每张截图上都叠着一层淡淡的、带标号的 8×12 网格,点按时报的是
+「第几格 + 格子里的哪三分之一」;目标太小时它会先放大,看到一个红色十字标出"会点在这儿",确认了再
+下手。同一个位置点到第四次会被拦下,糊涂了的模型只能停下来问你,而不是一直戳。
+
+**2. 能打字,但它不是输入法。**
+Android 13 及以上,它通过无障碍服务自带的输入通道打字 —— 和键盘走同一条 `commitText` 路径,打进
+当前焦点输入框,**而你自己的输入法一直保持在用**。不用装、不用切。更老的系统退回 ADBKeyboard。
+
+**3. 六十来个工具,且被要求诚实地失败。**
+屏幕、输入、通知、短信、通话、通讯录、日历、文件、相机、会议录音、TTS/ASR,以及那只狗。每一个
+在动手**之前**先检查自己的运行时权限,拒绝就返回 `denied("sms")`,而不是绕过去偷偷做。工具返回
+它**看到**的,不是它**希望**的。
+
+**4. 能用你的账号登录,却从不拿到你的密码。**
+保险箱(设备本机 Keystore AES-GCM 加密)存着你的账号;大脑能列出它们、能让密码被**打进**当前输入框,
+但没有任何一个工具会把密码返回给它 —— 打码的也不行。每一次填入,球上都会亮给你看。
+
+### 脑 —— 它会越来越懂你
-**3. 性格,不是主题皮肤。**
-十五种情绪(`ui/ball/Mood.kt`)在"他的感受 / 语音管线在干什么 / 手头有什么活儿"之间仲裁;
-另有一组待机小动作,**从不连续重复同一个** —— *"重复看起来像动画卡住了,不像性格"*。
-滑动那颗球,你换的是"在跟谁说话":一个形象同时带着 `persona`(怎么说话)和 `voice`(听起来
-什么样),而 persona 会被放在系统提示词的**最前面** —— 语气只有放在那儿才能改变模型的写法。
+**5. 情景:它和你一起摸索出来的做事方式。**(新)
+「陪我练英语」「帮我接管微信」「帮我比价购物」—— 一个情景就是一套目标、说话方式、规则和步骤,它会
+进入其中待一阵子。情景**不是预装的,是学出来的**:同一类事做过几次之后,它会提议存一个;你说好
+之前什么都不会保存。进入方式有四种:直接说、在「情景」面板里点、到点自动进入、或者打开某个 App 时
+自动进入。自主程度由情景自己的规则决定 —— 规则里写了"微信替我回",它就真的替你回。在情景里时,
+球下面会有一个小标签写着当前情景,点 ✕ 就退出。
-**4. 性格是"表达方式",永远不是许可证。**
-提示词 §11 把人格置于前十条之下:它可以改变措辞、节奏、听起来对这件事什么感觉 —— 但永远不改变
-他做什么、愿意做什么、以及告诉用户什么是真的。而**真出了事,面具是要摘下来的**。
+**6. 通知先过它这一关,再决定要不要打扰你。**(新)
+每条通知都会先走一次很便宜、不带工具的模型调用(约 500 token,一次完整回合约 22000)来分诊:
+**处理**、用一句话**告诉**你、或者**忽略**并记下原因。它默认偏向安静;拿不定主意时,退回老办法 ——
+弹出来问你。通知内容是陌生人写的,所以会被当成数据围起来;并且由通知触发的回合**在机制上**碰不到
+保险箱、也不能新建或进入情景 —— 不管模型被说动了什么。
-**5. 有自己的记忆,也有自己的日程。**
+**7. 有自己的记忆,也有自己的日程。**
`config/Notebook.kt` 是他记下的"关于这个人"以及"答应过要做的"。写进去的门槛只有一条:
*"下次跟这个人打交道,不知道这件事会不会让我搞错?"* 这本笔记本是**私人的** —— 对应工具是
`localOnly`,在发往 hub 的载荷里被过滤掉,习惯和承诺从不离开设备。上面还有一个真的调度器,
-规矩很硬:说出"我会提醒你"的那一刻,任务必须**已经设好并确认过**。
+规矩很硬:说出"我会提醒你"的那一刻,任务必须**已经设好并确认过**。而且这本笔记你看得到:
+**「它记得的事」**里列着每一条记忆和每一个定时任务,每一条都可以删。
-**6. 没人看着的时候,他一样。**
+**8. 没人看着的时候,他一样。**
对话安静下来之后,设备会发一个 `(to yourself …)` 的回合 —— Andee 独自复盘刚才的对话:
*"没人在看你,也没人在等你回答。"* 两条规矩:**不许出声**,**不许碰设备**。他去收尾自己许下的
-承诺,并且**宁可空手结束,也不为了记而编一条**。
+承诺,记下哪些习惯下次值得提议成情景,并且**宁可空手结束,也不为了记而编一条**。
+
+### 脸 —— 性格,不是主题皮肤
+
+**9. 五个形象、十五种情绪,表情它自己挑。**
+奶油、八戒、贱贱兔、憨憨熊、小恶魔,全部用手写 OpenGL ES 实时绘制。十五种情绪
+(`ui/ball/Mood.kt`)在"他的感受 / 语音管线在干什么 / 手头有什么活儿"之间仲裁;另有一组待机小
+动作,**从不连续重复同一个** —— *"重复看起来像动画卡住了,不像性格"*。现在表情由模型**在句子中间**
+自己挑 —— `[开心] 找到了,在第三页。[平静] 要我打开吗?` —— 脸会跟着声音同步变。滑动那颗球,你换的
+是"在跟谁说话":一个形象同时带着 `persona`(怎么说话)和 `voice`(听起来什么样)。
+
+**10. 性格是"表达方式",永远不是许可证。**
+提示词 §12 把人格置于它前面的所有章节之下:它可以改变措辞、节奏、听起来对这件事什么感觉 —— 但永远
+不改变他做什么、愿意做什么、以及告诉用户什么是真的。而**真出了事,面具是要摘下来的**。
+
+### 说和看
+
+**11. 说话、打字、或者给它看张照片。**
+点一下球,或者说「嘿 Andee」(唤醒词完全离线)。想打字也行:打字框最多能带六张照片,现拍或从相册
+选,带不带文字都可以。
+
+**12. 能留下来的答案。**
+凡是有结构的东西 —— 一份对比、一个计划、一份会议纪要 —— 它会写成一页放到屏幕上。**「产物」**面板最多
+留一百页,可以重新打开、删除,或者直接说"哪里要改"让它改。让它录会,它会占着麦克风录完一整场,边录
+边把逐字稿写到本机,结束后再整理成纪要。
+
+### 身体
+
+**13. 身体,不是屏幕。**
+设备是他穿着的东西。相机是眼睛,ASR + 离线唤醒词是耳朵,无障碍服务是手;而当有狗接上时,
+那是一条**腿**。提示词说得很直接:*"你得到的不是一个下属,是一个身体部位"*,不是遥控的宠物。
+身体各不相同,所以他会**去查**,而不是假设。
+
+**14. 机器狗算一条腿。**
+狗不是绑在手机 App 上的智能家居配件。它通过同一具身体上的 USB OTG 加密狗(Arduino Nano +
+NRF24L01)接入,所以一个请求 —— *走到厨房,告诉我台面上有什么* —— 能在一个任务里既移动一个实物、
+又透过相机去看。转弯是用手机自己的陀螺仪闭环测出来的;倾斜超过摔倒阈值,设备当场让它停下,不等大脑。
+它自己的安全规矩没得商量:原装遥控器必须关掉,前方 20cm 内有障碍它就拒绝前进,而且**动作中途拔掉
+加密狗,身体就发不出停止命令** —— 狗会把最后一个动作做完。最后这条是固件限制,App 不去粉饰它。
+接狗之前,先读[机器狗](#机器狗usb-加密狗)。
+
+**15. 大脑可以和身体分离。**
+大脑可以跑在设备本机(`brain = local`,默认,接任意 OpenAI 兼容端点),也可以待在别处、通过
+WebSocket hub 连到设备。两者之间只是一份普通的工具 schema 契约 —— 任何一半都能被换掉而不用
+重写另一半。**这才是"以任意智能硬件为身体"在这里是事实而非愿景的原因。**
-**7. 六十来个工具,且被要求诚实地失败。**
-屏幕、输入、通知、短信、通话、通讯录、日历、文件、相机、TTS/ASR,以及那只狗。每一个在动手
-**之前**先检查自己的运行时权限,拒绝就返回 `denied("sms")`,而不是绕过去偷偷做。工具返回它
-**看到**的,不是它**希望**的。
+### 工程
-**8. 没有 Compose,没有 DI 框架,没有协程。**
+**16. 没有 Compose,没有 DI 框架,没有协程。**
这是有意的。悬浮窗挂不上 Compose,所以 UI 全是手写 `View`;service 只有一个,所以依赖是手工
-构造函数注入;并发是 `Handler` + 独立 `Executor`。三万四千多行,全部自研 —— 没有一行是从别处
-拷来的。
+构造函数注入;并发是 `Handler` + 独立 `Executor`。四万两千多行 Kotlin,全部自研 —— 没有一行是从
+别处拷来的。
## 权限:每一项为什么,以及你拒绝了会怎样
@@ -198,6 +296,10 @@ WebSocket hub 连到设备。两者之间只是一份普通的工具 schema 契
`localOnly` 工具从发往 hub 的载荷里剔掉了。习惯和承诺在物理上无法被发出去。
- **唤醒词全离线。** 本机 MFCC + DTW 模板匹配(`app/.../wake/`)。存的是特征向量,从不存录音,
从不联网。
+- **陌生人的一条消息,给不了它权限。** 由通知触发的回合带着 `untrusted = true`,
+ `CommandDispatcher.dispatch` 会拒绝其中整个 `device.vault.` 前缀,以及 `scene.save` /
+ `scene.enter` / `scene.delete` —— 写在代码里,不在提示词里。提示词是第一层;这一层是在提示词被
+ 绕过去之后仍然挡得住的那层。
- **本机大脑不需要任何厂商后端。** `brain = local` 接任意 OpenAI 兼容端点是受支持的正式模式,
不是降级模式。
- **没修好的,是写下来的。** [SECURITY.md](SECURITY.md) 列了五条已知限制,包括调试用 WebSocket
@@ -221,10 +323,11 @@ WebSocket hub 连到设备。两者之间只是一份普通的工具 schema 契
## 它长什么样
- **没有常规主界面。** UI 全部是 `WindowManager` 加的 `TYPE_APPLICATION_OVERLAY` 悬浮层,核心是
- 那颗球(`ui/FloatingWindowUi`)。
-- **唯一的桌面入口是 `ui/SelfCheckActivity`(自检页)。** 它的存在是个教训:以前刻意不放
- LAUNCHER 图标,代价是无障碍没开的时候,用户手里是一台什么都不做的设备,而且没有任何地方能
- 告诉他为什么。
+ 那颗球(`ui/FloatingWindowUi`)。展开时是一张全屏卡片:上面是球的脸,下面是对话记录,旁边一排小
+ 控制条 —— **产物**(它做过的页面)、**情景**、**✓**(自检)、**⚙**(设置)。它一开始操作别的 App,
+ 就自己缩成角落里的一颗球让开位置;点一下说话,长按展开。
+- **桌面图标有两扇门。** 助手在运行时,点图标就是展开卡片 —— 和长按球是同一个动作。无障碍没开时,
+ 它显示自检卡片:那是手机唯一没有别的办法告诉你"它为什么什么都不做"的时刻。
- **活儿全在 `ScreenBodyService`(`AccessibilityService`)里。** 它构建整个对象图,并同时启动
两条网络通路。
@@ -481,7 +584,11 @@ Andee」。之后**亮屏状态下**说这句话,效果和点一下球完全
| `dog_move` | 往一个方向动 N 秒,**到时自己停**;默认轮式,`gait=true` 用腿走 |
| `dog_sequence` | 多步连做(转半圈 → 停一下 → 前进 → 退回),后台跑,结束自动停 |
| `dog_stop` | 急刹。用户说「停」时第一个调它 |
-| `dog_status` | 链路状态 + 狗在动什么。查不了电量,这条链路是单向的 |
+| `dog_turn` | 用平板的陀螺仪转一个量出来的角度,到角度就停 —— 唯一一个知道自己有没有做到的狗命令。优先用腿 |
+| `dog_speed` / `dog_pose` | 速度和姿态。是设置不是动作:从下一次移动起生效 |
+| `dog_status` | 链路状态、狗在动什么、以及它最近一次传回来的遥测 |
+| `dog_sense` | 刷新狗自己的读数 —— 前方距离和电量 —— 搭在射频应答包上回来。两端都要刷 v2 固件;没有时它会说"看不到",而不是报一个 0 |
+| `dog_calibrate` | 告诉平板"现在这样就是站直了",给摔倒保护用。基本用不上 —— 每次从静止开始移动都会自动做 |
### 接线与第一次使用
@@ -560,12 +667,13 @@ adb usb # 想切回来时执行
app/ Android 应用(Kotlin,无 Compose)
src/main/java/net/kuafuai/andee/
ScreenBodyService.kt 无障碍服务 —— 第一个该读的文件
- brain/ 智能体循环 —— LocalBrain,以及 LocalPrompt,
- 后者是角色的行为准则(§1–§10 是活儿,§11 是人格)
- ui/ball/ 那张脸 —— Mood(15 种情绪 + 待机小动作)、BallLook(形象即角色)
+ brain/ 智能体循环 —— LocalBrain、LocalPrompt(角色的行为准则:
+ §1–§10 是活儿,§11 情景,§12 人格,§13 当前情景)、NotificationTriage(通知分诊)
+ ui/ball/ 那张脸 —— Mood(15 种情绪 + 待机小动作)、BallLook(五个形象,每个都是一个角色)
net/ BodyWsServer、BodyWsClient、ToolSchemas(工具定义)
device/ 屏幕控制、权限、保险箱、机器狗
- ui/ 悬浮层:球、字幕、设置、自检页
+ config/ VoiceConfig(设置)、Notebook(记忆、待办、情景 —— 加密存储)
+ ui/ 悬浮层:球、字幕、设置、自检、情景、产物
i18n/AppLocale.kt 悬浮层怎么拿到自己的字符串
src/main/res/values{,-en}/strings.xml 两张面向用户的字符串表
docs/permissions.zh-CN.md 为什么它要这些权限